# gpt-oss-120b GPU requirements: VRAM, GPUs and servers

> gpt-oss-120b ships in MXFP4: about 59 GB of weights, so it runs on one 80 GB GPU. KV cache by context, fine-tuning memory and servers, with prices.

Source: https://overgpu.com/models/gpt-oss-120b  
Updated: 2026-11-03  
Publisher: OverGPU (https://overgpu.com/)

**In short:** Yes. In MXFP4 (native 4-bit), a single NVIDIA H100 SXM (80 GB, $2,190 a month at OverGPU) runs gpt-oss-120b with an 8k context and 8 concurrent requests.

## Model facts

- **Developer:** OpenAI
- **License:** Apache 2.0
- **Type:** Mixture of Experts
- **Parameters:** 117B (about 5.1B active per token)
- **Layers:** 36
- **Attention:** 64 query heads, 8 KV heads, head dimension 64
- **Context window:** 128k tokens

## How much VRAM gpt-oss-120b needs

Weights take parameters × bytes per parameter; serving adds the KV cache and about 10% runtime overhead. Totals assume an 8k context with 8 concurrent requests and a BF16 KV cache.

| Precision | Weights | Total to serve | Cheapest OverGPU server that fits |
| --- | --- | --- | --- |
| BF16 | 234 GB | 261 GB | 8× NVIDIA RTX PRO 6000 Blackwell ($10,490/month) |
| FP8 | 117 GB | 132 GB | NVIDIA B200 ($4,590/month) |
| MXFP4 (native 4-bit) | 59 GB | 68 GB | NVIDIA RTX PRO 6000 Blackwell ($1,390/month) |

## Which GPUs can run gpt-oss-120b?

Yes = 8 concurrent requests with an 8k context each; 1 request = a single 8k request at a time; No = does not fit.

| Configuration | BF16 | FP8 | MXFP4 |
| --- | --- | --- | --- |
| NVIDIA GeForce RTX 5090 (32 GB, $449/month) | No | No | No |
| NVIDIA L40S (48 GB, $890/month) | No | No | No |
| NVIDIA H100 SXM (80 GB, $2,190/month) | No | No | Yes |
| NVIDIA RTX PRO 6000 Blackwell (96 GB, $1,390/month) | No | No | Yes |
| NVIDIA H200 (141 GB, $3,090/month) | No | No | Yes |
| NVIDIA B200 (180 GB, $4,590/month) | No | Yes | Yes |
| NVIDIA B300 (288 GB, $5,190/month) | No | Yes | Yes |
| 8× NVIDIA HGX H100 (8 × 80 GB, $16,900/month) | Yes | Yes | Yes |
| 8× NVIDIA RTX PRO 6000 Blackwell (8 × 96 GB, $10,490/month) | Yes | Yes | Yes |
| 8× NVIDIA HGX H200 (8 × 141 GB, $23,900/month) | Yes | Yes | Yes |
| 8× NVIDIA HGX B200 (8 × 180 GB, $34,900/month) | Yes | Yes | Yes |
| 8× NVIDIA HGX B300 (8 × 288 GB, $39,900/month) | Yes | Yes | Yes |
| 8× AMD Instinct MI355X (8 × 288 GB, $19,900/month) | Yes | Yes | Yes |

## gpt-oss-120b KV cache by context length

About 37 MB per 1,000 tokens in flight with a BF16 cache.

| Context per request | 1 request | 8 requests | 32 requests |
| --- | --- | --- | --- |
| 4k tokens | 151 MB | 1.2 GB | 4.8 GB |
| 32k tokens | 1.2 GB | 9.7 GB | 39 GB |
| 128k tokens | 4.8 GB | 39 GB | 155 GB |

## Memory to fine-tune gpt-oss-120b

| Method | Approximate memory | Cheapest OverGPU server that fits |
| --- | --- | --- |
| QLoRA (4-bit base) | 88 GB | NVIDIA H200 ($3,090/month) |
| LoRA (BF16 base) | 290 GB | 8× NVIDIA RTX PRO 6000 Blackwell ($10,490/month) |
| Full fine-tune (AdamW, mixed precision) | 2,246 GB | More than 8 GPUs (GB300 NVL72) |

## About gpt-oss-120b

gpt-oss-120b is OpenAI’s open-weight Mixture-of-Experts model: about 117B parameters with roughly 5.1B active per token, 36 layers that alternate full attention and a 128-token sliding window, and Apache 2.0 weights. OpenAI releases the expert weights in MXFP4, a 4-bit format, and states that the model runs on a single 80 GB GPU.

Every figure comes from the model's published architecture and the formulas of the VRAM calculator (https://overgpu.com/tools/vram-calculator).

## Frequently asked questions

### How much VRAM does gpt-oss-120b need?

About 234 GB for the weights in BF16, 117 GB in FP8 and 59 GB in 4-bit, plus the KV cache: about 37 MB per 1,000 tokens in flight with a BF16 cache. Serving it in MXFP4 (native 4-bit) with an 8k context and 8 concurrent requests takes about 68 GB in total.

### Can gpt-oss-120b run on a single GPU?

Yes. In MXFP4 (native 4-bit), a single NVIDIA RTX PRO 6000 Blackwell (96 GB) serves it with an 8k context and 8 concurrent requests. Longer contexts or more users need more memory for the KV cache, so check your case in the VRAM calculator.

### What is the cheapest OverGPU server for gpt-oss-120b?

For MXFP4 (native 4-bit) serving with an 8k context and 8 concurrent requests, the NVIDIA RTX PRO 6000 Blackwell at $1,390 per month is the lowest-priced configuration that fits. For other contexts and concurrency, size it in the VRAM calculator.

### How much memory do I need to fine-tune gpt-oss-120b?

Roughly 88 GB with QLoRA, 290 GB with LoRA on BF16 weights, and 2,246 GB for a full fine-tune with AdamW in mixed precision, before tuning batch size and sequence length. Full fine-tunes of large models are sharded across GPUs with FSDP or ZeRO-3.

## Related pages

- [LLM VRAM calculator: will your model fit on a GPU?](https://overgpu.com/tools/vram-calculator): Pick Llama, Qwen, Mistral or gpt-oss, set the precision and context length, and see weights, KV cache and which GPUs fit.
- [How much VRAM do I need for an LLM?](https://overgpu.com/blog/vram-for-llms): Quick answers for 7B, 14B, 32B, 70B and 405B models at BF16, FP8 and 4-bit, plus KV cache by context and the GPU that fits.
- [Best GPU for LLM inference in 2026: a sizing guide by model size](https://overgpu.com/blog/best-gpu-for-llm-inference): Use our 2026 sizing table, from RTX 5090 to 8× B300, to choose.
- [LLM GPU requirements](https://overgpu.com/models): VRAM, KV cache and the cheapest servers for Llama, Qwen3, Mistral and gpt-oss models.