# Qwen3 235B-A22B GPU requirements: VRAM, GPUs and servers

> Qwen3 235B-A22B needs about 470 GB of VRAM in BF16, 235 GB in FP8 and 118 GB in 4-bit, plus KV cache. GPUs and servers that run it, with prices.

Source: https://overgpu.com/models/qwen3-235b-a22b  
Updated: 2026-11-03  
Publisher: OverGPU (https://overgpu.com/)

**In short:** Not on one GPU: in FP8 it needs about 272 GB, so it runs on an 8-GPU server such as the 8× NVIDIA HGX H100 ($16,900 a month at OverGPU).

## Model facts

- **Developer:** Alibaba Qwen
- **License:** Apache 2.0
- **Type:** Mixture of Experts
- **Parameters:** 235B (about 22B active per token)
- **Layers:** 94
- **Attention:** 64 query heads, 4 KV heads, head dimension 128
- **Context window:** 32k tokens native, about 128k with YaRN

## How much VRAM Qwen3 235B-A22B needs

Weights take parameters × bytes per parameter; serving adds the KV cache and about 10% runtime overhead. Totals assume an 8k context with 8 concurrent requests and a BF16 KV cache.

| Precision | Weights | Total to serve | Cheapest OverGPU server that fits |
| --- | --- | --- | --- |
| BF16 | 470 GB | 531 GB | 8× NVIDIA RTX PRO 6000 Blackwell ($10,490/month) |
| FP8 | 235 GB | 272 GB | 8× NVIDIA RTX PRO 6000 Blackwell ($10,490/month) |
| 4-bit | 118 GB | 143 GB | NVIDIA B200 ($4,590/month) |

## Which GPUs can run Qwen3 235B-A22B?

Yes = 8 concurrent requests with an 8k context each; 1 request = a single 8k request at a time; No = does not fit.

| Configuration | BF16 | FP8 | 4-bit |
| --- | --- | --- | --- |
| NVIDIA GeForce RTX 5090 (32 GB, $449/month) | No | No | No |
| NVIDIA L40S (48 GB, $890/month) | No | No | No |
| NVIDIA H100 SXM (80 GB, $2,190/month) | No | No | No |
| NVIDIA RTX PRO 6000 Blackwell (96 GB, $1,390/month) | No | No | No |
| NVIDIA H200 (141 GB, $3,090/month) | No | No | No |
| NVIDIA B200 (180 GB, $4,590/month) | No | No | Yes |
| NVIDIA B300 (288 GB, $5,190/month) | No | No | Yes |
| 8× NVIDIA HGX H100 (8 × 80 GB, $16,900/month) | Yes | Yes | Yes |
| 8× NVIDIA RTX PRO 6000 Blackwell (8 × 96 GB, $10,490/month) | Yes | Yes | Yes |
| 8× NVIDIA HGX H200 (8 × 141 GB, $23,900/month) | Yes | Yes | Yes |
| 8× NVIDIA HGX B200 (8 × 180 GB, $34,900/month) | Yes | Yes | Yes |
| 8× NVIDIA HGX B300 (8 × 288 GB, $39,900/month) | Yes | Yes | Yes |
| 8× AMD Instinct MI355X (8 × 288 GB, $19,900/month) | Yes | Yes | Yes |

## Qwen3 235B-A22B KV cache by context length

About 193 MB per 1,000 tokens in flight with a BF16 cache.

| Context per request | 1 request | 8 requests | 32 requests |
| --- | --- | --- | --- |
| 4k tokens | 789 MB | 6.3 GB | 25 GB |
| 32k tokens | 6.3 GB | 50 GB | 202 GB |
| 128k tokens | 25 GB | 202 GB | 807 GB |

## Memory to fine-tune Qwen3 235B-A22B

| Method | Approximate memory | Cheapest OverGPU server that fits |
| --- | --- | --- |
| QLoRA (4-bit base) | 177 GB | NVIDIA B300 ($5,190/month) |
| LoRA (BF16 base) | 583 GB | 8× NVIDIA RTX PRO 6000 Blackwell ($10,490/month) |
| Full fine-tune (AdamW, mixed precision) | 4,512 GB | More than 8 GPUs (GB300 NVL72) |

## About Qwen3 235B-A22B

Qwen3 235B-A22B is the flagship Mixture-of-Experts model of the Qwen3 family: 235B parameters with about 22B active per token, 94 layers and only 4 KV heads, which keeps its KV cache small for its size. Every expert must still sit in GPU memory, so the weights decide the server.

Every figure comes from the model's published architecture and the formulas of the VRAM calculator (https://overgpu.com/tools/vram-calculator).

## Frequently asked questions

### How much VRAM does Qwen3 235B-A22B need?

About 470 GB for the weights in BF16, 235 GB in FP8 and 118 GB in 4-bit, plus the KV cache: about 193 MB per 1,000 tokens in flight with a BF16 cache. Serving it in FP8 with an 8k context and 8 concurrent requests takes about 272 GB in total.

### Can Qwen3 235B-A22B run on a single GPU?

Not at an 8k context with 8 concurrent requests: in FP8 it needs about 272 GB, more than any single GPU holds. An 8-GPU server such as the 8× NVIDIA RTX PRO 6000 Blackwell runs it with tensor parallelism.

### What is the cheapest OverGPU server for Qwen3 235B-A22B?

For FP8 serving with an 8k context and 8 concurrent requests, the 8× NVIDIA RTX PRO 6000 Blackwell at $10,490 per month is the lowest-priced configuration that fits. For other contexts and concurrency, size it in the VRAM calculator.

### How much memory do I need to fine-tune Qwen3 235B-A22B?

Roughly 177 GB with QLoRA, 583 GB with LoRA on BF16 weights, and 4,512 GB for a full fine-tune with AdamW in mixed precision, before tuning batch size and sequence length. Full fine-tunes of large models are sharded across GPUs with FSDP or ZeRO-3.

## Related pages

- [LLM VRAM calculator: will your model fit on a GPU?](https://overgpu.com/tools/vram-calculator): Pick Llama, Qwen, Mistral or gpt-oss, set the precision and context length, and see weights, KV cache and which GPUs fit.
- [How much VRAM do I need for an LLM?](https://overgpu.com/blog/vram-for-llms): Quick answers for 7B, 14B, 32B, 70B and 405B models at BF16, FP8 and 4-bit, plus KV cache by context and the GPU that fits.
- [Best GPU for LLM inference in 2026: a sizing guide by model size](https://overgpu.com/blog/best-gpu-for-llm-inference): Use our 2026 sizing table, from RTX 5090 to 8× B300, to choose.
- [LLM GPU requirements](https://overgpu.com/models): VRAM, KV cache and the cheapest servers for Llama, Qwen3, Mistral and gpt-oss models.