LLM GPU requirements

Qwen3 235B-A22B GPU requirements: VRAM, GPUs and servers

How much GPU memory Qwen3 235B-A22B needs in BF16, FP8 and 4-bit, how its KV cache grows with context, what fine-tuning takes, and which servers run it, with monthly prices.

Parameters: 235B 22B active
Architecture: MoE 94 layers
KV heads: 4 × 128 dim
Context: 32k tokens

Open in the VRAM calculator

Cheapest servers that run it

FP8 weights · 8k context · 8 concurrent requests · about 272 GB

8× NVIDIA RTX PRO 6000 Blackwell In stock

8 × 96 GB GDDR7 · 8-GPU server · PCIe, no NVLink

$10,490/mo
8× NVIDIA HGX H100 In stock

8 × 80 GB HBM3 · 8-GPU server

$16,900/mo
8× AMD Instinct MI355X Limited

8 × 288 GB HBM3e · 8-GPU server

$19,900/mo

Delivered in under 10 minutes after payment.

Inference memory

How much VRAM Qwen3 235B-A22B needs

Weights take parameters × bytes per parameter. Serving adds the KV cache and about 10% runtime overhead; the totals below assume an 8k context with 8 concurrent requests and a BF16 KV cache.

Qwen3 235B-A22B memory by precision and the cheapest server that fits
Precision	Weights	Total to serve	Cheapest server that fits
BF16 2 bytes per parameter	470 GB	531 GB	8× NVIDIA RTX PRO 6000 Blackwell $10,490/mo
FP8 1 byte per parameter	235 GB	272 GB	8× NVIDIA RTX PRO 6000 Blackwell $10,490/mo
4-bit 0.5 bytes per parameter	118 GB	143 GB	NVIDIA B200 $4,590/mo

GGUF quantizations for llama.cpp and Ollama

Qwen3 235B-A22B GGUF file sizes and the smallest GPU for one 8k request
Quantization	Weights	Smallest GPU, one 8k request
Q8_0 about 8.5 bits per weight	250 GB	More than one GPU
Q6_K about 6.56 bits per weight	193 GB	NVIDIA B300 288 GB · $5,190/mo
Q5_K_M about 5.69 bits per weight	167 GB	NVIDIA B300 288 GB · $5,190/mo
Q4_K_M about 4.85 bits per weight	142 GB	NVIDIA B200 180 GB · $4,590/mo

Estimates with about 8% of GPU memory kept free. Quantized weights also keep some layers in higher precision, and frameworks differ in overhead; check the free memory your engine reports at startup. A model split across GPUs with tensor parallelism runs best on NVLink servers (HGX); the 8× RTX PRO 6000 server links its GPUs over PCIe.

GPUs and servers

Which GPUs can run Qwen3 235B-A22B?

Not on one GPU: in FP8 it needs about 272 GB, so it runs on an 8-GPU server such as the 8× NVIDIA HGX H100 ($16,900 a month at OverGPU). The table checks every configuration we rent in each precision.

Whether each GPU configuration runs Qwen3 235B-A22B in BF16, FP8 and 4-bit
Configuration	BF16	FP8	4-bit
NVIDIA GeForce RTX 5090 32 GB · $449/mo	—	—	—
NVIDIA L40S 48 GB · $890/mo	—	—	—
NVIDIA H100 SXM 80 GB · $2,190/mo	—	—	—
NVIDIA RTX PRO 6000 Blackwell 96 GB · $1,390/mo	—	—	—
NVIDIA H200 141 GB · $3,090/mo	—	—	—
NVIDIA B200 180 GB · $4,590/mo	—	—	Yes
NVIDIA B300 288 GB · $5,190/mo	—	—	Yes
8× NVIDIA HGX H100 8 × 80 GB · $16,900/mo	Yes	Yes	Yes
8× NVIDIA RTX PRO 6000 Blackwell 8 × 96 GB · $10,490/mo	Yes	Yes	Yes
8× NVIDIA HGX H200 8 × 141 GB · $23,900/mo	Yes	Yes	Yes
8× NVIDIA HGX B200 8 × 180 GB · $34,900/mo	Yes	Yes	Yes
8× NVIDIA HGX B300 8 × 288 GB · $39,900/mo	Yes	Yes	Yes
8× AMD Instinct MI355X 8 × 288 GB · $19,900/mo	Yes	Yes	Yes

Yes: 8 concurrent requests with an 8k context each. 1 req: a single 8k request at a time. —: the weights, KV cache and runtime do not fit. Same formulas as the tables above, with about 8% of GPU memory kept free.

KV cache

Qwen3 235B-A22B KV cache by context length

Each token in flight stores keys and values for every layer: 2 × 94 layers × 4 KV heads × 128 dimensions × 2 bytes = 188 KB per token in BF16. An FP8 KV cache halves these figures.

KV cache size by context length and number of concurrent requests
Context per request	1 request	8 requests	32 requests
4k tokens	789 MB	6.3 GB	25 GB
32k tokens	6.3 GB	50 GB	202 GB
128k tokens	25 GB	202 GB	807 GB

Fine-tuning

Memory to fine-tune Qwen3 235B-A22B

Rough totals before tuning batch size and sequence length: QLoRA keeps 4-bit base weights, LoRA keeps BF16 weights frozen, and a full fine-tune with AdamW needs about 16 bytes per parameter plus activations, sharded across GPUs.

Fine-tuning memory by method and the cheapest server that fits
Method	Approximate memory	Cheapest server that fits
QLoRA (4-bit base)	177 GB	NVIDIA B300 $5,190/mo
LoRA (BF16 base)	583 GB	8× NVIDIA RTX PRO 6000 Blackwell $10,490/mo
Full fine-tune (AdamW, mixed precision)	4,512 GB	More than 8 GPUs: GB300 NVL72

Full fine-tuning assumes sharded training (FSDP or ZeRO-3) with activation checkpointing. See the fine-tuning guide for methods and settings.

About Qwen3 235B-A22B

Qwen3 235B-A22B is the flagship Mixture-of-Experts model of the Qwen3 family: 235B parameters with about 22B active per token, 94 layers and only 4 KV heads, which keeps its KV cache small for its size. Every expert must still sit in GPU memory, so the weights decide the server.

Every figure on this page comes from the model’s published architecture and the same formulas as our VRAM calculator, where you can change the context length, concurrency and KV cache precision.

Model facts

Developer: Alibaba Qwen
License: Apache 2.0
Type: Mixture of Experts
Parameters: 235Babout 22B active per token
Layers: 94
Attention: 64 query heads, 4 KV headsgrouped-query attention, head dimension 128
Context window: 32k tokens native, about 128k with YaRN

FAQ

Qwen3 235B-A22B GPU FAQ

The short version. The help center has the details, and customers with an active server get answers by ticket, 24/7.

Visit the help center

How much VRAM does Qwen3 235B-A22B need?

About 470 GB for the weights in BF16, 235 GB in FP8 and 118 GB in 4-bit, plus the KV cache: about 193 MB per 1,000 tokens in flight with a BF16 cache. Serving it in FP8 with an 8k context and 8 concurrent requests takes about 272 GB in total.

Can Qwen3 235B-A22B run on a single GPU?

Not at an 8k context with 8 concurrent requests: in FP8 it needs about 272 GB, more than any single GPU holds. An 8-GPU server such as the 8× NVIDIA RTX PRO 6000 Blackwell runs it with tensor parallelism.

What is the cheapest OverGPU server for Qwen3 235B-A22B?

For FP8 serving with an 8k context and 8 concurrent requests, the 8× NVIDIA RTX PRO 6000 Blackwell at $10,490 per month is the lowest-priced configuration that fits. For other contexts and concurrency, size it in the VRAM calculator.

How much memory do I need to fine-tune Qwen3 235B-A22B?

Roughly 177 GB with QLoRA, 583 GB with LoRA on BF16 weights, and 4,512 GB for a full fine-tune with AdamW in mixed precision, before tuning batch size and sequence length. Full fine-tunes of large models are sharded across GPUs with FSDP or ZeRO-3.

Deploy a GPU server

Run Qwen3 235B-A22B on your own GPUs.

Choose a configuration, create your account in the same form and pay the first month from your balance. Your server is delivered in under 10 minutes.

Delivered in under 10 minutes
Cancel any month
Pay with crypto, up to +50% bonus

Configure 8× RTX PRO 6000 See all prices