# AI GPU comparison chart (2026)

> Data-center and workstation GPUs side by side: memory, bandwidth, FP8 and FP4, interconnect, power and monthly price.

Source: https://overgpu.com/compare  
Updated: 2026-10-23  
Publisher: OverGPU (https://overgpu.com/)

## AI GPU comparison chart

Vendor-published figures; compute is dense (without sparsity) where vendors publish it. Also available as CSV: https://overgpu.com/data/gpu-specs.csv

| GPU | Architecture | Memory | Bandwidth | FP8 | FP4 | Interconnect | Max power | OverGPU price |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| NVIDIA B300 | Blackwell Ultra | 288 GB HBM3e | 8 TB/s | 4,500 TFLOPS | 13,500 TFLOPS | NVLink 5 | 1,100 W | 1 GPU: $5,190/month; 8 GPUs: $39,900/month |
| NVIDIA B200 | Blackwell | 180 GB HBM3e | 8 TB/s | 4,500 TFLOPS | 9,000 TFLOPS | NVLink 5 | 1,000 W | 1 GPU: $4,590/month; 8 GPUs: $34,900/month |
| NVIDIA H200 | Hopper | 141 GB HBM3e | 4.8 TB/s | 1,979 TFLOPS | — | NVLink 4 | 700 W | 1 GPU: $3,090/month; 8 GPUs: $23,900/month |
| NVIDIA H100 SXM | Hopper | 80 GB HBM3 | 3.35 TB/s | 1,979 TFLOPS | — | NVLink 4 | 700 W | 1 GPU: $2,190/month; 8 GPUs: $16,900/month |
| AMD Instinct MI355X | CDNA 4 | 288 GB HBM3e | 8 TB/s | 5,000 TFLOPS | 10,100 TFLOPS | Infinity Fabric | 1,400 W | 8 GPUs: $19,900/month |
| NVIDIA RTX PRO 6000 Blackwell | Blackwell | 96 GB GDDR7 ECC | 1.6 TB/s | Supported | 4,000 AI TOPS | PCIe Gen5 x16 | 600 W | 1 GPU: $1,390/month; 8 GPUs: $10,490/month |
| NVIDIA L40S | Ada Lovelace | 48 GB GDDR6 ECC | 864 GB/s | 733 TFLOPS | — | PCIe Gen4 x16 | 350 W | 1 GPU: $890/month |
| NVIDIA GeForce RTX 5090 | Blackwell | 32 GB GDDR7 | 1.79 TB/s | Supported | 3,352 AI TOPS | PCIe Gen5 x16 | 575 W | 1 GPU: $449/month |
| NVIDIA GB300 NVL72 | Blackwell Ultra | 288 GB HBM3e | 8 TB/s | 5,000 TFLOPS | 15,000 TFLOPS | NVLink 5 | Rack-level | From $7,990 per GPU-month |
| NVIDIA Vera Rubin NVL72 | Rubin | 288 GB HBM4 | 19.2 TB/s | 17,500 TFLOPS | 35,000 TFLOPS | NVLink 6 | Rack-level | On request |

## Detailed comparisons

- [A100 vs H100: what changed, and why we rent the H100](https://overgpu.com/compare/a100-vs-h100): A100 vs H100: FP8 support, 3.35 vs 2.0 TB/s and NVLink 4 on the same 80 GB. We rent the H100, not the A100. See what changed and who still uses A100s.
- [B200 vs B300: is Blackwell Ultra worth the upgrade?](https://overgpu.com/compare/b200-vs-b300): B200 vs B300 compared: 180 vs 288 GB of HBM3e, the FP4 uplift and 400G vs 800G networking. See what fits, the cost per GB and when the B200 is enough.
- [H100 vs B200: is the H100 still worth renting in 2026?](https://overgpu.com/compare/h100-vs-b200): H100 vs B200: 80 vs 180 GB, 3.35 vs 8 TB/s, FP4 and NVLink 5, at $2,190 vs $4,590 a month. See the cost per job and when the H100 is still the smart buy.
- [H100 vs H200: same Hopper compute, very different memory](https://overgpu.com/compare/h100-vs-h200): H100 vs H200: same Hopper compute, 80 GB at 3.35 TB/s vs 141 GB at 4.8 TB/s. What fits on one GPU, the KV cache math and the monthly cost, compared.
- [H200 vs B200: Hopper or Blackwell for your next GPU server?](https://overgpu.com/compare/h200-vs-b200): H200 vs B200 compared: 141 vs 180 GB, 4.8 vs 8 TB/s, FP4 and NVLink 5. See what fits, what $1,500 more a month buys and when the H200 is the better buy.
- [L40S vs H100: which GPU fits your workload and budget?](https://overgpu.com/compare/l40s-vs-h100): L40S vs H100: 48 GB GDDR6 at 864 GB/s for $890 a month against 80 GB HBM3 at 3.35 TB/s for $2,190. See which models fit and which jobs each GPU suits best.
- [MI355X vs B200: memory, software and cost compared](https://overgpu.com/compare/mi355x-vs-b200): MI355X vs B200: 288 GB vs 180 GB of HBM, ROCm vs CUDA, 4-bit formats and monthly cost per GPU. See which 8-GPU server fits your models and budget.
- [MI355X vs H200: double the memory or the proven stack?](https://overgpu.com/compare/mi355x-vs-h200): MI355X vs H200: twice the memory per GPU and native FP4 against the mature Hopper and CUDA stack. Compare what fits, interconnect and monthly cost.
- [Monthly vs hourly GPU pricing: how to find your break-even](https://overgpu.com/compare/monthly-vs-hourly-gpu-pricing): Monthly vs hourly GPU pricing: the break-even formula, a utilization table with our prices, the extras hourly bills add and when hourly is the better deal.
- [RTX 5090 vs RTX 4090 for AI: what 8 GB more memory buys you](https://overgpu.com/compare/rtx-4090-vs-rtx-5090): RTX 5090 vs RTX 4090 for AI: 32 GB vs 24 GB, 1.79 TB/s vs 1,008 GB/s and FP4. Which LLMs and image models fit, and what renting a 5090 costs per month.
- [RTX 5090 vs RTX PRO 6000: when is 32 GB enough?](https://overgpu.com/compare/rtx-5090-vs-rtx-pro-6000): RTX 5090 vs RTX PRO 6000: a 32 GB card at $449 a month or a 96 GB pro card with ECC and MIG at $1,390. See exactly when 32 GB is enough for your work.
- [RTX PRO 6000 vs H100: more memory or more bandwidth?](https://overgpu.com/compare/rtx-pro-6000-vs-h100): RTX PRO 6000 vs H100: 96 GB of GDDR7 for $1,390 a month or 80 GB of HBM3 with NVLink for $2,190. See which suits 70B inference or multi-GPU training.
- [RTX PRO 6000 vs L40S: is 96 GB worth the step up?](https://overgpu.com/compare/rtx-pro-6000-vs-l40s): RTX PRO 6000 vs L40S for rendering, generative media and inference: 96 GB vs 48 GB, FP4 vs FP8, $1,390 vs $890 a month. See which card your work needs.

## How to read this AI GPU comparison chart

- **Memory:** Capacity per GPU decides which models, contexts and batch sizes fit at all.
- **Bandwidth:** Memory bandwidth sets how fast a GPU generates tokens once a model fits, because decoding reads the weights for every token.
- **FP8 and FP4:** Dense tensor throughput, as the vendor publishes it, drives training and prompt processing. FP4 is native on Blackwell GPUs and on the MI355X.
- **Interconnect:** NVLink or Infinity Fabric lets the GPUs of one server work as one memory pool for tensor parallelism.
- **Price:** Our flat monthly price per server, with the per-GPU-hour equivalent at 730 hours a month in each comparison.

## Which GPU for which workload

| Workload | Good fit | Why |
| --- | --- | --- |
| Image generation and models up to about 14B | [RTX 5090](https://overgpu.com/gpus/nvidia-rtx-5090), [L40S](https://overgpu.com/gpus/nvidia-l40s) | 32 to 48 GB at the lowest monthly prices |
| Serving 30B to 70B models | [RTX PRO 6000](https://overgpu.com/gpus/nvidia-rtx-pro-6000), [H200](https://overgpu.com/gpus/nvidia-h200) | 96 to 141 GB on one GPU |
| Fine-tuning and training up to 70B | [HGX H100](https://overgpu.com/gpus/nvidia-h100), [HGX H200](https://overgpu.com/gpus/nvidia-h200) | Eight GPUs on NVLink 4 and a mature software stack |
| The largest models, FP4 and big batches | [B200](https://overgpu.com/gpus/nvidia-b200), [B300](https://overgpu.com/gpus/nvidia-b300), [MI355X](https://overgpu.com/gpus/amd-mi355x) | 180 to 288 GB per GPU and native FP4 |
| More than eight GPUs in one NVLink domain | [GB300 NVL72](https://overgpu.com/gpus/nvidia-gb300-nvl72) | 72 GPUs in one rack |

To check a specific model, the [LLM GPU requirements](https://overgpu.com/models) pages list every configuration that runs it, and the [VRAM calculator](https://overgpu.com/tools/vram-calculator) sizes any other.

## How this chart is made

Specifications are the figures NVIDIA and AMD publish for each GPU: memory capacity and bandwidth, dense FP8 and FP4 throughput without sparsity, interconnect and maximum power. Prices are our current monthly prices, taken from the same catalog as the order page. Each detailed comparison adds what fits in memory, the price per GPU-hour and when to choose each GPU. Terms such as HBM3e, NVLink and FP4 are explained in the [GPU glossary](https://overgpu.com/glossary).

## GPU comparison FAQ

### Which GPU is best for LLM inference?

The one whose memory holds your model, its KV cache and some headroom at the lowest price. For 70B models in FP8, an H200 serves 8 concurrent 8k requests on one GPU; smaller models fit an RTX 5090, L40S or RTX PRO 6000.

### What is the difference between the H100 and the H200?

Both are Hopper GPUs with the same published compute. The H200 has 141 GB of HBM3e at 4.8 TB/s, against 80 GB of HBM3 at 3.35 TB/s on the H100, so it holds larger models and longer contexts.

### Is the B300 better than the B200?

The B300 (Blackwell Ultra) has 288 GB of HBM3e against 180 GB and a higher dense FP4 figure in NVIDIA's published specifications, and HGX B300 servers use 800G networking. If your model fits in 180 GB, the B200 costs less per month.

### Should I compare GPUs by price per hour?

Compare by what your work needs first: memory, then bandwidth or compute. Among the GPUs that fit, compare the monthly price at your real usage; the GPU cost calculator converts any hourly quote.