# A100 vs H100: what changed, and why we rent the H100

> A100 vs H100: FP8 support, 3.35 vs 2.0 TB/s and NVLink 4 on the same 80 GB. We rent the H100, not the A100. See what changed and who still uses A100s.

Source: https://overgpu.com/compare/a100-vs-h100  
Updated: 2026-10-22  
Publisher: OverGPU (https://overgpu.com/)

**In short:** With the same 80 GB, the H100 adds FP8, 64% more bandwidth, about 3.2 times the vendor-listed dense BF16 compute and faster NVLink, so it is the GPU to rent in 2026, while A100s make sense mainly where you already have them.

## A100 vs H100 specifications

| Specification | NVIDIA A100 SXM 80 GB | NVIDIA H100 SXM |
| --- | --- | --- |
| Architecture | Ampere | Hopper |
| GPU memory | 80 GB HBM2e | 80 GB HBM3 |
| Memory bandwidth | 2.04 TB/s | 3.35 TB/s |
| FP4 | — | — |
| FP8 | — | 1,979 TFLOPS |
| BF16 / FP16 | 312 TFLOPS | 989 TFLOPS |
| GPU interconnect | NVLink 3 · 600 GB/s | NVLink 4 · 900 GB/s |
| Form factor | SXM4 module | SXM5 module · HGX H100 |
| Max power | 400 W | 700 W |

## Monthly prices at OverGPU

| Configuration | Type | GPU memory | CPU | RAM | Storage | Network | Price per month (USD) | Per GPU-hour | Availability |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| [NVIDIA H100 SXM](https://overgpu.com/gpus/nvidia-h100) | Single GPU | 80 GB HBM3 | 24 vCPU | 225 GB | 2 TB NVMe | 25 Gbps | $2,190 | $3.00 | In stock, delivered in under 10 minutes after payment |
| [8× NVIDIA HGX H100](https://overgpu.com/gpus/nvidia-h100) | 8-GPU server | 8 × 80 GB HBM3 | 2× Xeon 8480+ · 112 cores | 2 TB DDR5 | 8× 3.84 TB NVMe | 8× 400G ConnectX-7 | $16,900 | $2.89 | In stock, delivered in under 10 minutes after payment |

Per GPU-hour = monthly price ÷ number of GPUs ÷ 730 hours, shown only to compare with hourly offers: every plan is billed as one flat monthly price.

## Choose the A100 if…

- You already own A100s or have them under contract, and your jobs run well on them.
- Your results were validated on Ampere and must be reproduced on identical hardware.
- Your models are small and an A100 already finishes the work in the time you need.

## Choose the H100 if…

- You want FP8, for example to fit a 70B model in about 70 GB on one GPU.
- Your inference is limited by memory bandwidth, where 3.35 TB/s against 2.0 TB/s matters.
- You train on several GPUs and want NVLink 4 at 900 GB/s per GPU.
- You want to rent from OverGPU, which offers the H100 but not the A100.

## Overview

The **A100 vs H100** comparison comes up because both GPUs have 80 GB of memory, which makes them look interchangeable on paper. They are not. To be clear from the start: OverGPU does not rent the A100. We offer the H100, as a single GPU or in an 8-GPU HGX server, along with newer Hopper and Blackwell GPUs. This page explains what changed between Ampere and Hopper, so you can judge whether an A100 you already own, or can get elsewhere, is still the right tool for your work.

## A100 vs H100: the short answer

The H100 keeps the same 80 GB and improves almost everything else: FP8 support, 64% more memory bandwidth (3.35 TB/s against about 2.0 TB/s), about 3.2 times the dense BF16 throughput NVIDIA lists (989 against 312 TFLOPS) and NVLink 4 at 900 GB/s per GPU against 600 GB/s. The A100 still makes sense where you already have it and your jobs fit its limits. For a new rental in 2026, the [H100](https://overgpu.com/gpus/nvidia-h100) is the GPU we offer and the one we recommend.

## FP8: the biggest practical difference

The A100 (Ampere) has no FP8 support. Its fastest floating-point format for training is BF16 or FP16, at 312 TFLOPS dense per NVIDIA. The H100 (Hopper) introduced FP8 tensor cores and the Transformer Engine, with 1,979 TFLOPS of dense FP8 listed by NVIDIA.

FP8 helps twice. It halves weight memory compared with BF16, so a 70B model drops from about 140 GB to about 70 GB and fits on one 80 GB H100, tightly. And the math itself runs in FP8 on the tensor cores. On an A100, the same model needs INT8 or 4-bit weight-only quantization, or two GPUs. FP8 checkpoints are now common for open models, and Hopper runs them natively. Our [FP8 and FP4 explainer](https://overgpu.com/blog/fp8-fp4-precision-explained) covers the formats and their accuracy trade-offs.

Same capacity, different options. Approximate weights for common model sizes:

| Model and precision | Weights (approx.) | A100, 80 GB | H100, 80 GB |
| --- | --- | --- | --- |
| 8B in BF16 | 16 GB | Yes | Yes |
| 32B in BF16 | 64 GB | Tight | Tight |
| 32B in 8 bits | 32 GB | INT8 or weight-only formats | Native FP8 |
| 70B in 8 bits | 70 GB | INT8, tight | Native FP8, tight |
| 70B in 4 bits | 35 GB | Weight-only 4-bit | Weight-only 4-bit |

## Memory bandwidth and inference

Both cards hold 80 GB, but the H100 uses HBM3 at 3.35 TB/s while the A100 80 GB uses HBM2e at about 2.0 TB/s. When a model generates text, each new token requires reading the weights and the KV cache from memory, so at small and medium batch sizes decoding follows bandwidth more than compute. With 64% more bandwidth, the H100 reads the same model in less time at every step, which means shorter waits per token and more users per GPU.

That gives a simple way to judge value if you can rent A100 time elsewhere. On bandwidth-bound inference, an A100 is only the cheaper option if its hourly cost is below roughly 60% of an H100's (2.0 ÷ 3.35). On compute-bound BF16 training, the bar drops to roughly a third (312 ÷ 989), and lower again if your H100 job can use FP8. Real jobs rarely scale exactly with vendor ratios, so treat these thresholds as rough guides and measure your own workload.

Hopper also added features that kernel authors rely on: the Tensor Memory Accelerator for asynchronous data movement, thread block clusters and DPX instructions for dynamic-programming algorithms such as sequence alignment. Attention kernels tuned for Hopper, such as FlashAttention-3, use them and do not run on the A100.

## NVLink and multi-GPU scaling

In 8-GPU HGX systems, the A100 uses third-generation NVLink at 600 GB/s per GPU and the H100 fourth-generation NVLink at 900 GB/s. Tensor-parallel inference and training with sharded optimizer states exchange data between GPUs constantly, so 50% more link bandwidth leaves more of each step for computation. Our [8-GPU H100 servers](https://overgpu.com/products/8-gpu-servers) pair eight H100s with two Xeon 8480+ CPUs (112 cores), 2 TB of DDR5, 8× 3.84 TB NVMe and 8× 400G ConnectX-7 networking, for $16,900 a month.

On one 8-GPU server, memory sets the ceiling for full fine-tuning. With Adam at roughly 16–20 bytes per parameter before activations, 640 GB of GPU memory holds a full fine-tune of about a 30B model, on A100s or H100s alike. The H100 server simply works through each step with more compute and faster links. For serving, the same 640 GB holds a 405B model in 8 bits, about 405 GB of weights: natively in FP8 on H100s, through INT8 on A100s.

## Who might still use A100s

The A100 remains a capable GPU, and there are honest reasons to keep using one:

- **You already own or contract them.** Hardware you have paid for is cheap to keep running for jobs that fit.
- **Reproducibility.** Results validated on Ampere, for papers or regulated pipelines, may need identical hardware.
- **Small, undemanding jobs.** Classic computer vision, small language models or batch jobs that already finish in time do not need Hopper.
- **Power limits.** At 400 W against 700 W per GPU, the A100 fits facilities with tight power budgets, even though the H100 does about 1.8 times more dense BF16 work per watt on NVIDIA's figures.
- **Short, occasional jobs.** If you need a GPU for a few hours a month, renting by the hour elsewhere, A100 or not, can cost less than any monthly server; our [monthly vs hourly comparison](https://overgpu.com/compare/monthly-vs-hourly-gpu-pricing) shows the break-even.

For a new monthly rental, these reasons rarely outweigh FP8 and the bandwidth gain. If 80 GB is not enough, our [H100 vs H200 comparison](https://overgpu.com/compare/h100-vs-h200) shows what 141 GB adds; if your models are small, the [L40S vs H100 comparison](https://overgpu.com/compare/l40s-vs-h100) covers a lower-cost option at $890 a month.

## H100 cost and moving from A100

A single H100 at OverGPU costs $2,190 a month, about $3.00 per GPU-hour over a 730-hour month, with 24 vCPUs, 225 GB of RAM, 2 TB of NVMe and 25 Gbps networking. The 8-GPU HGX H100 server is $16,900, about $2.89 per GPU-hour. Moving from A100 to H100 rarely needs code changes, since CUDA, PyTorch and the main inference engines support both, but a short checklist helps:

- Use a CUDA 12 build of your framework. The H100 is compute capability 9.0, against 8.0 for the A100.
- Rebuild custom CUDA extensions for compute capability 9.0 if they were compiled only for Ampere.
- Try FP8 weights and an FP8 KV cache in your inference engine, and compare outputs on your evaluation set before switching.
- Re-tune batch sizes: with more bandwidth and compute, the best setting usually changes.

Our [PyTorch and vLLM guide](https://overgpu.com/docs/pytorch-vllm) covers setup on our images, which come with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled.

## Frequently asked questions

### Does OverGPU rent the A100?

No. We rent the H100, H200, B200, B300 and other current GPUs, but not the A100. If you are moving from A100s, the H100 is the closest step up, with the same 80 GB of memory.

### Is the H100 worth it over the A100?

For new work, yes. It adds FP8, 64% more memory bandwidth, about 3.2 times the dense BF16 throughput NVIDIA lists and faster NVLink, with the same 80 GB.

### Can the A100 run FP8 models?

Not natively, because Ampere has no FP8 tensor cores. You can convert the model to INT8 or a 4-bit weight-only format, or run it in BF16 on more GPUs.

### Do the A100 and H100 have the same memory?

Both have 80 GB in the SXM versions compared here, but the H100 uses HBM3 at 3.35 TB/s while the A100 uses HBM2e at about 2.0 TB/s. The capacity is the same; the speed at which it can be read is not.

### Will my A100 code run on an H100?

In most cases yes, with a current CUDA version and framework build. Our Ubuntu images come with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled.

## Related pages

- [Rent NVIDIA H100 SXM servers from $2,190 per month](https://overgpu.com/gpus/nvidia-h100): Rent a dedicated NVIDIA H100 SXM for $2,190/month (about $3.00 per GPU-hour at 24/7) or an 8-GPU HGX H100 for $16,900. Flat price, delivered in under 10 minutes.
- [H100 vs H200: same Hopper compute, very different memory](https://overgpu.com/compare/h100-vs-h200): Same Hopper compute, 80 GB at 3.35 TB/s vs 141 GB at 4.8 TB/s. What fits on one GPU, the KV cache math and the monthly cost, compared.
- [L40S vs H100: which GPU fits your workload and budget?](https://overgpu.com/compare/l40s-vs-h100): 48 GB GDDR6 at 864 GB/s for $890 a month against 80 GB HBM3 at 3.35 TB/s for $2,190. See which models fit and which jobs each GPU suits best.
- [H100 vs B200: is the H100 still worth renting in 2026?](https://overgpu.com/compare/h100-vs-b200): 80 vs 180 GB, 3.35 vs 8 TB/s, FP4 and NVLink 5, at $2,190 vs $4,590 a month. See the cost per job and when the H100 is still the smart buy.
- [LLM VRAM calculator: will your model fit on a GPU?](https://overgpu.com/tools/vram-calculator): Pick Llama, Qwen, Mistral or gpt-oss, set the precision and context length, and see weights, KV cache and which GPUs fit.
- [GPU servers for research and HPC, on fixed monthly budgets](https://overgpu.com/solutions/research-hpc): H100, H200 and MI355X with vendor FP64 figures, CUDA or ROCm, on fixed monthly budgets. Match the GPU to your code.