# 8-GPU server rental: bare-metal NVIDIA HGX and AMD Instinct

> Bare-metal 8-GPU server rental: NVIDIA HGX B300, B200, H200, H100, AMD MI355X or 8x RTX PRO 6000. 50 TB bandwidth, monthly terms, delivered in under 10 minutes.

Source: https://overgpu.com/products/8-gpu-servers  
Updated: 2026-10-23  
Publisher: OverGPU (https://overgpu.com/)

## Key points

- **Bare metal, no hypervisor:** Both CPUs, all system memory, every NVMe drive and all eight GPUs are yours, with root access over SSH.
- **NVLink on every HGX server:** NVIDIA lists 1.8 TB/s per GPU over NVLink 5 on B300 and B200, and 900 GB/s over NVLink 4 on H200 and H100.
- **Up to 2,304 GB of GPU memory:** 8 × 288 GB on HGX B300 and MI355X, 1,440 GB on HGX B200, 1,128 GB on HGX H200 and 640 GB on HGX H100.
- **Ready to scale out:** Eight 800G ConnectX-8 adapters on HGX B300, eight 400G ConnectX-7 on HGX B200, H200 and H100, and eight 400G RoCE on MI355X.
- **50 TB of bandwidth included:** 50 TB of outbound traffic per month under fair use with no egress fees, plus public IPv4, an IPv6 /64 and DDoS mitigation.
- **Monthly terms:** A flat monthly price, no setup fee, cancel any month. Term and volume pricing by ticket once you have an active server.

## Recommended configurations

- [8× NVIDIA HGX B300](https://overgpu.com/gpus/nvidia-b300): $39,900 per month — 2,304 GB of HBM3e for the largest models and longest contexts
- [8× NVIDIA HGX B200](https://overgpu.com/gpus/nvidia-b200): $34,900 per month — 1,440 GB of HBM3e on Blackwell for training and serving
- [8× NVIDIA HGX H200](https://overgpu.com/gpus/nvidia-h200): $23,900 per month — 1,128 GB of HBM3e: a 405B model in BF16 on one server

## Overview

8-GPU server rental at OverGPU gives you an entire physical machine: eight whole GPUs, two server CPUs, 1.5 to 4 TB of system memory and local NVMe, with no hypervisor and no other tenant. You can choose NVIDIA HGX B300, B200, H200 or H100, AMD Instinct MI355X, or a server with eight RTX PRO 6000 Blackwell cards for PCIe workloads, each at one flat monthly price.

## What you get with 8-GPU server rental

A bare metal 8-GPU server runs your operating system directly on the hardware. You get root over SSH, every CPU core, all eight GPUs and all local drives. You can tune the kernel, pin data loaders to the right NUMA node, change driver branches or run your own container runtime, within our acceptable use policy.

NVIDIA servers arrive with Ubuntu 22.04 or 24.04 and NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled. MI355X servers get ROCm images. Every server is covered by a 99.9% monthly availability SLA for power, network and hardware, with service credits if we fall short.

## The six configurations compared

| Server | GPUs | GPU link | CPU and RAM | Storage and network | Price/mo |
| --- | --- | --- | --- | --- | --- |
| NVIDIA HGX B300 | 8 × 288 GB HBM3e | NVLink 5 | 2× Xeon 6776P, 128 cores, 4 TB RAM | 8× 3.84 TB NVMe, 8× 800G ConnectX-8 | $39,900 |
| NVIDIA HGX B200 | 8 × 180 GB HBM3e | NVLink 5 | 2× Xeon 8570, 112 cores, 2 TB RAM | 8× 3.84 TB NVMe, 8× 400G ConnectX-7 | $34,900 |
| NVIDIA HGX H200 | 8 × 141 GB HBM3e | NVLink 4 | 2× Xeon 8480+, 112 cores, 2 TB RAM | 8× 3.84 TB NVMe, 8× 400G ConnectX-7 | $23,900 |
| NVIDIA HGX H100 | 8 × 80 GB HBM3 | NVLink 4 | 2× Xeon 8480+, 112 cores, 2 TB RAM | 8× 3.84 TB NVMe, 8× 400G ConnectX-7 | $16,900 |
| AMD Instinct MI355X | 8 × 288 GB HBM3e | Infinity Fabric | 2× EPYC 9575F, 128 cores, 3 TB RAM | 8× 3.84 TB NVMe, 8× 400G RoCE | $19,900 |
| 8× RTX PRO 6000 Blackwell | 8 × 96 GB GDDR7 | PCIe Gen5, no NVLink | 2× EPYC 9555, 128 cores, 1.5 TB RAM | 4× 3.84 TB NVMe, 2× 100 GbE | $10,490 |

Over a 730-hour month, that is about $6.83 per GPU-hour on HGX B300, $4.09 on HGX H200 and $1.80 on the RTX PRO 6000 server.

## NVLink domains, and when PCIe is enough

On an HGX board, all eight GPUs share one NVLink domain. Traffic between tensor-parallel shards, gradient all-reduces and expert routing in mixture-of-experts models travels over NVLink instead of PCIe or the network. Per NVIDIA's published specifications, NVLink 5 gives each B300 and B200 GPU 1.8 TB/s, and NVLink 4 gives each H200 and H100 900 GB/s. AMD lists seven Infinity Fabric links of 153.6 GB/s per MI355X GPU. After handover, an NCCL all-reduce test confirms that the links behave as expected; our [multi-GPU and NCCL guide](https://overgpu.com/docs/multi-gpu-nccl) shows how to run one.

The 8× RTX PRO 6000 server is different: its GPUs talk over PCIe Gen5 and have no NVLink. That suits work where each GPU runs on its own, such as one model replica per GPU for inference, rendering or batch jobs. It is a weaker fit for tensor-parallel training, where GPUs exchange data on every step. Our [NVLink vs PCIe guide](https://overgpu.com/blog/nvlink-vs-pcie) explains the trade-off in more detail.

## When to choose 8 GPUs

- **The model does not fit one GPU.** A 405B-parameter model needs roughly 810 GB for BF16 weights alone. An HGX H200 with 1,128 GB holds it with room for the KV cache. An HGX H100 with 640 GB does not, though it can hold the same model in FP8 at about 405 GB.
- **Full fine-tuning of large models.** With Adam, plan roughly 16 to 20 bytes per parameter before activations. For a 70B model that is about 1.1 to 1.4 TB, which fits within the 2,304 GB of an HGX B300 or MI355X server with room left for activations.
- **Throughput on one machine.** Data-parallel training or several inference replicas, fed from local NVMe and up to 4 TB of RAM.
- **Cost per GPU.** An HGX H200 server costs $23,900 a month, against $24,720 for eight single H200 plans.

If one GPU is enough, a [single GPU server](https://overgpu.com/products/single-gpu-servers) keeps the bill smaller. If one server is not enough, several can be linked by [private networking](https://overgpu.com/products/private-networking), or a [GB300 NVL72](https://overgpu.com/gpus/nvidia-gb300-nvl72) rack gives you one 72-GPU NVLink domain.

## HGX server rental: storage, bandwidth and ordering

HGX and MI355X servers carry eight 3.84 TB NVMe drives, about 30.7 TB raw, and the RTX PRO 6000 server carries four. Local storage is not backed up by us unless agreed in writing, so keep a copy of the checkpoints you cannot lose somewhere else. Inbound traffic is not counted, so pulling datasets and model weights onto the server does not touch your 50 TB monthly allowance.

Ordering works like every OverGPU order: on the [deployment page](https://overgpu.com/deploy), choose the server, set up the OS image, optional preinstalled software and your login (root password or SSH key), create a free account or log in, and pay the first month from your balance. Each top-up is between $100 and $10,000, so funding the first month of a larger server may take several [crypto top-ups](https://overgpu.com/pay-with-crypto); bonus credit counts toward the payment. If your balance is short, the order is held for 3 hours and paid automatically once your top-ups cover it; if it is not paid by then, it is cancelled and nothing is charged.

The server is delivered in under 10 minutes after payment, even when stock is limited, with its IP address and SSH details in your dashboard. Orders of several 8-GPU servers may require verification. Once you have an active server, you can ask by ticket for term pricing for 6 to 12 months and volume pricing; current list prices are on our [pricing page](https://overgpu.com/pricing).

## Frequently asked questions

### Are 8-GPU servers bare metal or virtual machines?

Bare metal. The whole physical server is assigned to you alone, with root access over SSH and no hypervisor between you and the eight GPUs.

### Does the 8× RTX PRO 6000 server have NVLink?

No. Its GPUs connect over PCIe Gen5, which suits one model per GPU, rendering and batch work better than tensor-parallel training. For NVLink, choose an HGX server.

### How much bandwidth comes with an 8-GPU server?

50 TB of outbound traffic per month under fair use, with no egress fees. Inbound traffic is not counted.

### Can I connect several 8-GPU servers together?

Yes. Several 8-GPU servers can be linked by private networking, which you can ask for in the order notes, and each HGX server keeps its listed network adapters, such as 8× 400G ConnectX-7. For a single 72-GPU NVLink domain, choose a GB300 NVL72 rack.

### Is there term or volume pricing?

Yes, for customers with an active server, who can ask by ticket. We offer term pricing for 6 to 12 months and volume pricing for several servers, usually from three.

## Related pages

- [Private network between GPU servers](https://overgpu.com/products/private-networking): Link your OverGPU servers on an isolated network for data pipelines and multi-node jobs. See how it works.
- [Rent NVIDIA B300 and HGX B300 servers from $5,190 per month](https://overgpu.com/gpus/nvidia-b300): One dedicated 288 GB B300 for $5,190 (about $7.11 per GPU-hour at 24/7) or an 8-GPU HGX B300 server for $39,900. Paid in crypto.
- [B200 vs B300: is Blackwell Ultra worth the upgrade?](https://overgpu.com/compare/b200-vs-b300): 180 vs 288 GB of HBM3e, the FP4 uplift and 400G vs 800G networking. See what fits, the cost per GB and when the B200 is enough.
- [MI355X vs B200: memory, software and cost compared](https://overgpu.com/compare/mi355x-vs-b200): 288 GB vs 180 GB of HBM, ROCm vs CUDA, 4-bit formats and monthly cost per GPU. See which 8-GPU server fits your models and budget.
- [NCCL tests on a multi-GPU server: all_reduce_perf step by step](https://overgpu.com/docs/multi-gpu-nccl): Check NVLink topology, build nccl-tests, read busbw from all_reduce_perf and launch torchrun jobs across 8 GPUs.
- [NVLink vs PCIe: bandwidth, topology and when it matters](https://overgpu.com/blog/nvlink-vs-pcie): Per-GPU bandwidth, NVSwitch and NVL72, which multi-GPU workloads need NVLink and which run fine on PCIe, with worked numbers.