8-GPU server rental: bare-metal NVIDIA HGX and AMD Instinct

8-GPU server rental at OverGPU gives you an entire physical machine: eight whole GPUs, two server CPUs, 1.5 to 4 TB of system memory and local NVMe, with no hypervisor and no other tenant. You can choose NVIDIA HGX B300, B200, H200 or H100, AMD Instinct MI355X, or a server with eight RTX PRO 6000 Blackwell cards for PCIe workloads, each at one flat monthly price.

What you get with 8-GPU server rental

A bare metal 8-GPU server runs your operating system directly on the hardware. You get root over SSH, every CPU core, all eight GPUs and all local drives. You can tune the kernel, pin data loaders to the right NUMA node, change driver branches or run your own container runtime, within our acceptable use policy.

NVIDIA servers arrive with Ubuntu 22.04 or 24.04 and NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled. MI355X servers get ROCm images. Every server is covered by a 99.9% monthly availability SLA for power, network and hardware, with service credits if we fall short.

The six configurations compared

Over a 730-hour month, that is about $6.83 per GPU-hour on HGX B300, $4.09 on HGX H200 and $1.80 on the RTX PRO 6000 server.

NVLink domains, and when PCIe is enough

On an HGX board, all eight GPUs share one NVLink domain. Traffic between tensor-parallel shards, gradient all-reduces and expert routing in mixture-of-experts models travels over NVLink instead of PCIe or the network. Per NVIDIA's published specifications, NVLink 5 gives each B300 and B200 GPU 1.8 TB/s, and NVLink 4 gives each H200 and H100 900 GB/s. AMD lists seven Infinity Fabric links of 153.6 GB/s per MI355X GPU. After handover, an NCCL all-reduce test confirms that the links behave as expected; our multi-GPU and NCCL guide shows how to run one.

The 8× RTX PRO 6000 server is different: its GPUs talk over PCIe Gen5 and have no NVLink. That suits work where each GPU runs on its own, such as one model replica per GPU for inference, rendering or batch jobs. It is a weaker fit for tensor-parallel training, where GPUs exchange data on every step. Our NVLink vs PCIe guide explains the trade-off in more detail.

When to choose 8 GPUs

• The model does not fit one GPU. A 405B-parameter model needs roughly 810 GB for BF16 weights alone. An HGX H200 with 1,128 GB holds it with room for the KV cache. An HGX H100 with 640 GB does not, though it can hold the same model in FP8 at about 405 GB.
• Full fine-tuning of large models. With Adam, plan roughly 16 to 20 bytes per parameter before activations. For a 70B model that is about 1.1 to 1.4 TB, which fits within the 2,304 GB of an HGX B300 or MI355X server with room left for activations.
• Throughput on one machine. Data-parallel training or several inference replicas, fed from local NVMe and up to 4 TB of RAM.
• Cost per GPU. An HGX H200 server costs $23,900 a month, against $24,720 for eight single H200 plans.

If one GPU is enough, a single GPU server keeps the bill smaller. If one server is not enough, several can be linked by private networking, or a GB300 NVL72 rack gives you one 72-GPU NVLink domain.

HGX server rental: storage, bandwidth and ordering

HGX and MI355X servers carry eight 3.84 TB NVMe drives, about 30.7 TB raw, and the RTX PRO 6000 server carries four. Local storage is not backed up by us unless agreed in writing, so keep a copy of the checkpoints you cannot lose somewhere else. Inbound traffic is not counted, so pulling datasets and model weights onto the server does not touch your 50 TB monthly allowance.

Ordering works like every OverGPU order: on the deployment page, choose the server, set up the OS image, optional preinstalled software and your login (root password or SSH key), create a free account or log in, and pay the first month from your balance. Each top-up is between $100 and $10,000, so funding the first month of a larger server may take several crypto top-ups; bonus credit counts toward the payment. If your balance is short, the order is held for 3 hours and paid automatically once your top-ups cover it; if it is not paid by then, it is cancelled and nothing is charged.

The server is delivered in under 10 minutes after payment, even when stock is limited, with its IP address and SSH details in your dashboard. Orders of several 8-GPU servers may require verification. Once you have an active server, you can ask by ticket for term pricing for 6 to 12 months and volume pricing; current list prices are on our pricing page.