Dedicated vs shared GPU: what changes when you share

The dedicated vs shared GPU question sounds like a pricing detail, but it decides how much memory you can allocate, how steady your latency is, whose code runs next to your data and how deeply you can debug. "Shared" covers very different technologies, from hardware partitions with strong isolation to plain time-slicing with almost none. This guide explains each one, what it changes for AI workloads, and when sharing is a perfectly good choice.

Dedicated vs shared GPU: five ways a GPU gets shared

A dedicated GPU is a whole physical card assigned to one customer, with all of its memory, compute and power budget. Everything else is some form of sharing:

• vGPU. NVIDIA's virtual GPU software lets several virtual machines share one physical GPU. Each VM gets a fixed slice of the GPU's memory, called a profile, and in the common time-sliced mode the GPU's engines switch between VMs under a scheduler. The guest sees a smaller GPU.
• MIG (Multi-Instance GPU). A hardware feature of NVIDIA's high-end data-center GPUs since the A100, such as the H100 and H200, and of the RTX PRO 6000 Blackwell, that splits one GPU into isolated instances, each with its own memory, cache and compute units. An H100 can be cut into up to seven instances, an RTX PRO 6000 into up to four.
• Time-slicing. Several processes or containers take turns on one GPU, for example when a Kubernetes cluster advertises one GPU as several. Each sees the full memory, but it is one pool with no limit per tenant, and NVIDIA's own documentation for this feature notes that, unlike MIG, it provides no memory or fault isolation. MPS (Multi-Process Service) lets processes run concurrently instead of taking turns, with similarly limited isolation.
• Containers on shared hosts. You get one or more whole GPUs, but inside a container on a host that also runs other customers' containers. The GPU is yours; the CPUs, RAM, disks, network, kernel and driver are shared.
• Marketplace hosts. Independent operators list their machines on a marketplace. The GPU may be whole, but hardware quality, uptime, network and physical security vary by host, and each host's operator controls the machine.

Memory: what you can actually allocate

Memory is the first thing sharing takes away, and for LLMs it is usually the binding constraint. A MIG instance or a vGPU profile gives you a fixed fraction of the card: the smallest MIG instance on an 80 GB H100 has 10 GB. A 70B model in FP8 needs about 70 GB for its weights alone, so no slice smaller than the whole card will hold it, and even a 32B model in BF16, at about 64 GB, will not fit a half-card slice.

Time-slicing looks better on paper, since every tenant sees the full memory. In practice the pool is first come, first served. Many frameworks reserve memory up front: vLLM takes 90% of the GPU by default and JAX preallocates 75%. Two of them on one time-sliced GPU collide, and a neighbor's allocation can make your job fail with an out-of-memory error that has nothing to do with your code. Our guide to VRAM for LLMs shows how much memory common models need, which tells you quickly whether a slice can work at all.

Performance consistency and noisy neighbors

Averages hide most sharing costs; tail latency shows them. Under time-slicing, each tenant waits while another runs, so with two busy tenants each gets at most about half of the GPU's time, and context switches add their own overhead. An inference request that arrives during a neighbor's turn simply waits.

MIG is far more predictable, because each instance owns its compute units and its share of memory bandwidth. The price is a hard cap: a one-seventh slice of an H100 stays a one-seventh slice even when the rest of the card is idle.

Whole GPUs on shared hosts have subtler issues. Tokenization, data loading and preprocessing compete for shared CPU cores; datasets and checkpoints compete for shared disks and network; GPUs behind the same PCIe switch compete for host bandwidth. For training, consistency matters more than it seems: in synchronous data-parallel training, every step waits for the slowest GPU, so one throttled worker slows all of them.

Security isolation

Every sharing layer adds a trust boundary: the hypervisor, the vGPU manager, the GPU driver, the container runtime and the host kernel. Each one has to hold for your data to stay private from the next tenant.

• Containers share the host kernel. A kernel or driver vulnerability can cross from one tenant to another, which is why containers are considered a weaker boundary than virtual machines.
• Time-sliced GPUs share one device without memory or fault isolation, so a crash or a misbehaving process can affect everyone on the card.
• Marketplace hosts are run by operators you have no contract with beyond the marketplace's terms, and they have physical and administrative access to the machine that holds your weights and data.
• Leftovers matter. Ask what happens to disks when the previous tenant leaves: are they wiped, and how?

Sharing inside your own organization is a different trust model. Partitioning your own GPU with MIG to run several internal services raises none of these questions, because every neighbor is you.

Debugging and observability

When a job fails in the middle of the night, visibility decides how fast you fix it. With a whole GPU and root access, nvidia-smi shows every process and every byte of memory, profilers such as Nsight Systems and Nsight Compute see the whole device, and GPU errors (Xid messages) appear in a kernel log you can read. You also choose the driver and CUDA versions, and you can still run your own containers on top (see our Docker and containers guide).

Shared setups narrow that view. In a container on a shared host, the driver version is fixed by the host, your CUDA user-space libraries must stay compatible with it, and the kernel log where GPU faults are reported is often out of reach. On vGPU, profiling tools are typically disabled unless the host enables them. On a time-sliced GPU, utilization numbers mix your work with your neighbors', so a slow run might be your code or someone else's job, and you cannot tell which. Reproducibility suffers for the same reason. For monitoring a dedicated card, see our GPU monitoring guide.

When sharing is fine

Sharing is a trade-off, and often the right one:

• Notebooks and prototyping on small models, where a 10–20 GB slice is plenty and latency does not matter.
• CI and smoke tests that need a GPU for minutes, not hours.
• Teaching and workshops, where many students each need a little GPU.
• Small internal services such as embedding or classification models, which you can pack onto your own GPU with MIG.
• Bursty batch jobs that tolerate variable speed and restart cleanly.

Go dedicated when you need the full memory, steady latency, long uninterrupted runs, fast multi-GPU communication or a clear answer to who can access the machine. Our guide on how to choose a GPU server helps with the next step: sizing the card.

How to check what you are renting

Whatever the listing says, a few commands tell you what you actually have:

nvidia-smi -L
nvidia-smi --query-gpu=name,memory.total,memory.used,mig.mode.current \
  --format=csv
nvidia-smi -q | grep -i "virtualization mode"
nvidia-smi topo -m

• nvidia-smi -L lists the GPUs, with MIG devices underneath them if partitioning is active.
• memory.total should match the full card; a vGPU profile shows a fraction of it. A memory.used value well above a few hundred megabytes before you start anything means something else is on the GPU. mig.mode.current should read Disabled, or N/A on GPUs without MIG.
• The virtualization mode reads None on bare metal, Pass-Through in a VM that owns the whole GPU, and VGPU in a vGPU guest.
• nvidia-smi topo -m shows how GPUs connect: NV entries mean NVLink, while PIX, PXB, NODE or SYS mean PCIe paths.

Check the host too: nproc, free -g and lsblk show the CPU, RAM and disks you were promised.

How OverGPU does it

We only rent whole physical GPUs, each assigned to one customer: no vGPU slicing, no MIG partitions and no time-sharing.

• 8-GPU servers are bare metal. The whole machine is yours: GPUs, CPUs, RAM, NVMe and network adapters, with no hypervisor in between.
• Single-GPU plans are a dedicated VM with the full GPU passed through. The guest driver talks to the physical card, all of its memory is yours, and the VM boundary isolates you from the host.
• You have root over SSH on Ubuntu 22.04 or 24.04 images with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled, or on ROCm images for AMD. You pick the versions and the tools.
• Disks are securely erased between customers (see data deletion), and private networking between your own servers is available on request.

If you want one card, our single-GPU servers range from a 32 GB RTX 5090 to a 288 GB B300. If you need NVLink and the whole host, our 8-GPU servers are bare-metal NVIDIA and AMD machines.

Dedicated vs shared GPU memory

“Shared GPU memory” has a second meaning. On a PC, Windows reports dedicated GPU memory, the VRAM on the card, and shared GPU memory, a part of system RAM the GPU can borrow when VRAM runs out. Borrowed RAM is reached over PCIe, tens of times slower than the GPU's own memory, so a model that spills into it slows down sharply. On a server the rule is the same: the weights, the KV cache and the activations should fit in GPU memory. That is why GPU memory, not system RAM, decides which models a server can serve; the VRAM calculator checks it for your model.

Frequently asked questions

What is the difference between a dedicated and a shared GPU?

A dedicated GPU is a whole physical card assigned to one customer, with all of its memory and compute. A shared GPU is split between tenants through vGPU profiles, MIG partitions, time-slicing or a shared host, which reduces your memory, your performance consistency or your isolation.

Is a MIG instance the same as a dedicated GPU?

No. A MIG instance is a hardware-isolated slice with its own memory and compute units, which makes it the strongest form of sharing, but you only get part of the GPU: on an 80 GB H100 the smallest instance has 10 GB.

How can I tell if my cloud GPU is shared?

Run nvidia-smi and check that the total memory matches the full card, that MIG mode is disabled and that the virtualization mode is None or Pass-Through rather than VGPU. Memory already in use before you start anything is another sign of a neighbor.

Are OverGPU single-GPU plans virtual machines?

Yes. Single-GPU plans are a dedicated VM with the full GPU passed through, so the GPU and all of its memory are yours alone. Our 8-GPU servers are bare-metal machines.

When is a shared GPU good enough?

For notebooks, small models, CI tests, teaching and bursty jobs that tolerate variable latency, a slice is often enough. Move to a dedicated GPU when you need the full memory, steady latency or long uninterrupted runs.