Ubuntu GPU Image: CUDA and Docker Preinstalled

Every server ships with a ready-to-run OS image. Here is what it contains, how to check exact versions and how to keep the GPU stack stable through updates.

Every OverGPU server with NVIDIA GPUs is delivered with an Ubuntu GPU image with CUDA preinstalled, and AMD servers come with a ROCm image, so you can run a workload right after your first login. This page lists what the images contain, how to check the exact versions on your server, how to choose between Ubuntu 22.04 and 24.04, and how to keep the driver stable when you update packages.

What is in our images

Operating system Ubuntu 22.04 LTS or 24.04 LTS. Debian or Rocky Linux on request.
NVIDIA servers NVIDIA driver, CUDA, Docker and the NVIDIA Container Toolkit, preinstalled.
AMD servers ROCm images for AMD Instinct MI355X.
Access Full root over SSH with your public key.
Software Install anything allowed by our acceptable use policy.

Exact versions depend on when your server was installed, so this page does not list them: the commands below show what is on your machine. The OS images overview summarizes the choices.

Check exact versions on your server

cat /etc/os-release        # Ubuntu release
uname -r                   # running kernel
nvidia-smi                 # driver version, newest CUDA it supports
nvcc --version             # CUDA toolkit used for compiling
ls -d /usr/local/cuda*     # toolkits installed side by side
docker --version
nvidia-ctk --version       # NVIDIA Container Toolkit
dpkg -l | grep -i nvidia   # NVIDIA packages and versions

The "CUDA Version" printed by nvidia-smi is the newest CUDA release the driver supports, not the installed toolkit; nvcc shows the toolkit. If nvcc is not found but /usr/local/cuda exists, run export PATH=/usr/local/cuda/bin:$PATH. The driver and CUDA guide explains how the two versions fit together.

On AMD servers:

amd-smi version                 # tool, library and ROCm versions
rocm-smi --showdriverversion    # kernel driver version
cat /opt/rocm/.info/version     # installed ROCm release

Ubuntu 22.04 or 24.04?

Release	Ubuntu 22.04 LTS	Ubuntu 24.04 LTS
System Python	3.10	3.12
Canonical standard support	Until 2027	Until 2029
pip install outside a virtual environment	Allowed, not advised	Blocked (externally-managed-environment)
Good fit	Stacks already pinned to 22.04	New projects

Choose 24.04 for new work, and 22.04 when a tool, vendor package or internal build is only certified there. Containers make the choice less critical: a container brings its own user space and only the driver comes from the host, so a 22.04-based image runs on a 24.04 server. See Docker GPU containers. On either release, install Python packages in a virtual environment (venv or uv), not system-wide.

ROCm images, Debian and Rocky Linux

AMD Instinct MI355X servers are delivered with a ROCm image. Check the GPUs with rocm-smi, then follow the ROCm setup guide for user groups, PyTorch and vLLM.

If you need Debian or Rocky Linux instead of Ubuntu, ask for it in the order notes when you deploy the server. Commands in these docs target Ubuntu; on Rocky Linux, use dnf instead of apt, and some package names differ.

Reinstall or change the OS

Reinstalls are done on request through support: open a support ticket from your dashboard with the server's IP address and the image you want. Before you ask:

Copy everything you need off the server. Treat a reinstall as erasing the server's data; local storage is not backed up by us.
Note your custom settings: users and keys, /etc/fstab entries, systemd units and firewall rules.

After the reinstall, the server's host key changes. Remove the old entry on your computer with ssh-keygen -R 203.0.113.10 (use your server's IP), log in again and redo your setup.

Keep the driver pinned during apt upgrades

An apt upgrade, including Ubuntu's automatic security updates (unattended-upgrades) when they are enabled, can replace the NVIDIA driver while the old kernel module is still loaded. nvidia-smi then fails with "Driver/library version mismatch" until you reboot, and running jobs can crash. Holding the driver packages lets you decide when the driver changes.

List the NVIDIA packages:

dpkg -l | grep -i nvidia | awk '{print $2}'

Hold the driver packages from that list. Their names depend on how the driver was installed and include the driver branch number (NNN below). For example:
```
sudo apt-mark hold nvidia-driver-NNN-server nvidia-utils-NNN-server \
  libnvidia-compute-NNN-server
```
Use the exact names from your list. Leave the container toolkit packages, whose names contain nvidia-container, unheld. On 8-GPU HGX servers, also hold the Fabric Manager package (its name starts with nvidia-fabricmanager), which must always match the driver version.
Check with apt-mark showhold. The next sudo apt upgrade lists those packages as kept back.

Held packages also miss driver security fixes, so review them regularly. To upgrade on your schedule, run sudo apt-mark unhold with the same names, then sudo apt update && sudo apt upgrade, reboot and check nvidia-smi. Kernel updates are not held: after any reboot, confirm nvidia-smi still works, and if it fails after a kernel update, unhold and upgrade the driver packages, then reboot.