# Ubuntu GPU image with CUDA preinstalled

> What our Ubuntu GPU image with CUDA preinstalled contains, how to check driver and toolkit versions, pick 22.04 or 24.04 and keep the driver pinned safely.

Source: https://overgpu.com/docs/os-images  
Updated: 2026-10-22  
Publisher: OverGPU (https://overgpu.com/)

Every OverGPU server with NVIDIA GPUs is delivered with an Ubuntu GPU image with CUDA preinstalled, and AMD servers come with a ROCm image, so you can run a workload right after your first login. This page lists what the images contain, how to check the exact versions on your server, how to choose between Ubuntu 22.04 and 24.04, and how to keep the driver stable when you update packages.

## What is in our images

- **Operating system:** Ubuntu 22.04 LTS or 24.04 LTS. Debian or Rocky Linux on request.
- **NVIDIA servers:** NVIDIA driver, CUDA, Docker and the NVIDIA Container Toolkit, preinstalled.
- **AMD servers:** ROCm images for AMD Instinct MI355X.
- **Access:** Full root over SSH with your public key.
- **Software:** Install anything allowed by our [acceptable use policy](https://overgpu.com/legal/acceptable-use).

Exact versions depend on when your server was installed, so this page does not list them: the commands below show what is on your machine. The [OS images overview](https://overgpu.com/products/os-images) summarizes the choices.

## Check exact versions on your server

```
cat /etc/os-release        # Ubuntu release
uname -r                   # running kernel
nvidia-smi                 # driver version, newest CUDA it supports
nvcc --version             # CUDA toolkit used for compiling
ls -d /usr/local/cuda*     # toolkits installed side by side
docker --version
nvidia-ctk --version       # NVIDIA Container Toolkit
dpkg -l | grep -i nvidia   # NVIDIA packages and versions
```

The "CUDA Version" printed by `nvidia-smi` is the newest CUDA release the driver supports, not the installed toolkit; `nvcc` shows the toolkit. If `nvcc` is not found but `/usr/local/cuda` exists, run `export PATH=/usr/local/cuda/bin:$PATH`. The [driver and CUDA guide](https://overgpu.com/docs/nvidia-drivers-cuda) explains how the two versions fit together.

On AMD servers:

```
amd-smi version                 # tool, library and ROCm versions
rocm-smi --showdriverversion    # kernel driver version
cat /opt/rocm/.info/version     # installed ROCm release
```

## Ubuntu 22.04 or 24.04?

| Release | Ubuntu 22.04 LTS | Ubuntu 24.04 LTS |
| --- | --- | --- |
| System Python | 3.10 | 3.12 |
| Canonical standard support | Until 2027 | Until 2029 |
| pip install outside a virtual environment | Allowed, not advised | Blocked (externally-managed-environment) |
| Good fit | Stacks already pinned to 22.04 | New projects |

Choose 24.04 for new work, and 22.04 when a tool, vendor package or internal build is only certified there. Containers make the choice less critical: a container brings its own user space and only the driver comes from the host, so a 22.04-based image runs on a 24.04 server. See [Docker GPU containers](https://overgpu.com/docs/docker-containers). On either release, install Python packages in a virtual environment (venv or uv), not system-wide.

## ROCm images, Debian and Rocky Linux

AMD Instinct MI355X servers are delivered with a ROCm image. Check the GPUs with `rocm-smi`, then follow the [ROCm setup guide](https://overgpu.com/docs/amd-rocm) for user groups, PyTorch and vLLM.

If you need Debian or Rocky Linux instead of Ubuntu, ask for it in the order notes when you [deploy the server](https://overgpu.com/deploy). Commands in these docs target Ubuntu; on Rocky Linux, use `dnf` instead of `apt`, and some package names differ.

## Reinstall or change the OS

Reinstalls are done on request through support: [open a support ticket](https://overgpu.com/help/getting-support) from your dashboard with the server's IP address and the image you want. Before you ask:

1. Copy everything you need off the server. Treat a reinstall as erasing the server's data; local storage is not backed up by us.
2. Note your custom settings: users and keys, `/etc/fstab` entries, systemd units and firewall rules.

After the reinstall, the server's host key changes. Remove the old entry on your computer with `ssh-keygen -R 203.0.113.10` (use your server's IP), log in again and redo your setup.

## Keep the driver pinned during apt upgrades

An apt upgrade, including Ubuntu's automatic security updates (unattended-upgrades) when they are enabled, can replace the NVIDIA driver while the old kernel module is still loaded. nvidia-smi then fails with "Driver/library version mismatch" until you reboot, and running jobs can crash. Holding the driver packages lets you decide when the driver changes.

1. List the NVIDIA packages: `dpkg -l | grep -i nvidia | awk '{print $2}'`
2. Hold the driver packages from that list. Their names depend on how the driver was installed and include the driver branch number (NNN below). For example: `sudo apt-mark hold nvidia-driver-NNN-server nvidia-utils-NNN-server \ libnvidia-compute-NNN-server` Use the exact names from your list. Leave the container toolkit packages, whose names contain `nvidia-container`, unheld. On 8-GPU HGX servers, also hold the Fabric Manager package (its name starts with `nvidia-fabricmanager`), which must always match the driver version.
3. Check with `apt-mark showhold`. The next `sudo apt upgrade` lists those packages as kept back.

> Held packages also miss driver security fixes, so review them regularly. To upgrade on your schedule, run `sudo apt-mark unhold` with the same names, then `sudo apt update && sudo apt upgrade`, reboot and check `nvidia-smi`. Kernel updates are not held: after any reboot, confirm nvidia-smi still works, and if it fails after a kernel update, unhold and upgrade the driver packages, then reboot.

## Related pages

- [GPU server OS images with CUDA, ROCm and Docker](https://overgpu.com/products/os-images): Ubuntu 22.04 and 24.04 with NVIDIA drivers, CUDA, Docker and the Container Toolkit, plus ROCm for AMD. See what is preinstalled.
- [Check CUDA version and NVIDIA driver on Ubuntu](https://overgpu.com/docs/nvidia-drivers-cuda): Check NVIDIA driver and CUDA version with nvidia-smi and nvcc, see how they must match and add a CUDA toolkit on Ubuntu 24.04 without a new driver.
- [ROCm setup for AMD Instinct MI355X](https://overgpu.com/docs/amd-rocm): Check GPUs with rocm-smi and amd-smi, set user groups, run PyTorch and vLLM in ROCm containers and test RCCL.
- [GPU server quickstart](https://overgpu.com/docs/quickstart): Log in over SSH, check the GPUs, update Ubuntu and run a first GPU container, with a check at each step.