# Check CUDA version and NVIDIA driver on Ubuntu

> Check NVIDIA driver and CUDA version with nvidia-smi and nvcc, see how they must match and add a CUDA toolkit on Ubuntu 24.04 without a new driver.

Source: https://overgpu.com/docs/nvidia-drivers-cuda  
Updated: 2026-10-22  
Publisher: OverGPU (https://overgpu.com/)

To check NVIDIA driver and CUDA version on a GPU server you need two commands, because they answer different questions: `nvidia-smi` reports the driver, and `nvcc` reports the CUDA toolkit you compile with. This page explains both, how they must line up, how to install another CUDA toolkit on Ubuntu 24.04 or 22.04 side by side, and how to fix the errors you are most likely to meet.

## Read the driver and CUDA version in nvidia-smi

```
nvidia-smi
nvidia-smi --query-gpu=index,name,driver_version,memory.total --format=csv
cat /proc/driver/nvidia/version
```

The header of `nvidia-smi` shows three versions:

- **NVIDIA-SMI** is the version of the tool itself, normally the same as the driver.
- **Driver Version** is the installed driver.
- **CUDA Version** is the newest CUDA release this driver supports. It does not mean that toolkit is installed.

`/proc/driver/nvidia/version` shows the kernel module that is actually loaded. If it differs from the installed driver packages, for example after an upgrade, a reboot is pending.

## Check the CUDA toolkit with nvcc

```
nvcc --version
ls -d /usr/local/cuda*
```

The line "Cuda compilation tools, release X.Y" gives the toolkit version. `/usr/local/cuda` points to the default toolkit, and other versions live in `/usr/local/cuda-X.Y`. If `nvcc` is not found, add the toolkit to your environment (append the lines to `~/.bashrc` to keep them):

```
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
```

Frameworks often ignore this toolkit. PyTorch wheels and most containers ship their own CUDA libraries and only need a driver that is new enough, as the [PyTorch and vLLM guide](https://overgpu.com/docs/pytorch-vllm) shows. Check what PyTorch sees with:

```
python3 -c "import torch; print(torch.__version__, torch.version.cuda,
  torch.cuda.is_available())"
```

## How driver and toolkit versions fit together

- **The driver must be at least as new as the code.** The CUDA Version shown by nvidia-smi should be equal to or higher than the CUDA version your toolkit, wheel or container targets. Older toolkits run on newer drivers.
- **Minor version compatibility has limits.** Within one major release, such as 12.x, NVIDIA lets many applications built with a newer minor version run on an older driver of the same major release, but features such as PTX just-in-time compilation may fail. Test before you rely on it.
- **The GPU architecture sets a floor.** Hopper GPUs (H100, H200) work with CUDA 12 and later. Blackwell GPUs (B200, RTX PRO 6000, RTX 5090) need CUDA 12.8 or later and framework builds that include Blackwell; for B300, check NVIDIA's release notes for the minimum version.

Rule of thumb: keep the installed driver, and pick toolkits, wheels and container tags whose CUDA version is at or below the CUDA Version in nvidia-smi.

## Install another CUDA toolkit on Ubuntu 24.04

You can add a toolkit side by side from NVIDIA's apt repository without replacing the driver. If `/etc/apt/sources.list.d/` already has a CUDA repository entry, skip step 1.

1. Add NVIDIA's repository key. On 22.04, replace `ubuntu2404` with `ubuntu2204`: `repo=https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64 wget $repo/cuda-keyring_1.1-1_all.deb sudo dpkg -i cuda-keyring_1.1-1_all.deb sudo apt update`
2. List the toolkit versions available: `apt-cache search --names-only '^cuda-toolkit-[0-9]+-[0-9]+$'`
3. Install the toolkit-only package for the version you need, for example 12.8: `sudo apt install -y cuda-toolkit-12-8` Always use `cuda-toolkit-X-Y`. The `cuda`, `cuda-X-Y` and `cuda-drivers` packages also install a driver.
4. Select it in your shell and check: `export PATH=/usr/local/cuda-12.8/bin:$PATH export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} nvcc --version` You should see release 12.8. When the toolkits come from NVIDIA's packages, `sudo update-alternatives --config cuda` changes the default that `/usr/local/cuda` points to.

With NVIDIA's repository enabled, a later `apt upgrade` may offer newer driver packages. Read the list before you confirm, or hold the driver as described in [OS images](https://overgpu.com/docs/os-images). If only one project needs another CUDA version, a container is often simpler: see [Docker GPU containers](https://overgpu.com/docs/docker-containers).

## Enable persistence mode

Persistence mode keeps the driver initialized when no process is using the GPU, so jobs start without waiting for the driver to load. Check it, then turn it on:

```
nvidia-smi --query-gpu=index,persistence_mode --format=csv
sudo nvidia-smi -pm 1
```

You should see "Enabled" for every GPU. The setting does not survive a reboot. If it is disabled after a restart, apply it at boot from the root crontab: run `sudo crontab -e` and add the line `@reboot /usr/bin/nvidia-smi -pm 1`.

## Troubleshooting common errors

### NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver

The kernel module is not loaded, often after a kernel update for which it was not built. Collect:

```
uname -r
lsmod | grep nvidia
dkms status
sudo dmesg | grep -i -E 'nvidia|nvrm'
```

If `dkms status` lists the NVIDIA module for other kernels but not for the one shown by `uname -r`, install the matching headers, rebuild and reboot:

```
sudo apt install -y linux-headers-$(uname -r)
sudo dkms autoinstall
sudo reboot
```

If `dkms` is not installed, the driver uses prebuilt modules: send us the output above in a [support ticket](https://overgpu.com/help/getting-support) from your dashboard.

### Failed to initialize NVML: Driver/library version mismatch

The driver packages were upgraded while the old module was loaded. Run `sudo reboot`. Afterwards, nvidia-smi and `/proc/driver/nvidia/version` should show the same version. Holding the driver packages prevents this.

### CUDA error 802, system not yet initialized

On 8-GPU HGX servers, NVIDIA Fabric Manager must be running, at exactly the driver's version, before CUDA can use the NVSwitch fabric. Check `systemctl status nvidia-fabricmanager`, restart it with `sudo systemctl restart nvidia-fabricmanager`, and open a ticket from your dashboard if it does not start.

### CUDA driver version is insufficient for CUDA runtime version

The program, wheel or container was built for a newer CUDA than the driver supports. Use a build for an older CUDA release, or open a ticket from your dashboard about a driver upgrade. For hardware errors (Xid messages in `dmesg`), see [GPU monitoring](https://overgpu.com/docs/gpu-monitoring).

## Frequently asked questions

### Why does nvidia-smi show a different CUDA version than nvcc?

nvidia-smi shows the newest CUDA version the driver supports, while nvcc shows the toolkit you compile with. A toolkit older than the driver's CUDA Version is normal and works.

### Do I need the CUDA toolkit to run PyTorch?

No. PyTorch wheels and most containers bring their own CUDA libraries and only need a recent enough driver. Install a toolkit when you compile CUDA code yourself.

### Can I upgrade the NVIDIA driver myself?

Yes, you have full root access. Plan a reboot, keep Fabric Manager at the same version as the driver on 8-GPU HGX servers, and open a ticket from your dashboard first if you want advice.

## Related pages

- [Ubuntu GPU image with CUDA preinstalled](https://overgpu.com/docs/os-images): What our Ubuntu GPU image with CUDA preinstalled contains, how to check driver and toolkit versions, pick 22.04 or 24.04 and keep the driver pinned safely.
- [Docker GPU containers with the NVIDIA Container Toolkit](https://overgpu.com/docs/docker-containers): Use --gpus, select devices, reserve GPUs in Compose, fix PyTorch shared memory and run ROCm containers.
- [Serve an LLM with vLLM](https://overgpu.com/docs/pytorch-vllm): Set up Python with uv, install PyTorch and vLLM, test the OpenAI-compatible API and run it as a service.
- [GPU and AI infrastructure glossary](https://overgpu.com/glossary): Clear, vendor-neutral definitions of the AI infrastructure terms that matter, from HBM3e and NVLink to FSDP, KV cache and FP8. Look one up.