# GPU server OS images with CUDA, ROCm and Docker > GPU server OS images: Ubuntu 22.04 and 24.04 with NVIDIA drivers, CUDA, Docker and the Container Toolkit, plus ROCm for AMD. See what is preinstalled. Source: https://overgpu.com/products/os-images Updated: 2026-10-23 Publisher: OverGPU (https://overgpu.com/) ## Key points - **Ubuntu 22.04 and 24.04:** Both long-term support releases, with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled. - **ROCm images for AMD:** AMD Instinct MI355X servers get ROCm images, the AMD software stack for GPU compute. - **Full root over SSH:** Log in with an SSH key or a password, with full root. Install, update or replace anything allowed by our acceptable use policy. - **Debian or Rocky Linux:** Available on request for teams that standardize on another distribution. - **Reinstall on request:** Ask our 24/7 support for a fresh install when you want a clean start or a different image. - **Software preinstalled for you:** vLLM, Open WebUI, ComfyUI, JupyterLab and more, installed and tested before delivery at no extra cost. ## Recommended configurations - [NVIDIA H100 SXM](https://overgpu.com/gpus/nvidia-h100): $2,190 per month — 80 GB H100 on the Ubuntu CUDA image, ready for GPU containers - [8× AMD Instinct MI355X](https://overgpu.com/gpus/amd-mi355x): $19,900 per month — Eight MI355X GPUs with 288 GB each on a ROCm image - [NVIDIA RTX PRO 6000 Blackwell](https://overgpu.com/gpus/nvidia-rtx-pro-6000): $1,390 per month — 96 GB Blackwell GPU for CUDA inference and rendering ## Overview GPU server OS images at OverGPU come with the GPU stack already in place. NVIDIA servers boot Ubuntu 22.04 or 24.04 with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled, and AMD servers get ROCm images. You log in as root over SSH and start with your own code instead of a driver install. ## What is in our GPU server OS images | Image | For | Preinstalled | | --- | --- | --- | | Ubuntu 24.04 LTS | NVIDIA GPUs | NVIDIA drivers, CUDA, Docker, NVIDIA Container Toolkit | | Ubuntu 22.04 LTS | NVIDIA GPUs | NVIDIA drivers, CUDA, Docker, NVIDIA Container Toolkit | | ROCm image | AMD Instinct MI355X | ROCm | | Debian or Rocky Linux | On request | Discussed with your order | We do not list fixed driver or CUDA version numbers on this page, because they move with GPU generations and releases. After login, `nvidia-smi` shows the driver version and the highest CUDA version that driver supports. If your project depends on a specific driver branch, mention it in the order notes, or install it yourself with root access. On single-GPU plans, the image runs inside your dedicated VM, with the whole GPU passed through. On 8-GPU servers, it runs directly on the bare-metal machine. The checks below work the same way on both; only the topology output differs. ## The Ubuntu CUDA image in practice Pick 24.04 for a newer kernel and toolchain. Pick 22.04 if your build scripts or base containers still target it. Then run three checks: ``` nvidia-smi nvidia-smi topo -m docker run --rm --gpus all ubuntu nvidia-smi ``` The first lists every GPU with its memory, utilization and driver. The second prints the topology, and on an HGX server it shows NVLink between all eight GPUs. The third proves that containers reach the GPU through the NVIDIA Container Toolkit, which mounts the driver libraries and devices into the container when it starts. Containers are the easiest way to keep CUDA versions under control. A framework image brings its own CUDA libraries, and the host only needs a driver recent enough for them. If an image needs a newer CUDA than the host driver supports, it typically refuses to start, so compare against the CUDA version shown by `nvidia-smi` first. Our [Docker and containers guide](https://overgpu.com/docs/docker-containers) and [NVIDIA drivers and CUDA guide](https://overgpu.com/docs/nvidia-drivers-cuda) go further. If you install frameworks directly instead, use one virtual environment per project and pick wheels built for a CUDA version your driver supports. PyTorch, for example, publishes builds for several CUDA versions, so the same rule applies: check `nvidia-smi` first, then choose the build. ## ROCm images for AMD Instinct AMD Instinct MI355X servers get ROCm images instead of CUDA. ROCm is AMD's software stack for GPU compute, and frameworks such as PyTorch and vLLM support it. Check the GPUs with `rocm-smi`, or `amd-smi` on recent ROCm releases, and list the devices with `rocminfo`. On an MI355X server, both should show all eight GPUs. Containers built for ROCm reach the GPUs through the `/dev/kfd` and `/dev/dri` devices, passed with `--device` flags. Code that stays at the framework level usually moves over without changes. Custom CUDA kernels need porting with HIP. The [AMD ROCm guide](https://overgpu.com/docs/amd-rocm) covers the details. ## Other distributions, root access and reinstalls Debian or Rocky Linux are available on request, for teams whose tooling, hardening baselines or configuration management already target one of them. Mention the distribution in the order notes for our engineers, together with what you expect to find on the image. Every image gives you root over SSH: you choose the image, then log in with an SSH key or a root password, both set in the order form when you [deploy a server](https://overgpu.com/deploy). The [SSH access guide](https://overgpu.com/docs/ssh-access) covers keys and first login. You can install, upgrade or remove anything allowed by our acceptable use policy, including a different driver branch or container runtime. For a clean start or a different image, open a support ticket from your dashboard to ask for a reinstall, at any hour. Back up what you need first: local storage is not backed up by us unless agreed in writing. ## Preinstalled software, at no extra cost To start from a ready application rather than a bare system, pick one in the order form when you [deploy a server](https://overgpu.com/deploy). Our engineers install and test it on your image before delivery, at no extra cost, and the note that comes with your server explains how to open it. - **Serve and chat with models:** vLLM, SGLang, Ollama, Open WebUI with Ollama, llama.cpp, Text Generation WebUI, Hugging Face Text Generation Inference and NVIDIA Triton Inference Server, plus Langflow to build agents. - **Train and fine-tune:** PyTorch or TensorFlow with JupyterLab, Unsloth and Axolotl. - **Images and video:** ComfyUI, Stable Diffusion WebUI Forge, AUTOMATIC1111, InvokeAI, Fooocus, SwarmUI and Wan2GP, plus Kohya’s GUI, AI Toolkit and FluxGym to train LoRAs. - **Notebooks, code and more:** JupyterLab, code-server, Whisper, NVIDIA RAPIDS, Blender and a remote Linux desktop. Some of them run on NVIDIA GPUs only, and the order form shows only what fits the server you chose. The software stays yours to update, configure or remove. ## Keep your stack reproducible - Pin framework containers by tag or digest rather than `latest`. - Hold driver packages during long training runs, for example with `apt-mark hold`, and upgrade between runs. - Keep your setup in a script or a Dockerfile, so a reinstall or a second server goes through the same steps. - Save the output of `nvidia-smi` with each experiment, so results can be traced to a driver version. - After a kernel or driver update, reboot and run `nvidia-smi` before you start a long job, not after it fails. If something does not behave as expected after handover, our engineers can help with drivers, containers and frameworks as part of [expert onboarding](https://overgpu.com/products/expert-onboarding). ## Frequently asked questions ### Which operating systems can I choose? Ubuntu 22.04 or 24.04 with NVIDIA drivers, CUDA, Docker and the NVIDIA Container Toolkit preinstalled, ROCm images for AMD servers, and Debian or Rocky Linux on request. ### Which driver and CUDA versions are installed? They depend on the GPU and the image release, so we do not list fixed numbers. Run nvidia-smi after login to see the driver version and the highest CUDA version it supports, and tell us in the order notes if you need a specific driver branch. ### Do GPU containers work out of the box? Yes, on the Ubuntu images. Docker and the NVIDIA Container Toolkit are preinstalled, so docker run --gpus all gives a container access to the GPUs. ### Can I install my own driver or software? Yes. You have full root access over SSH and can install anything allowed by our acceptable use policy. ### Can you preinstall software such as vLLM or ComfyUI? Yes, at no extra cost. Pick it in the order form: our engineers install and test it on your image before delivery and tell you how to open it. Some applications run on NVIDIA GPUs only. ### Can I reinstall the operating system? Yes, on request through a support ticket from your dashboard, 24/7. Back up your data first, because local storage is not backed up by us unless agreed in writing. ## Related pages - [Ubuntu GPU image with CUDA preinstalled](https://overgpu.com/docs/os-images): What our Ubuntu GPU image with CUDA preinstalled contains, how to check driver and toolkit versions, pick 22.04 or 24.04 and keep the driver pinned safely. - [Check CUDA version and NVIDIA driver on Ubuntu](https://overgpu.com/docs/nvidia-drivers-cuda): Check NVIDIA driver and CUDA version with nvidia-smi and nvcc, see how they must match and add a CUDA toolkit on Ubuntu 24.04 without a new driver. - [ROCm setup for AMD Instinct MI355X](https://overgpu.com/docs/amd-rocm): Check GPUs with rocm-smi and amd-smi, set user groups, run PyTorch and vLLM in ROCm containers and test RCCL. - [Docker GPU containers with the NVIDIA Container Toolkit](https://overgpu.com/docs/docker-containers): Use --gpus, select devices, reserve GPUs in Compose, fix PyTorch shared memory and run ROCm containers. - [SSH into a GPU server](https://overgpu.com/docs/ssh-access): Create an ed25519 key on macOS, Linux or Windows, connect, add teammates and disable password login. Copy-ready steps. - [GPU server setup help from our engineers](https://overgpu.com/products/expert-onboarding): First-boot checks, environment and migration advice from our engineers, and sizing tools before you order.