# GPU server quickstart > GPU server quickstart for new OverGPU servers. Log in over SSH, check the GPUs, update Ubuntu and run a first GPU container, with a check at each step. Source: https://overgpu.com/docs/quickstart Updated: 2026-10-22 Publisher: OverGPU (https://overgpu.com/) This GPU server quickstart takes you from delivery to a first workload running on your GPUs. It applies to every OverGPU server: single-GPU plans, which are a dedicated virtual machine with the full GPU passed through, and 8-GPU servers, which are bare metal. Each step ends with a check, so you know it worked before you move on. ## What you receive When your server is ready, its IP address and SSH details appear in your dashboard. Access is over SSH with the public key you added to your order, and you have full root access. Every server has a public IPv4 address and an IPv6 /64, with DDoS mitigation on its public IPs. Our Ubuntu 22.04 and 24.04 images come with the NVIDIA driver, CUDA, Docker and the NVIDIA Container Toolkit preinstalled. AMD Instinct servers run our ROCm images. If you have not sent a public key yet, start with [SSH into a GPU server](https://overgpu.com/docs/ssh-access). ## Log in over SSH 1. Open a terminal on your computer (PowerShell on Windows) and connect with the IP address and username shown in your dashboard. Examples on this site use `root` and the documentation address `203.0.113.10`: `ssh root@203.0.113.10` 2. On the first connection, SSH asks you to confirm the host key. Type `yes`. 3. Check the operating system: `cat /etc/os-release` You should see the Ubuntu version you ordered. Commands on this page use `sudo`. It works the same when you are logged in as root. ## Check that every GPU is visible On NVIDIA servers: ``` nvidia-smi nvidia-smi -L ``` You should see one row per GPU (one on a single-GPU plan, eight on an 8-GPU server), with the driver version and a "CUDA Version" in the header. That CUDA Version is the newest CUDA release the driver supports, not necessarily the toolkit that is installed. `nvidia-smi -L` lists each GPU with its UUID. On AMD Instinct MI355X servers: ``` rocm-smi ``` You should see eight GPUs. The [ROCm setup guide](https://overgpu.com/docs/amd-rocm) covers amd-smi, rocminfo and user groups. If a GPU is missing on any server, note which one and open a ticket from your dashboard before you start work. ## Update the system ``` sudo apt update sudo apt upgrade -y [ -f /var/run/reboot-required ] && echo "Reboot required" ``` If a dialog asks about a modified configuration file, keep the version currently installed. If the last command prints "Reboot required", run `sudo reboot`, wait a few minutes, log in again and rerun `nvidia-smi`. Always reboot after an update that touches the NVIDIA driver. Until you do, nvidia-smi can fail with a "Driver/library version mismatch" error, because the old kernel module is still loaded. ## Run a first container on the GPU On NVIDIA servers, this starts a plain Ubuntu container with access to all GPUs: ``` docker run --rm --gpus all ubuntu nvidia-smi ``` You should see the same table as on the host. The NVIDIA Container Toolkit mounts the host's driver into the container, so the image itself needs no driver. As a non-root user, prefix `docker` with `sudo`. On AMD servers, pass the GPU device files instead (install Docker first if your image does not include it): ``` docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video \ rocm/pytorch:latest \ python3 -c "import torch; print(torch.cuda.device_count())" ``` It should print `8`. The image is large, so the first pull takes a while. [Docker GPU containers](https://overgpu.com/docs/docker-containers) covers selecting GPUs, Docker Compose and shared memory for PyTorch. ## Keep jobs running with tmux A process started in a plain SSH session stops when the connection drops. Run long jobs inside tmux: 1. Start a named session with `tmux new -s work`. If tmux is missing, install it with `sudo apt install -y tmux`. 2. Start your job, for example `python3 train.py`. 3. Detach by pressing Ctrl+B, then D. The job keeps running on the server. 4. Later, list sessions with `tmux ls` and reattach with `tmux attach -t work`. Anything that must survive a reboot, such as an inference endpoint, belongs in a systemd service instead. The guide to [serving an LLM with vLLM](https://overgpu.com/docs/pytorch-vllm) includes one. ## Troubleshooting - **Permission denied (publickey).** SSH did not offer the right key, or the username is wrong. Point to your key with `ssh -i ~/.ssh/id_ed25519 root@203.0.113.10` and check the username shown in your dashboard. - **nvidia-smi fails after an update.** Reboot first. If it still fails, collect the output of `sudo dmesg | grep -i nvidia` and include it in a ticket from your dashboard. - **could not select device driver "" with capabilities: [[gpu]].** Docker is not set up for the NVIDIA runtime. Run `sudo nvidia-ctk runtime configure --runtime=docker`, then `sudo systemctl restart docker`. ## Where to go next - [Mount your NVMe data drives](https://overgpu.com/docs/storage-nvme). On 8-GPU servers they may arrive unmounted, depending on your order. - [Set up the firewall](https://overgpu.com/docs/networking-firewall) before you expose any service to the internet. - [Connect VS Code or Jupyter](https://overgpu.com/docs/remote-development) to work on the server from your laptop. Stuck at any step? Our engineers answer 24/7: [open a support ticket](https://overgpu.com/help/getting-support) from your dashboard. ## Frequently asked questions ### Are the NVMe data drives mounted when I receive the server? Not always. Depending on your order, the data drives on 8-GPU servers may be delivered unmounted, so check with lsblk and follow the NVMe storage guide to format and mount them. ### Is a single-GPU plan a virtual machine? Yes. Single-GPU plans are a dedicated virtual machine with the full GPU passed through, while 8-GPU servers are bare metal. In both cases the GPU is assigned to you alone. ### Can I change the operating system after delivery? Yes. Request a reinstall from the server's page in your dashboard, which opens a ticket. Copy your data off first, because local storage is not backed up by us. ## Related pages - [Check CUDA version and NVIDIA driver on Ubuntu](https://overgpu.com/docs/nvidia-drivers-cuda): Check NVIDIA driver and CUDA version with nvidia-smi and nvcc, see how they must match and add a CUDA toolkit on Ubuntu 24.04 without a new driver. - [Ubuntu GPU image with CUDA preinstalled](https://overgpu.com/docs/os-images): What our Ubuntu GPU image with CUDA preinstalled contains, how to check driver and toolkit versions, pick 22.04 or 24.04 and keep the driver pinned safely. - [nvidia-smi commands for monitoring GPU servers](https://overgpu.com/docs/gpu-monitoring): Live views, CSV logs, nvtop, DCGM exporter, ECC and Xid errors, plus rocm-smi equivalents. - [How long to get a GPU server](https://overgpu.com/help/delivery-and-provisioning): Single-GPU and 8-GPU servers are delivered in under 10 minutes after payment. Here are the five steps to root access.