Are the NVMe data drives mounted when I receive the server?

Not always. Depending on your order, the data drives on 8-GPU servers may be delivered unmounted, so check with lsblk and follow the NVMe storage guide to format and mount them.

Is a single-GPU plan a virtual machine?

Yes. Single-GPU plans are a dedicated virtual machine with the full GPU passed through, while 8-GPU servers are bare metal. In both cases the GPU is assigned to you alone.

Can I change the operating system after delivery?

Yes. Request a reinstall from the server's page in your dashboard, which opens a ticket. Copy your data off first, because local storage is not backed up by us.

GPU Server Quickstart: From Delivery to First Job

Everything between your server's delivery and your first GPU job. Log in, check the GPUs, update, run a container and keep work running after you disconnect.

This GPU server quickstart takes you from delivery to a first workload running on your GPUs. It applies to every OverGPU server: single-GPU plans, which are a dedicated virtual machine with the full GPU passed through, and 8-GPU servers, which are bare metal. Each step ends with a check, so you know it worked before you move on.

What you receive

When your server is ready, its IP address and SSH details appear in your dashboard. Access is over SSH with the public key you added to your order, and you have full root access. Every server has a public IPv4 address and an IPv6 /64, with DDoS mitigation on its public IPs.

Our Ubuntu 22.04 and 24.04 images come with the NVIDIA driver, CUDA, Docker and the NVIDIA Container Toolkit preinstalled. AMD Instinct servers run our ROCm images. If you have not sent a public key yet, start with SSH into a GPU server.

Log in over SSH

Open a terminal on your computer (PowerShell on Windows) and connect with the IP address and username shown in your dashboard. Examples on this site use root and the documentation address 203.0.113.10:
```
ssh root@203.0.113.10
```
On the first connection, SSH asks you to confirm the host key. Type yes.
Check the operating system:
```
cat /etc/os-release
```
You should see the Ubuntu version you ordered.

Commands on this page use sudo. It works the same when you are logged in as root.

Check that every GPU is visible

On NVIDIA servers:

nvidia-smi
nvidia-smi -L

You should see one row per GPU (one on a single-GPU plan, eight on an 8-GPU server), with the driver version and a "CUDA Version" in the header. That CUDA Version is the newest CUDA release the driver supports, not necessarily the toolkit that is installed. nvidia-smi -L lists each GPU with its UUID.

On AMD Instinct MI355X servers:

rocm-smi

You should see eight GPUs. The ROCm setup guide covers amd-smi, rocminfo and user groups. If a GPU is missing on any server, note which one and open a ticket from your dashboard before you start work.

Update the system

sudo apt update
sudo apt upgrade -y
[ -f /var/run/reboot-required ] && echo "Reboot required"

If a dialog asks about a modified configuration file, keep the version currently installed. If the last command prints "Reboot required", run sudo reboot, wait a few minutes, log in again and rerun nvidia-smi.

Always reboot after an update that touches the NVIDIA driver. Until you do, nvidia-smi can fail with a "Driver/library version mismatch" error, because the old kernel module is still loaded.

Run a first container on the GPU

On NVIDIA servers, this starts a plain Ubuntu container with access to all GPUs:

docker run --rm --gpus all ubuntu nvidia-smi

You should see the same table as on the host. The NVIDIA Container Toolkit mounts the host's driver into the container, so the image itself needs no driver. As a non-root user, prefix docker with sudo.

On AMD servers, pass the GPU device files instead (install Docker first if your image does not include it):

docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video \
  rocm/pytorch:latest \
  python3 -c "import torch; print(torch.cuda.device_count())"

It should print 8. The image is large, so the first pull takes a while. Docker GPU containers covers selecting GPUs, Docker Compose and shared memory for PyTorch.

Keep jobs running with tmux

A process started in a plain SSH session stops when the connection drops. Run long jobs inside tmux:

Start a named session with tmux new -s work. If tmux is missing, install it with sudo apt install -y tmux.
Start your job, for example python3 train.py.
Detach by pressing Ctrl+B, then D. The job keeps running on the server.
Later, list sessions with tmux ls and reattach with tmux attach -t work.

Anything that must survive a reboot, such as an inference endpoint, belongs in a systemd service instead. The guide to serving an LLM with vLLM includes one.

Troubleshooting

Permission denied (publickey). SSH did not offer the right key, or the username is wrong. Point to your key with ssh -i ~/.ssh/id_ed25519 root@203.0.113.10 and check the username shown in your dashboard.
nvidia-smi fails after an update. Reboot first. If it still fails, collect the output of sudo dmesg | grep -i nvidia and include it in a ticket from your dashboard.
could not select device driver "" with capabilities: [[gpu]]. Docker is not set up for the NVIDIA runtime. Run sudo nvidia-ctk runtime configure --runtime=docker, then sudo systemctl restart docker.

Where to go next

Mount your NVMe data drives. On 8-GPU servers they may arrive unmounted, depending on your order.
Set up the firewall before you expose any service to the internet.
Connect VS Code or Jupyter to work on the server from your laptop.

Stuck at any step? Our engineers answer 24/7: open a support ticket from your dashboard.