# GPU server documentation

> Guides to connect to an OverGPU server and run workloads: SSH, drivers and CUDA, ROCm, Docker, vLLM, NCCL, monitoring, storage and firewalls.

Source: https://overgpu.com/docs  
Publisher: OverGPU (https://overgpu.com/)

## Get started

- [GPU server quickstart](https://overgpu.com/docs/quickstart): GPU server quickstart for new OverGPU servers. Log in over SSH, check the GPUs, update Ubuntu and run a first GPU container, with a check at each step.
- [SSH into a GPU server](https://overgpu.com/docs/ssh-access): How to SSH into a GPU server. Create an ed25519 key on macOS, Linux or Windows, connect, add teammates and disable password login. Copy-ready steps.
- [VS Code Remote SSH and Jupyter on a GPU server](https://overgpu.com/docs/remote-development): Use VS Code Remote SSH on your GPU server, run Jupyter on a remote GPU through an SSH tunnel and move data with rsync and scp. Safe defaults, step by step.
- [Ubuntu GPU image with CUDA preinstalled](https://overgpu.com/docs/os-images): What our Ubuntu GPU image with CUDA preinstalled contains, how to check driver and toolkit versions, pick 22.04 or 24.04 and keep the driver pinned safely.

## GPU software

- [Check CUDA version and NVIDIA driver on Ubuntu](https://overgpu.com/docs/nvidia-drivers-cuda): Check NVIDIA driver and CUDA version with nvidia-smi and nvcc, see how they must match and add a CUDA toolkit on Ubuntu 24.04 without a new driver.
- [ROCm setup for AMD Instinct MI355X](https://overgpu.com/docs/amd-rocm): ROCm setup for AMD Instinct MI355X servers. Check GPUs with rocm-smi and amd-smi, set user groups, run PyTorch and vLLM in ROCm containers and test RCCL.
- [Docker GPU containers with the NVIDIA Container Toolkit](https://overgpu.com/docs/docker-containers): Docker GPU guide for the NVIDIA Container Toolkit. Use --gpus, select devices, reserve GPUs in Compose, fix PyTorch shared memory and run ROCm containers.
- [Serve an LLM with vLLM](https://overgpu.com/docs/pytorch-vllm): Serve an LLM with vLLM on your GPU server. Set up Python with uv, install PyTorch and vLLM, test the OpenAI-compatible API and run it as a service.
- [NCCL tests on a multi-GPU server: all_reduce_perf step by step](https://overgpu.com/docs/multi-gpu-nccl): Run NCCL tests on a multi-GPU server. Check NVLink topology, build nccl-tests, read busbw from all_reduce_perf and launch torchrun jobs across 8 GPUs.
- [nvidia-smi commands for monitoring GPU servers](https://overgpu.com/docs/gpu-monitoring): nvidia-smi monitoring commands to monitor GPU usage on Linux. Live views, CSV logs, nvtop, DCGM exporter, ECC and Xid errors, plus rocm-smi equivalents.

## Server setup

- [Mount NVMe drives on Ubuntu](https://overgpu.com/docs/storage-nvme): Mount NVMe drives on Ubuntu. Find them with lsblk, format and add to fstab by UUID, or build an NVMe RAID 0 with mdadm. Plus health checks and backups.
- [GPU server firewall setup](https://overgpu.com/docs/networking-firewall): GPU server firewall setup with ufw. Allow SSH first, filter IPv6, avoid the Docker port trap and secure a vLLM endpoint with TLS and an API key.