# GPU server firewall setup > GPU server firewall setup with ufw. Allow SSH first, filter IPv6, avoid the Docker port trap and secure a vLLM endpoint with TLS and an API key. Source: https://overgpu.com/docs/networking-firewall Updated: 2026-10-22 Publisher: OverGPU (https://overgpu.com/) A GPU server firewall setup has one goal: only the ports you mean to open are reachable. Every OverGPU server has a public IPv4 address and an IPv6 /64, so anything listening on all interfaces is exposed unless you filter it. This page sets up ufw, explains how Docker bypasses it, then shows how to secure a vLLM endpoint with a reverse proxy, TLS and an API key. ## Set up ufw, allowing SSH first Allow SSH before you enable the firewall, or you will lock yourself out: ``` sudo apt install -y ufw sudo ufw allow OpenSSH sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw enable sudo ufw status verbose ``` Answer `y` when ufw warns that the command may disrupt SSH connections. You should see `Status: active`, the default policies, and a rule allowing `22/tcp (OpenSSH)` for IPv4 and IPv6. Keep your session open and test a new SSH connection from a second terminal. With a fixed office or VPN address, you can restrict SSH to it. Add the new rule before deleting the old one: ``` sudo ufw allow from 198.51.100.7 to any port 22 proto tcp sudo ufw delete allow OpenSSH ``` A softer option is `sudo ufw limit OpenSSH`, which rate-limits repeated connections from one address. Key-only login, covered in [SSH access](https://overgpu.com/docs/ssh-access), matters more than either. ## Filter IPv6 too ``` grep IPV6 /etc/default/ufw sudo ss -tulpn ``` ufw filters IPv6 when `/etc/default/ufw` contains `IPV6=yes`, the Ubuntu default. With `IPV6=no`, ufw leaves IPv6 unfiltered and services listening on `[::]` stay reachable through your /64. Set it to `yes`, run `sudo ufw disable && sudo ufw enable`, then add your rules again so they get IPv6 copies. `ss -tulpn` lists every listening socket with its process: anything on `0.0.0.0`, `*` or `[::]` listens on all interfaces. Make sure you need it there and that the firewall covers it, or bind it to `127.0.0.1`. ## Docker bypasses ufw for published ports Docker manages its own iptables rules. When you publish a port with `-p 8000:8000`, Docker redirects that traffic to the container through its own rules, and it never passes the ufw rules that protect the host. The port is open to the internet even with `default deny incoming`, which catches many vLLM, Jupyter and monitoring setups. 1. Publish on the loopback interface only: `-p 127.0.0.1:8000:8000`, or `"127.0.0.1:8000:8000"` in Compose. 2. Check the binding with `sudo ss -tlnp | grep 8000`. It should show `127.0.0.1:8000`. 3. From your laptop, `curl http://203.0.113.10:8000/v1/models` (with your server's IP) should fail to connect. 4. Expose the service through a reverse proxy on the host, which ufw does filter. To make loopback the default for every published port, add `"ip": "127.0.0.1"` to `/etc/docker/daemon.json`, keeping existing entries such as the NVIDIA runtime, then restart Docker. For finer control, Docker evaluates the `DOCKER-USER` iptables chain before its own rules. ## Put a reverse proxy with TLS in front Point a DNS name (here `llm.example.com`) at your server's addresses and open the web ports, then pick Caddy or nginx. Both configurations below expose only the `/v1/` API paths. ``` sudo ufw allow 80,443/tcp ``` ### Caddy Caddy obtains and renews certificates automatically. It is packaged in Ubuntu 24.04 (`sudo apt install -y caddy`); on 22.04, install it from Caddy's own repository. Replace the contents of `/etc/caddy/Caddyfile` with: ``` llm.example.com { handle /v1/* { reverse_proxy 127.0.0.1:8000 } handle { respond 404 } } ``` Apply it with `sudo systemctl reload caddy`. Once the certificate is issued, `https://llm.example.com/v1/models` answers over HTTPS. ### nginx ``` sudo apt install -y nginx certbot python3-certbot-nginx ``` Create `/etc/nginx/sites-available/llm`: ``` server { listen 80; server_name llm.example.com; location /v1/ { proxy_pass http://127.0.0.1:8000; proxy_set_header Host $host; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_buffering off; proxy_read_timeout 600s; } location / { return 404; } } ``` Enable it, then request a certificate. certbot adds the HTTPS configuration and sets up renewal: ``` sudo ln -s /etc/nginx/sites-available/llm /etc/nginx/sites-enabled/ sudo nginx -t && sudo systemctl reload nginx sudo certbot --nginx -d llm.example.com ``` `proxy_buffering off` lets streamed tokens reach clients as they are generated, and the longer timeout covers long generations. ## Require an API key on OpenAI-compatible servers TLS protects traffic in transit; an API key decides who may use the model. vLLM requires a bearer token when you start it with `--api-key` or set `VLLM_API_KEY`: ``` openssl rand -hex 32 # generate a key vllm serve Qwen/Qwen2.5-7B-Instruct --host 127.0.0.1 --port 8000 \ --api-key YOUR_KEY ``` Test from your laptop: ``` curl https://llm.example.com/v1/models -H "Authorization: Bearer YOUR_KEY" curl -i https://llm.example.com/v1/models ``` The first command should list the model and the second should return status 401. vLLM applies the key to the `/v1` routes, and other routes such as `/metrics` may stay open, which is why the proxy exposes nothing else. Prefer the environment variable, as in the systemd setup from [serving an LLM with vLLM](https://overgpu.com/docs/pytorch-vllm): a key typed on the command line is visible to other local users in the process list. Other OpenAI-compatible servers have similar options. ## Private networking between your servers On request, we connect your servers on a private network. Use it for traffic between them: data sync, NCCL and torchrun rendezvous, internal APIs and metrics. Allow it in ufw by subnet, or by interface (find it with `ip -br addr`): ``` sudo ufw allow from 10.0.0.0/24 # or sudo ufw allow in on PRIVATE_INTERFACE ``` Replace the subnet or interface with those of your private network. See [private networking](https://overgpu.com/products/private-networking) to request it. ## DDoS mitigation is included Every public IP address on your servers has DDoS mitigation, included in the price. It protects against network-level attacks; it does not replace a firewall, TLS, authentication or rate limits on your own services. If you see unusual traffic or think your server is under attack, [open a support ticket](https://overgpu.com/help/getting-support) from your dashboard. More on [DDoS protection](https://overgpu.com/products/ddos-protection). ## Troubleshooting - **Locked out after enabling ufw.** Do not keep retrying. Open a ticket from your dashboard (it does not need SSH) with the server and the rules you added; see [how tickets work](https://overgpu.com/help/getting-support). - **Caddy or certbot cannot get a certificate.** Check that the DNS name points to the server, that ports 80 and 443 are allowed in ufw, and that nothing else listens on them (`sudo ss -tlnp`). - **A container port is reachable despite ufw.** It is published on all interfaces. Recreate the container with `-p 127.0.0.1:PORT:PORT`. ## Related pages - [GPU server DDoS protection, included on every public IP](https://overgpu.com/products/ddos-protection): Learn what it covers, what it does not, and how to harden inference APIs. - [Private network between GPU servers](https://overgpu.com/products/private-networking): Link your OverGPU servers on an isolated network for data pipelines and multi-node jobs. See how it works. - [Serve an LLM with vLLM](https://overgpu.com/docs/pytorch-vllm): Set up Python with uv, install PyTorch and vLLM, test the OpenAI-compatible API and run it as a service. - [SSH into a GPU server](https://overgpu.com/docs/ssh-access): Create an ed25519 key on macOS, Linux or Windows, connect, add teammates and disable password login. Copy-ready steps.