SSH into a GPU server

To SSH into a GPU server from OverGPU, you need a key pair: the private key stays on your computer and the public key goes on the server. We install the public key you send with your order, so your first login works without a password. If you chose a root password instead, you can add your key after delivery. This page covers SSH key setup on macOS, Linux and Windows, a config entry, access for teammates and basic hardening. If your server is already delivered, the quickstart picks up after the first login.

Create an ed25519 key

ed25519 keys are short, fast and supported by every current OpenSSH release. The same command works on macOS, Linux and in PowerShell on Windows 10 or 11, which include the OpenSSH client:

ssh-keygen -t ed25519 -C "alice-laptop"

1. Press Enter to accept the default location: ~/.ssh/id_ed25519 on macOS and Linux, C:\Users\you\.ssh\id_ed25519 on Windows.
2. Choose a passphrase. It protects the key if your laptop is lost.
3. Display the public key:

   # macOS and Linux
   cat ~/.ssh/id_ed25519.pub
   # Windows PowerShell
   Get-Content $env:USERPROFILE\.ssh\id_ed25519.pub

   You should see a single line starting with ssh-ed25519. The text after -C is only a label.

Send the public key with your order

When you deploy a server, paste the whole .pub line into the order form, or select a key you saved in your dashboard. If you forgot, open a support ticket from your dashboard once the server is delivered. Share only the file ending in .pub: we only need the public key, and the private key should never leave your computer.

Chose a password instead?

If you picked password login when you ordered, connect with ssh root@203.0.113.10 and type the root password shown in your dashboard. Then copy your public key to the server, so you can switch to key login:

ssh-copy-id root@203.0.113.10

Windows has no ssh-copy-id; in PowerShell, run this instead:

Get-Content $env:USERPROFILE\.ssh\id_ed25519.pub | ssh root@203.0.113.10 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Once key login works, change the password with passwd, then turn off password login.

Connect to the server

ssh root@203.0.113.10

Use the IP address and username shown in your dashboard; this site uses root and the documentation address 203.0.113.10 in examples. On the first connection, SSH shows the server's host key fingerprint and asks you to confirm. Type yes and it is saved in ~/.ssh/known_hosts. You should land on a shell prompt, and whoami prints your username. If your key is not in the default location, add -i followed by its path.

Save the connection in ~/.ssh/config

A config entry saves typing and is reused by scp, rsync and VS Code. Add this to ~/.ssh/config on your computer (C:\Users\you\.ssh\config on Windows):

Host gpu1
    HostName 203.0.113.10
    User root
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes
    ServerAliveInterval 60

Now ssh gpu1 connects directly. IdentitiesOnly yes makes SSH offer only this key, which avoids "Too many authentication failures" when your agent holds several. ServerAliveInterval keeps idle sessions from being dropped.

Give teammates access

Give each person their own key, so you can revoke one without touching the others. Ask each teammate for their public key, then pick one approach.

Add a key to an existing account

nano ~/.ssh/authorized_keys    # paste the key on a new line and save
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Each line of authorized_keys is one key. To revoke access, delete the line.

Create a separate account (recommended)

sudo adduser --disabled-password bob
sudo mkdir -p /home/bob/.ssh
sudo nano /home/bob/.ssh/authorized_keys    # paste Bob's public key
sudo chown -R bob:bob /home/bob/.ssh
sudo chmod 700 /home/bob/.ssh
sudo chmod 600 /home/bob/.ssh/authorized_keys

adduser asks for a full name and other details; press Enter to skip them. The account has no password, so Bob can only log in with his key: ssh bob@203.0.113.10 should give him a shell. If he needs administrator rights, set a password he will use for sudo with sudo passwd bob, then run sudo usermod -aG sudo bob. The docker group is equivalent to root, so add only people you would trust with sudo.

Turn off password login

Key-only login shuts out password guessing. Check the effective settings first:

sudo sshd -T | grep -Ei \
  '^(passwordauthentication|kbdinteractiveauthentication|permitrootlogin)'

If passwordauthentication shows yes, add a drop-in file. OpenSSH keeps the first value it reads for most settings, and files in /etc/ssh/sshd_config.d/ are read in alphabetical order, so the 00- prefix makes these lines win:

sudo tee /etc/ssh/sshd_config.d/00-keys-only.conf >/dev/null <<'EOF'
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin prohibit-password
EOF
sudo sshd -t && sudo systemctl reload ssh

Keep your current session open. From a second terminal, confirm that key login still works and that password login is refused:

ssh gpu1 'echo key login works'
ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password gpu1

The second command should end with Permission denied (publickey).

Optional: fail2ban

With passwords off, scanners cannot get in, but they still fill your logs. fail2ban bans addresses after repeated failures, and Ubuntu's package enables its sshd jail by default:

sudo apt install -y fail2ban
sudo systemctl enable --now fail2ban
sudo fail2ban-client status sshd

You should see the sshd jail and its counters. If the service does not start, read sudo journalctl -u fail2ban; on minimal images without /var/log/auth.log, add backend = systemd under an [sshd] section in /etc/fail2ban/jail.local and restart it.

Port forwarding basics

SSH can carry other connections, so you can reach a service that listens only on the server's loopback interface without opening a port:

# your localhost:8888 reaches 127.0.0.1:8888 on the server
ssh -N -L 8888:127.0.0.1:8888 gpu1
# several ports at once
ssh -N -L 8888:127.0.0.1:8888 -L 8000:127.0.0.1:8000 gpu1

-N opens the tunnel without a remote shell; stop it with Ctrl+C. Remote development uses this for JupyterLab, and the firewall guide covers services that must be public.

Troubleshooting

• Permission denied (publickey). Run ssh -v gpu1 to see which keys are offered, and check the username. On the server, ~/.ssh must be mode 700 and authorized_keys mode 600, both owned by that user.
• REMOTE HOST IDENTIFICATION HAS CHANGED. Expected after a reinstall: remove the old entry with ssh-keygen -R 203.0.113.10. If the server was not reinstalled, do not connect, and open a ticket from your dashboard.
• Locked out after a change. Do not keep retrying. Open a support ticket from your dashboard with the server IP and what you changed.