How to Lock Down SSH Access on a VPS Without Locking Yourself Out
To harden SSH on a VPS, you switch authentication from passwords to public keys, then restrict who can log in and from where, and you do it in an order that keeps an existing session open while you test each change in a second one. The single most important habit is never closing your current working connection until a brand new one has succeeded with the new settings.
The reason this matters is that SSH is the front door to almost everything else on the box. Once you disable password authentication, a misconfigured key or a typo in the config file stops being an annoyance and becomes a lockout, and on many providers a full reinstall is the only recovery. Everything below is arranged so that the risky step comes last and is easy to undo.
Understand what you are changing before you harden SSH on a VPS
The SSH protocol has two broad phases: the transport layer negotiates encryption and verifies the server's host key, and then the authentication layer decides whether the client is allowed in. The server's behaviour is driven by /etc/ssh/sshd_config and by drop-in files under /etc/ssh/sshd_config.d/. On most current distributions the main file ends with an Include line, and drop-ins are read in lexical order, with the first value set for a given keyword winning in the common case. That ordering trips people up: a directive you add to the main file can be silently overridden by an earlier drop-in.
Before editing anything, confirm which files are actually in play and back them up:
sudo sshd -T | grep -Ei '^(passwordauthentication|permitrootlogin|pubkeyauthentication|port|allowusers)'
sudo cp -a /etc/ssh /etc/ssh.bak
The sshd -T command prints the effective configuration after all includes and defaults are applied, which is exactly what the running daemon will use on its next start. If the output does not match what you think you wrote, you are editing the wrong file.
Get keys working before you turn passwords off
Generate a key pair on your workstation if you do not already have one. Ed25519 is a sensible default; it is compact and widely supported.
ssh-keygen -t ed25519 -a 100 -C "you@laptop"
ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]
The second command appends your public key to ~/.ssh/authorized_keys on the server and fixes the permissions on that directory. Permissions are a common silent failure: sshd refuses to use an authorized_keys file that is group or world writable, or whose parent directory is. The home directory itself must not be writable by anyone but the owner. If key login fails with a prompt for a password anyway, check the server log with sudo journalctl -u ssh -n 50 or sudo tail /var/log/auth.log; it will usually say plainly that it ignored the key file because of ownership or mode.
Test the key login in a new terminal while your original session stays open:
ssh -o PreferredAuthentications=publickey -i ~/.ssh/id_ed25519 [email protected]
Forcing the authentication method proves the key path works rather than falling back to a password you happened to type. Only when this succeeds should you touch the config.
Disabling passwords and limiting access
Now edit the config. Put your changes in a drop-in so they are easy to find and easy to remove, and give it a name that sorts before any provider-supplied file you do not control.
# /etc/ssh/sshd_config.d/10-hardening.conf
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitRootLogin prohibit-password
MaxAuthTries 3
LoginGraceTime 20
KbdInteractiveAuthentication matters because some distributions route password prompts through the keyboard-interactive path even when PasswordAuthentication is off, so disabling both closes the gap. PermitRootLogin prohibit-password blocks root from using a password while still allowing key-based root login; set it to no if you would rather log in as an ordinary user and use sudo.
Then validate the file before reloading. This step catches syntax errors that would otherwise take the daemon down on restart:
sudo sshd -t
sudo systemctl reload ssh
The reload, rather than a restart, keeps existing sessions alive. Your current connection stays up even if the new config is wrong, which is the whole point. Open a brand new terminal and log in again to confirm keys still work and passwords are refused.
To narrow access further, use AllowUsers or AllowGroups to name exactly who may connect, and consider binding the daemon to a specific address with ListenAddress if the box has several. A firewall rule that admits SSH only from addresses you control is worth more than any config tweak, but treat it with the same caution: add the rule, confirm it works, then remove the old permissive one.
Keep a way back in
Before you reload with password logins disabled, make sure the provider's out-of-band console works and that you know how to reach it. Most VPS control panels offer a serial or VNC console that bypasses SSH entirely, and that is your recovery path if you lock yourself out. Log into it once while everything is healthy so you are not learning the interface under pressure.
Keep the original session open until a fresh login has succeeded. If something breaks, you can revert from that session:
sudo rm /etc/ssh/sshd_config.d/10-hardening.conf
sudo systemctl reload ssh
Restoring the backup you made earlier, sudo cp -a /etc/ssh.bak/. /etc/ssh/, achieves the same thing if you changed more than one file. Either way, the recovery is a single command in a session you never closed.
What to do next
Once key-only login is confirmed working, review the rest of the effective config with sshd -T and trim anything you do not need, such as agent or TCP forwarding on a box that should not have it. Then set up a second key on a different machine or a hardware token so a lost laptop is not a lost server, and note where your provider's console lives. Make these changes one at a time, testing a fresh login after each, and you will never have to guess which edit locked you out.
