SSH Hardening on a VPS: What Actually Reduces Risk (and What Doesn't)

Almost every "SSH hardening" checklist online says the same three things: change the port, install fail2ban, disable root login. Two of those barely matter, and one of them is often configured in a way that does nothing. This article separates the changes that genuinely shrink your attack surface from the ones that only shrink your log files, and shows the configuration and recovery steps that matter on a real VPS.

The short answer

On a fresh Linux VPS, three changes eliminate the overwhelming majority of realistic SSH risk:

1. Public key authentication only — PasswordAuthentication no and KbdInteractiveAuthentication no, verified with sshd -T, not just written into a file.
2. A restricted network path — SSH reachable only from an allowlisted IP range or only over a VPN interface, enforced by the firewall, not by sshd.
3. Patched sshd and a small config surface — unattended security updates, no legacy crypto, no agent forwarding, no unnecessary features enabled.

Everything else — non-standard ports, fail2ban, port knocking, TOTP prompts — is defence in depth or noise reduction. Useful in specific cases, but if password authentication is still enabled anywhere on the box, none of it saves you.

Why keys-only is the change that matters

Internet-wide SSH scanning is continuous and automated. Bots try root, admin, ubuntu, git, oracle with dictionary passwords across the entire IPv4 space. This traffic is not targeted at you; it is a background weather condition.

Public key authentication removes the entire class of attack. There is no secret transmitted that can be guessed, and Ed25519 or RSA-3072+ keys are not brute-forceable in any practical sense. Once passwords are off, a brute-force log entry is just wasted bandwidth on the attacker's side.

The catch is that "off" has to be actually true. Two common failure modes:

Ubuntu's config includes. Modern Ubuntu images ship /etc/ssh/sshd_config with Include /etc/ssh/sshd_config.d/*.conf near the top, and cloud images frequently drop a 50-cloud-init.conf containing PasswordAuthentication yes. Because sshd uses first-match-wins semantics, that include silently overrides the PasswordAuthentication no you edited 40 lines lower in the main file. Always confirm the effective config:

sudo sshd -T | grep -Ei 'passwordauth|kbdinteractive|permitrootlogin|pubkeyauth'

sshd -T prints the resolved configuration. If it disagrees with your edit, look in /etc/ssh/sshd_config.d/.

Per-user overrides and other services. A Match User deploy block, or a panel/container that runs its own sshd on another port, can reintroduce password auth. Check for extra listeners with ss -tlnp | grep sshd.

A baseline sshd configuration

Put your changes in a single file such as /etc/ssh/sshd_config.d/10-hardening.conf on distributions that support includes (and make sure it sorts before any cloud-init file, or delete that file if you don't need it):

PermitRootLogin prohibit-password
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey

AllowUsers admin deploy
MaxAuthTries 3
LoginGraceTime 20
MaxSessions 5

AllowAgentForwarding no
AllowTcpForwarding no
X11Forwarding no
PermitTunnel no

LogLevel VERBOSE

Notes on the choices:

• prohibit-password allows root login by key but not by password. Whether to permit root at all is less important than people think — if your unprivileged user has passwordless sudo, "no root login" is cosmetic. What matters is that no account accepts a password.
• AuthenticationMethods publickey is stricter than disabling password auth alone: it fails closed if some future include re-enables another method.
• AllowUsers (or AllowGroups) turns account enumeration into a non-issue. System accounts, database users and forgotten test accounts simply cannot authenticate.
• AllowTcpForwarding no is right for a plain server, but breaks ssh -L tunnels and ProxyJump through that host. If you rely on port forwarding to reach a database or admin panel, keep it on and restrict it per user with a Match block and PermitOpen.
• LogLevel VERBOSE records the key fingerprint used for each successful login. That is the single most useful log line during an incident review.

Validate and apply:

sudo sshd -t          # syntax check — never skip this
sudo systemctl reload ssh   # or sshd, depending on distro

Keep your current session open while you test a new connection from a second terminal. A syntax error plus a blind restart is the classic way to lock yourself out.

Socket activation changes how the port works

Recent Ubuntu releases start sshd through ssh.socket rather than a long-running daemon. If systemd owns the socket, Port 2222 in sshd_config is ignored. You have to override the socket unit:

sudo systemctl edit ssh.socket

[Socket]
ListenStream=
ListenStream=2222

Then sudo systemctl daemon-reload && sudo systemctl restart ssh.socket. If you change the port and cannot connect, this is usually why.

Keys: generation, storage and the parts people skip

Use Ed25519 unless you need compatibility with something ancient:

ssh-keygen -t ed25519 -C "laptop-2024"

Practical points that matter more than the algorithm:

• Always use a passphrase. A key file without one is a plaintext credential sitting in a home directory that gets backed up, synced and occasionally stolen. ssh-agent means you type it once per session.
• Hardware-backed keys are a real upgrade. ssh-keygen -t ed25519-sk stores the private key on a FIDO2 security key, so a compromised laptop cannot export it. Add -O resident if you want the credential recoverable from the token itself, and keep a second enrolled token as a spare.
• One key per device, not per server. Rotating or revoking a lost laptop then means deleting one line from authorized_keys files, not regenerating your identity.
• Never forward your agent to servers you don't fully control. Anyone with root on the remote host can use your agent to authenticate elsewhere while your session is open. Use ssh -J bastion target (ProxyJump) instead; it keeps the private key operations on your machine.
• Restrict what a key can do where it makes sense. Backup or CI keys belong in authorized_keys with prefixes like restrict,command="/usr/local/bin/backup-wrapper",from="203.0.113.0/24".

Verify the host key on first connect

This is the step nearly everyone skips, and it is the one that is unique to setting up a new server. The first time you connect, SSH asks you to accept a fingerprint. If you type yes blindly, you are trusting whatever the network hands you at that moment.

Get the fingerprint out of band — from your provider's console, serial output, or the cloud-init log visible in the control panel — and compare it:

# on the server, via console
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

Then pin it locally instead of relying on the prompt:

ssh-keyscan -t ed25519 server.example.net >> ~/.ssh/known_hosts

Set StrictHostKeyChecking accept-new in your client config so unknown hosts are recorded but changed keys hard-fail. A "REMOTE HOST IDENTIFICATION HAS CHANGED" warning after a legitimate rebuild is annoying; the same warning during an active interception attempt is the only signal you will get.

If you rebuild servers often, keep host keys in your backup set or regenerate them and re-pin deliberately. Testing that you can actually restore configuration like this is part of a working encrypted off-site backup routine.

Network gating beats brute-force banning

The most effective hardening step after keys-only is making port 22 invisible to the internet.

Option A: firewall allowlist. If your admin traffic comes from a static IP or a small range, allow only that. With nftables:

table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;
    ct state established,related accept
    iif lo accept
    ip saddr 203.0.113.10 tcp dport 22 accept
    tcp dport { 80, 443 } accept
  }
}

Option B: SSH only over a VPN. Bind sshd to an internal interface and reach it through an encrypted tunnel:

ListenAddress 10.8.0.1

Now SSH is not exposed publicly at all. This pairs naturally with a self-hosted WireGuard VPN on the same VPS, where the tunnel interface becomes your management network. The trade-off is a hard dependency: if WireGuard fails to start after a kernel upgrade, your only way in is the provider console. Keep console access working and tested before you commit to this design.

Where fail2ban and friends fit. With keys-only auth, banning IPs does not prevent compromise — there is nothing to guess. What it does is reduce log volume, CPU wakeups and connection churn, which is genuinely nice on a small VPS. Modern OpenSSH also includes PerSourcePenalties, which penalises misbehaving source addresses in the daemon itself; check whether your version has it enabled with sshd -T | grep -i persource. If you do run fail2ban, allowlist your own networks (ignoreip) — locking yourself out because a script fumbled three connections is a common self-inflicted outage.

Changing the port cuts scanner noise dramatically and costs nothing, but treat it as tidying, not security. Anyone doing a targeted scan finds a service on 2222 in seconds.

Crypto, patching and exposure

Defaults in current OpenSSH releases are sound; the value is in removing legacy options rather than inventing your own list. Audit rather than guess — ssh-audit (open source, runs against a host or locally) reports which key exchange, cipher and MAC algorithms your daemon offers and flags weak ones.

Two things worth knowing:

• Recent OpenSSH versions negotiate a hybrid post-quantum key exchange by default (the sntrup761x25519 and, in newer releases, mlkem768x25519 methods). This protects against future decryption of recorded traffic. You get it by keeping OpenSSH current, not by hand-editing cipher lists.
• Remote pre-authentication vulnerabilities in sshd are rare but not hypothetical — CVE-2024-6387 in 2024 was exactly that. The defence is boring: enable unattended security updates, and keep the daemon off the public internet where practical. Both reduce the window where a fresh CVE matters.

sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

Common mistakes

Editing config and restarting without a second session. Test in parallel. Better, add a rollback timer before you touch anything risky:

sudo sh -c 'sleep 300 && cp /etc/ssh/sshd_config.bak /etc/ssh/sshd_config && systemctl reload ssh' &

Cancel it once you have confirmed a new login works.

Assuming the config file you edited is the one in effect. Includes, Match blocks and socket units all override intuition. sshd -T is authoritative.

Adding TOTP on top of keys and calling it 2FA. A key plus a hardware token is already two factors, and it is stronger than a key plus a shared TOTP secret stored on the same server. TOTP is worth adding when passwords cannot be eliminated, or when you want an interactive second step for shared administrative accounts — configured as AuthenticationMethods publickey,keyboard-interactive, never as an alternative to keys.

Leaving authorized_keys unmonitored. Persistence via an added key is quiet and survives password changes. Periodically diff the file, or track it in configuration management so unexpected entries stand out.

Confusing SSH hardening with server privacy. A perfectly locked-down sshd tells you nothing about who else can read the disk. Whoever operates the hypervisor can, in principle, access a running VM's memory and storage — which is why full disk encryption on a VPS has real but limited value, and why choosing a provider and jurisdiction is a separate decision from configuring your daemon.

What SSH hardening does not protect against

Be honest about the boundaries:

• Application-layer compromise. If your web app has an RCE, the attacker does not need SSH.
• Compromised client devices. Malware on your laptop can use your agent, your terminal and your session. Hardware keys limit exfiltration of the key material but not misuse while you are logged in.
• Supply-chain and dependency attacks. The xz backdoor of 2024 targeted sshd through a library, not through its configuration.
• Provider-level access. Hypervisor operators are outside your threat model's reach. If that matters to you, it is an operational and jurisdictional question — including how you sign up and pay, which is why some users prefer anonymous, no-KYC provisioning paid in Monero — not something sshd_config can fix.

FAQ

Should I disable root login entirely? prohibit-password is enough. Full PermitRootLogin no is slightly better hygiene for shared teams because it forces an audit trail through named accounts, but it provides little extra protection when those accounts hold passwordless sudo.

Is port knocking worth it? Rarely. A firewall allowlist or a VPN gives the same "invisible service" result with fewer moving parts and no risk of a knock daemon failing silently.

How do I get back in if I lock myself out? Through the provider's console or VNC, or by booting a rescue system and mounting the disk to fix sshd_config and authorized_keys. Confirm that this path works before you need it — including whether you know the root password required at a console prompt.

Do I need fail2ban if passwords are disabled? Not for security. For log hygiene and reduced load, it is reasonable. Allowlist your own addresses.

Is Ed25519 or RSA better? Ed25519: smaller, faster, no parameter choices to get wrong. Use RSA-3072 or larger only for compatibility with old systems.

Takeaway

SSH hardening is not a long checklist. Make authentication keys-only and verify it with sshd -T, restrict who can reach the port at the network layer, keep the daemon patched, and pin host keys so you would notice interception. Everything after that is optional refinement — and none of it substitutes for the boring parts: a tested recovery path, monitored authorized_keys, and a realistic view of what a locked-down daemon can and cannot protect.