Hardening SSH Beyond the Basics: Keys, Certs, and 2FA
Past disabling passwords lies a quieter, sturdier front door
Contents
Every guide to securing SSH opens with the same three commands: generate a key, disable password login, change the port. That advice is genuinely good, and if you have done all three you have already shut out the overwhelming bulk of the noise that hammers port 22 every second of every day. The trouble is that most people stop there and assume the job is finished. It is closer to finished than it is to started, but the interesting problems begin exactly where the beginner guides end.
The reason to keep going is scale. One key on one laptop connecting to one server is trivial to secure. The moment you have six machines, three of your own devices, and a habit of spinning up a new VM every fortnight, the humble authorized_keys file stops being a security control and becomes a filing problem. Who has access to what? When someone’s laptop is lost, how many authorized_keys files do you have to comb through to revoke it? That question is what pushes serious setups towards SSH certificates. Before we get there, let me clear up the foundations, because a certificate authority sitting on top of sloppy key hygiene just gives you sloppy hygiene with extra steps.
Keys, done properly
If you generated an RSA key years ago and never revisited it, generate a fresh Ed25519 key today. Ed25519 keys are small, fast, and rely on a curve that has held up extremely well since it became widely available. The signing algorithm avoids a class of implementation mistakes that plagued older schemes, and there is no practical downside for the homelab.
| |
The -a 100 sets the number of KDF rounds used to protect the private key on disk, so that a stolen key file is expensive to brute-force even if your passphrase is mediocre. And yes, use a passphrase. A private key with no passphrase is a plaintext password sitting in a dotfile, and the first thing any attacker who lands on your machine does is copy ~/.ssh/ wholesale.
On the server side, the two settings that matter live in /etc/ssh/sshd_config. Turn off password authentication entirely and turn off keyboard-interactive too, because leaving the latter on quietly re-opens the password path on some distributions:
| |
PermitRootLogin prohibit-password is the sensible middle ground. It allows key-based root login for automation that genuinely needs it while forbidding anyone from typing a root password over the wire. If you have no such automation, no is stricter and better. Whatever you choose, understanding why each line is there matters more than copying the block, because the day something breaks you need to know which knob you turned.
The reverse-proxy of the SSH world: certificates
Here is the mental model that made certificates finally click for me. An authorized_keys file is a guest list at a door: every key that might ever knock has to be written down in advance, on every door. A certificate authority is a bouncer with a stamp. You trust the bouncer once; the bouncer stamps a wristband on anyone allowed in that night; the door only checks for a valid stamp. Add a new guest and you never touch the door — you tell the bouncer. That single indirection is what turns SSH access from a filing problem back into a security control.
Concretely, you create a CA key that exists only to sign other keys. It never logs into anything itself. You keep it offline and treated like the crown jewels it is.
| |
Every server you own gets one line telling it to trust certificates signed by that CA. Copy the public half of the CA key to each host and point sshd at it:
| |
Now, when someone needs access, you sign their existing public key and hand back a certificate. The certificate carries a validity window, a list of principals (think of these as roles or usernames the certificate is good for), and a key ID that shows up in the logs:
| |
That command produces ~/.ssh/id_ed25519-cert.pub. The client sends it automatically alongside the key. The flags earn their keep: -I is the identity stamped into every audit log line, so you can trace a session back to a specific person and device; -n smarc,deploy limits which usernames the certificate may assume; and -V +8w makes it expire in eight weeks. That expiry is the quiet superpower. A leaked certificate stops working on its own, and a laptop that falls in a canal takes its access to the grave within two months whether or not you remember to revoke anything.
If you want revocation faster than expiry, sshd reads a revocation list:
| |
You add the compromised key or certificate serial to that file, push it out, and the door slams immediately. For a small estate the expiry window alone is usually enough, and the revocation list is the emergency brake you hope never to pull.
The payoff compounds with host certificates too. Sign each server’s host key with a host CA, drop the CA’s public half into your client known_hosts with a @cert-authority line, and the “The authenticity of host … can’t be established” prompt disappears for good. You stop training yourself to hit yes blindly, which is worth more than it sounds, because that reflex is exactly what a man-in-the-middle relies on. Getting the reverse-proxy layer of your homelab tidy pairs well with a broader look at what you are actually defending, which I dig into in a homelab threat model.
A genuine second factor
Certificates prove you hold a key. A second factor proves it is really you holding it. These are different questions, and the gap between them is where a stolen-but-unlocked laptop lives. There are two honest ways to add a second factor to SSH, and they suit different tastes.
The first keeps the key itself on hardware. A security key that speaks the FIDO2 protocol can hold an SSH credential that physically cannot be copied off the device, and that requires a touch to authorise each use:
| |
The -sk suffix means the private key material stays on the security key; your disk only holds a stub that is useless without the hardware. verify-required adds a PIN on top of the touch. This is my preferred approach because it collapses “something you have” and “something you are/know” into a single tap, and the private key never touches a general-purpose computer where malware could grab it. I wrote up the full hardware-token workflow — SSH, commit signing, and sudo — in a Yubikey for SSH, GPG, and sudo if you want to go that route.
The second way bolts a time-based one-time password onto the login via PAM, so you type a six-digit code after your key is accepted. It works without any special hardware, which is its whole appeal. The wiring uses libpam-google-authenticator and two edits. In /etc/pam.d/sshd you add the module, and in sshd_config you tell SSH to demand a key and the second factor together:
| |
| |
The comma in publickey,keyboard-interactive is doing critical work: it means both are required, in sequence. A leaked key alone no longer gets anyone in; they would also need the rotating code seeded on your phone. The catch is that you have just re-enabled keyboard-interactive, so be deliberate about it, and make sure the TOTP prompt is the only thing that path can reach.
When it breaks: troubleshooting
The single most useful habit when SSH refuses to cooperate is to stop guessing and ask it what it is thinking. Verbose client output shows exactly which keys and certificates were offered and how the server responded:
| |
Read it from the top. You will see lines like Offering public key and Server accepts key, or the far more common Authentications that can continue: publickey, which tells you the server rejected whatever you sent and is waiting for something else.
Certificate ignored, falls back to plain key. Check the certificate is actually valid and not expired with ssh-keygen -L -f ~/.ssh/id_ed25519-cert.pub. Read the Valid: line; a certificate whose window has passed is silently skipped. The other classic cause is a principal mismatch — you signed for -n alice but are logging in as bob. The principal list has to contain the username you are connecting as.
Locked out after enabling AuthenticationMethods. This is the one that produces genuine panic. Before you touch that setting, open a second SSH session and leave it connected. Edit and reload sshd in the first; test the new rules in a third. If the third fails, your still-open second session lets you undo the change. On a physical or console-accessible box this matters less, but for a remote VM it is the difference between a quick fix and a support ticket.
sign_and_send_pubkey: signing failed for … agent refused operation. Almost always a permissions problem on the private key, or an agent that has forgotten it. chmod 600 the key, confirm ssh-add -l lists it, and re-add it if not. Hardware keys throw this when you forget to physically touch the device — the operation is genuinely waiting for your finger.
Host key warnings after moving to host certificates. If you signed host keys but still get warnings, your known_hosts probably still holds the old bare host key, which takes precedence. Remove the stale entry with ssh-keygen -R mylab.local and let the @cert-authority line do its job.
One general principle: change one thing at a time and keep that spare session open. SSH hardening locks you out of your own house far more often than it locks out attackers, and every lock-out I have ever caused came from stacking three changes and reloading before testing any of them.
Is it worth it?
For a single machine you rarely touch, honest answer: keys plus disabled passwords is plenty, and certificates are overkill you will resent maintaining. The break-even arrives with the third or fourth server, or the second household member, or the first time you have to revoke access in a hurry and realise you cannot remember every box a laptop could reach. At that point certificates stop being a party trick and start saving you real time and real risk, and the eight-week expiry quietly does the revocation you keep forgetting to do.
The second factor is a separate decision with a lower threshold. If a single machine holds anything you would hate to lose — a password vault, family photos, the keys to everything else — put a real second factor on it today, and lean towards the hardware kind so the credential never sits on a disk. Once the front door is this solid, the next question is what happens when something still hammers on it, which is where an active defence like CrowdSec or Fail2ban earns its place. Harden the lock, then watch the door.
Related Content
