From Zero to SSH Hero: Securely Hardening a Linux Server in 2025

<p>The first time I stood up a public-facing VPS and left <code>auth.log</code> open in a terminal, the entries started scrolling within four minutes. Not within a day, not within an hour — within four minutes. A fresh box with a public IP gets found that fast, and the moment it is found, a queue of automated bots begins guessing <code>root</code>/<code>123456</code>, <code>admin</code>/<code>admin</code>, <code>ubuntu</code>/<code>password</code> against port 22, around the clock, forever. None of those attempts is sophisticated. They do not need to be. They are betting that out of the thousands of servers they hit, a handful will have left a weak password on. Your job is to make absolutely sure yours is not one of them, and the good news is that the controls that achieve this take about ten minutes and have not fundamentally changed in years.</p> <p>This is the SSH setup I run on every new server before it touches the open internet. It is opinionated, it explains why each step matters, and it includes the bit most guides skip: how to recover when you inevitably lock yourself out.</p> <h2 id="why-bother--the-threat-is-real-and-dumb">Why bother — the threat is real and dumb</h2><div class="ad-unit ad-in-article" aria-label="Advertisement"> <span class="ad-label">Advertisement</span> <ins class="adsbygoogle" style="display:block;text-align:center" data-ad-client="ca-pub-3726833845844946" data-ad-slot="3291553914" data-ad-format="auto" data-full-width-responsive="true"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </div> <p>It helps to understand exactly what you are defending against, because it changes how you prioritise. The overwhelming majority of SSH attacks are not targeted. They are wide, indiscriminate credential-stuffing sweeps run by botnets that have already compromised other machines. They find your port 22, throw a dictionary at it, and move on if nothing sticks. The defence against this is almost insultingly simple: if there is no password to guess, there is nothing for them to win.</p> <p>That single insight — <strong>kill password authentication entirely</strong> — does more than every other hardening step combined. Everything else in this article is defence in depth on top of that one decision. Skip the keys and bolt on fail2ban and a non-standard port, and you have built a fence around an unlocked door. Get the keys right first.</p> <h2 id="step-one-generate-a-proper-key-pair">Step one: generate a proper key pair</h2> <p>Use ed25519, not RSA. Ed25519 keys are short, fast, and based on a curve with no known practical weaknesses; RSA-2048 is ageing and RSA keys are needlessly long. On your <strong>local</strong> machine — never on the server — run:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">ssh-keygen -t ed25519 -C <span class="s2">&#34;me@my-laptop&#34;</span> -f ~/.ssh/id_ed25519 </span></span></code></pre></td></tr></table> </div> </div><p>Give it a passphrase. A passphrase means a stolen private key is not immediately usable, and your SSH agent will cache it so you only type it once per session. Then copy the public half to the server:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected] </span></span></code></pre></td></tr></table> </div> </div><p><code>ssh-copy-id</code> appends your public key to <code>~/.ssh/authorized_keys</code> with the correct permissions, which is exactly where it belongs. Test that key login works <em>before</em> you disable passwords — log out, log back in, and confirm you are not prompted for a password. This ordering matters enormously: prove the new door opens before you brick up the old one.</p> <h2 id="step-two-a-sshd_config-that-holds-up">Step two: a sshd_config that holds up</h2><div class="ad-unit ad-in-article" aria-label="Advertisement"> <span class="ad-label">Advertisement</span> <ins class="adsbygoogle" style="display:block;text-align:center" data-ad-client="ca-pub-3726833845844946" data-ad-slot="3291553914" data-ad-format="auto" data-full-width-responsive="true"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </div> <p>Now harden the daemon. Edit <code>/etc/ssh/sshd_config</code> (or, better, drop a file into <code>/etc/ssh/sshd_config.d/</code> so distro upgrades do not clobber your changes). These are the settings that matter:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span><span class="lnt">2 </span><span class="lnt">3 </span><span class="lnt">4 </span><span class="lnt">5 </span><span class="lnt">6 </span><span class="lnt">7 </span><span class="lnt">8 </span><span class="lnt">9 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl"># /etc/ssh/sshd_config.d/99-hardening.conf </span></span><span class="line"><span class="cl">PasswordAuthentication no </span></span><span class="line"><span class="cl">KbdInteractiveAuthentication no </span></span><span class="line"><span class="cl">PermitRootLogin no </span></span><span class="line"><span class="cl">PubkeyAuthentication yes </span></span><span class="line"><span class="cl">AllowUsers admin deploy </span></span><span class="line"><span class="cl">MaxAuthTries 3 </span></span><span class="line"><span class="cl">LoginGraceTime 20 </span></span><span class="line"><span class="cl">X11Forwarding no </span></span></code></pre></td></tr></table> </div> </div><p>Each line earns its place. <code>PasswordAuthentication no</code> and <code>KbdInteractiveAuthentication no</code> together slam the door on credential stuffing — both are needed, because disabling one and leaving the other is a classic mistake that leaves a password path open. <code>PermitRootLogin no</code> forces attackers (and you) to go through an unprivileged account and <code>sudo</code>, so a single compromised key is not instant root. <code>AllowUsers</code> is an allow-list: only the named accounts may log in at all, which neatly shuts out every service account and stale user on the box. <code>MaxAuthTries 3</code> and a short <code>LoginGraceTime</code> cut off slow drip attacks.</p> <p>Before you restart the daemon, <strong>validate the config</strong>:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo sshd -t </span></span></code></pre></td></tr></table> </div> </div><p>If that prints nothing, the syntax is good. If it complains, fix it now — a typo here can stop <code>sshd</code> from starting and lock you out. Then reload:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">sudo systemctl reload ssh </span></span></code></pre></td></tr></table> </div> </div><p>Use <code>reload</code>, not <code>restart</code>, and keep your existing session open while you test a <em>new</em> connection in a second terminal. If the new connection works, you are safe to close the first. If it does not, you still have the original session to fix things.</p> <h2 id="step-three-should-you-move-off-port-22">Step three: should you move off port 22?</h2> <p>Changing the SSH port from 22 to something else is the most argued-about item on every hardening list. Here is the honest position: it does <strong>not</strong> improve security in any cryptographic sense. A targeted attacker runs a port scan and finds your SSH on port 47213 in seconds. What it <em>does</em> do is eliminate the firehose of low-effort bots that only ever knock on port 22, which means your <code>auth.log</code> becomes readable and fail2ban has far less noise to chew through. That is a real operational benefit, not a security one. Set it if you like a quiet log; do not kid yourself it is a defence.</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Port 2222 </span></span></code></pre></td></tr></table> </div> </div><p>If you do move it and you are behind a firewall, remember to open the new port and, ideally, close the old one.</p> <h2 id="step-four-fail2ban-for-the-persistent-ones">Step four: fail2ban for the persistent ones</h2> <p>Even with keys-only auth, you will still see connection attempts hammering the daemon. <code>fail2ban</code> watches the auth log and temporarily firewalls off any IP that misbehaves. Install it, then create an override under the jail directory so package updates do not overwrite your settings:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span><span class="lnt">2 </span><span class="lnt">3 </span><span class="lnt">4 </span><span class="lnt">5 </span><span class="lnt">6 </span><span class="lnt">7 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl"># /etc/fail2ban/jail.d/sshd.conf </span></span><span class="line"><span class="cl">[sshd] </span></span><span class="line"><span class="cl">enabled = true </span></span><span class="line"><span class="cl">port = 2222 </span></span><span class="line"><span class="cl">maxretry = 3 </span></span><span class="line"><span class="cl">findtime = 10m </span></span><span class="line"><span class="cl">bantime = 1h </span></span></code></pre></td></tr></table> </div> </div><p>Three strikes inside ten minutes earns a one-hour ban. Restart with <code>sudo systemctl restart fail2ban</code> and check it is watching with <code>sudo fail2ban-client status sshd</code>. For a layered approach that also shares threat intelligence across the community, CrowdSec is the modern alternative I dig into in <a href="/story/locking-out-the-bots-fail2ban-and-crowdsec/">locking out the bots with fail2ban and CrowdSec</a>.</p> <h2 id="step-five-hardware-backed-keys-the-strong-move">Step five: hardware-backed keys (the strong move)</h2> <p>If you want to genuinely raise the bar, put the private key somewhere it physically cannot be copied: a FIDO2 security key. Modern OpenSSH supports this natively with the <code>-sk</code> key types:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">ssh-keygen -t ed25519-sk -O resident -f ~/.ssh/id_ed25519_sk </span></span></code></pre></td></tr></table> </div> </div><p>Now logging in requires a physical tap on the hardware token. A stolen laptop is no longer a stolen key, because the secret never left the device and never will. This is the single biggest upgrade after going keys-only, and I have written about living with one full-time in <a href="/story/yubikey-for-everything-ssh-gpg-fido2-and-the-paperweight-drawer/">YubiKey for everything</a>.</p> <h2 id="a-client-config-that-saves-you-from-yourself">A client config that saves you from yourself</h2> <p>Most lockouts and fat-finger errors come from typing connection details by hand every time. Put them in <code>~/.ssh/config</code> on your local machine instead, once, correctly:</p> <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span><span class="lnt">2 </span><span class="lnt">3 </span><span class="lnt">4 </span><span class="lnt">5 </span><span class="lnt">6 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">Host mylab </span></span><span class="line"><span class="cl"> HostName 192.168.1.50 </span></span><span class="line"><span class="cl"> User admin </span></span><span class="line"><span class="cl"> Port 2222 </span></span><span class="line"><span class="cl"> IdentityFile ~/.ssh/id_ed25519_sk </span></span><span class="line"><span class="cl"> IdentitiesOnly yes </span></span></code></pre></td></tr></table> </div> </div><p>Now <code>ssh mylab</code> carries the right user, port, and key automatically — no chance of accidentally offering the wrong key and burning a <code>MaxAuthTries</code> attempt. The <code>IdentitiesOnly yes</code> line is quietly important: without it, your SSH agent throws every key it holds at the server in turn, which on a server with a low <code>MaxAuthTries</code> can get you rejected before it even reaches the correct one. Pinning the identity makes every login deterministic.</p> <p>Resist the urge to enable agent forwarding (<code>ForwardAgent yes</code>) as a default. It is convenient for hopping between hosts, but a compromised intermediate server can use your forwarded agent to impersonate you onward. Turn it on per-command when you genuinely need it, never globally.</p> <h2 id="troubleshooting-what-goes-wrong-and-how-to-recover">Troubleshooting: what goes wrong, and how to recover</h2> <p>This is the section every other guide leaves out, and it is the one you will actually need.</p> <p><strong>You reloaded sshd and now you are locked out.</strong> This is why you keep the original session open. If you still have it, revert your change, run <code>sudo sshd -t</code>, and reload again. If you closed everything and cannot get in, you need out-of-band access: most VPS providers offer a web console or &ldquo;rescue mode&rdquo; that gets you a root shell without SSH. Boot into it, fix <code>sshd_config</code>, restart, done. The lesson learned the hard way is to always know your provider&rsquo;s console exists <em>before</em> you need it.</p> <p><strong>&ldquo;Permission denied (publickey)&rdquo; even though the key is right.</strong> Nine times in ten this is file permissions. <code>authorized_keys</code> must be <code>600</code>, <code>~/.ssh</code> must be <code>700</code>, and the home directory must not be group-writable, or <code>sshd</code> silently refuses the key on security grounds. Run <code>chmod 700 ~/.ssh &amp;&amp; chmod 600 ~/.ssh/authorized_keys</code>. Add <code>-v</code> to your ssh command to see exactly which key it offered and why the server rejected it.</p> <p><strong>fail2ban banned your own IP.</strong> It happens — you fat-finger a passphrase a few times and your home address gets jailed. Unban yourself from the console with <code>sudo fail2ban-client set sshd unbanip &lt;your-ip&gt;</code>, and consider adding your static management IP to the <code>ignoreip</code> directive so you cannot lock yourself out this way again.</p> <p><strong>sshd will not start after editing.</strong> Almost always a syntax error you would have caught with <code>sshd -t</code>. Read the exact line number in <code>journalctl -u ssh</code>, fix it, validate, retry. Never restart <code>sshd</code> on a remote box without validating first.</p> <h2 id="keep-it-that-way">Keep it that way</h2> <p>Hardening is not a one-off. Turn on unattended security upgrades so OpenSSH itself stays patched, and periodically audit <code>~/.ssh/authorized_keys</code> to remove keys for people and machines that no longer need access. Forward your auth logs somewhere you will actually look at them; failed-login spikes and any successful root attempt are worth an alert. If you want to go further, feeding these logs into a central pipeline is exactly the use case I cover in <a href="/story/from-logs-to-insights-building-a-real-time-siem-pipeline-with-open-source-tools/">building a real-time SIEM pipeline</a>.</p> <h2 id="is-it-worth-it-who-is-this-for">Is it worth it? Who is this for</h2> <p>If your server has a public IP, this is not optional — it is the baseline, and the whole sequence takes about ten minutes the first time and two minutes thereafter once it is muscle memory. For a purely internal box on a trusted home LAN that is never exposed, you can relax the port-changing theatre, but I would still go keys-only and disable root login even there, because lateral movement is a thing and &ldquo;internal&rdquo; networks get breached.</p> <p>The person who does <em>not</em> need all of this is someone running a throwaway lab VM behind a firewall with no inbound access at all. For everyone whose box answers on port 22 from the open internet, the equation is simple: ten minutes of setup against the certainty — not the risk, the certainty — of being probed within minutes of going live. Do the keys, validate the config, keep a console handy, and you will never be the unlocked door the bots are hoping for.</p>