What a TLS Handshake Is Actually Saying
Every padlock icon hides a negotiation, and the negotiation is the interesting part
Contents
Every time a padlock icon appears in a browser bar, a small, fast negotiation just finished behind the scenes, and I only really understood what that negotiation contained after setting up my own reverse proxy with a self-signed certificate and watching it fail in ways the browser refused to explain in plain language. “Your connection is not private” turns out to be the browser’s summary of a genuinely specific handshake step failing, and once you can read openssl s_client output instead of trusting a vague error page, most TLS problems stop being mysterious and start being one of about four specific things going wrong in a well-understood sequence.
The Problem the Handshake Actually Solves
Before any encrypted data can flow, two computers that have never spoken before need to agree on a shared secret key, over a network that anyone might be listening to, without ever transmitting that key in a form an eavesdropper could steal. This sounds like it shouldn’t be possible — how do you agree on a secret in front of an audience — and the mathematical trick that makes it possible, key exchange algorithms like Diffie-Hellman, is genuinely one of the more elegant ideas in applied cryptography: both sides can combine their own private value with a public value shared over the open connection and independently arrive at the same shared secret, while anyone watching the exchange only sees the public values and can’t feasibly derive the secret from them.
That’s only half the problem, though. Agreeing on a secret with somebody doesn’t help if that somebody is actually an attacker impersonating your bank rather than your bank itself, so the handshake also has to solve identity: proving the server you’ve connected to really is the one whose name is in the address bar. This is where certificates and certificate authorities come in — a chain of digital signatures anchored in a small set of root authorities your operating system or browser already trusts, so the server can prove “a trusted authority vouches that I am who I claim to be” without you having ever met that authority yourself.
The Handshake, Step by Step
Modern TLS 1.3 has trimmed this down from earlier versions, but the sequence still solves both problems in order. The client opens with a ClientHello, listing the TLS versions and cipher suites it supports and including its half of the key exchange material already, an optimisation TLS 1.3 added specifically to shave a round trip off earlier versions. The server responds with a ServerHello containing its own half of the key exchange, immediately followed by its certificate and a signature proving it holds the private key matching that certificate — this is the identity proof, and it’s checked against the chain of trust up to a root certificate authority baked into the client’s trust store.
| |
That “Verify return code: 0 (ok)” line is the single most useful piece of diagnostic output TLS gives you — it means the certificate chain validated correctly, back to a root the client already trusts. If it says anything else, the identity half of the handshake failed, and no amount of the encryption itself being technically fine will make a browser proceed, because a perfectly encrypted connection to an unverified impersonator is worse than no connection at all. Once both sides have exchanged key material and the certificate checks out, both independently derive the same symmetric session key from the exchanged values — this is the part Diffie-Hellman actually makes possible — and everything after this point uses fast symmetric encryption (AES, ChaCha20) rather than the slower asymmetric maths that only gets used during the handshake itself.
Why Two Kinds of Cryptography, Not One
This split — slow asymmetric cryptography for the handshake, fast symmetric cryptography for the actual data — exists because the two jobs have genuinely different requirements. Asymmetric key exchange lets two strangers agree on a secret without a shared prior secret, which is exactly the property you need at connection start, but it’s computationally expensive relative to how much data you’d want to encrypt with it directly. Symmetric encryption is fast enough to encrypt gigabytes per second on ordinary hardware but requires both sides to already share the same key, which is exactly what they didn’t have before the handshake. Using asymmetric crypto only to bootstrap a symmetric key, then switching to symmetric encryption for the actual session, gets the security property of the first and the speed of the second, and essentially every modern secure protocol — not just TLS — follows this same two-phase pattern for the same reason.
Why Certificates Expire and Why That’s Deliberate
A certificate’s expiry date isn’t bureaucratic friction; it bounds how long a compromised or mis-issued certificate can be abused before it stops being accepted anywhere, and it forces the ecosystem to periodically re-verify that whoever controls a domain still controls it, rather than trusting a proof of ownership from years ago indefinitely. Automated issuance through Let’s Encrypt made short-lived certificates — 90 days, sometimes far shorter with newer short-lived certificate profiles — practical at scale precisely because renewal became a background cron job instead of a manual, once-a-year chore nobody remembered to do until it broke in front of customers. This is also why letting a renewal job silently fail is one of the more embarrassing homelab outages to have: everything was fine for 89 days, and then it very much wasn’t, at 3am, with a browser warning that offers no explanation beyond “not private.”
Why the Second Connection Feels Instant
Anyone who’s compared the first load of a site to a reload a moment later has noticed the second one feels faster, and session resumption is a real, deliberate part of why. Running the full handshake — asymmetric key exchange, certificate verification, signature checks — on every single connection would be wasteful when the same two parties just did all of that work seconds ago, so TLS 1.3 lets a server issue a session ticket after the first handshake completes, an encrypted blob the client can present on a subsequent connection to skip straight to a resumed session using previously negotiated key material, verified cryptographically without redoing the certificate chain walk from scratch.
| |
That Reused line is the tell that resumption worked — the handshake still happens, but it’s a lighter negotiation confirming the previously established session rather than a full identity-and-key-exchange from zero. TLS 1.3 goes further with 0-RTT (zero round-trip time) resumption, letting a returning client send encrypted application data in the very first packet of a resumed connection, before the server has even replied — a genuine latency win, with a genuine trade-off attached: that first 0-RTT request can potentially be replayed by an attacker who captured it, since there’s no fresh round trip to guarantee freshness the way a full handshake does, which is why 0-RTT is generally restricted to idempotent requests (a GET, not a POST that charges a card) by anything implementing it carefully.
Certificate Pinning and Why It Fell Out of Favour
For a while, mobile apps and some high-security services used certificate pinning — hard-coding the expected certificate or public key inside the client, rather than trusting whatever the operating system’s root store says, so that even a compromised certificate authority couldn’t successfully issue a fraudulent certificate an attacker could use against that specific app. This closes a real gap: the trust model described earlier assumes every root certificate authority in your OS trust store is trustworthy, and history has shown that assumption fails occasionally, when a CA mis-issues a certificate or gets compromised outright.
Pinning fell out of favour for a mundane but serious operational reason: it makes routine certificate rotation dangerous. Renew a certificate on a normal schedule, forget to ship an updated pin in the app first, and every copy of that app in the field loses connectivity simultaneously, with no way to push a fix through the very channel that just broke. Several high-profile outages traced back to exactly this failure mode, and the industry response has mostly been to rely on shorter certificate lifetimes, certificate transparency logs (public, auditable records of every certificate issued for a domain, so mis-issuance is at least detectable quickly), and stricter CA oversight rather than pinning at the client. It’s a reasonable illustration of a security control that solves a real problem while introducing an operational failure mode severe enough that most teams eventually decided the trade wasn’t worth it for anything short of the highest-value targets.
Troubleshooting: Reading TLS Failures Correctly
When a browser says a connection isn’t private, openssl s_client -connect host:443 -servername host is the first diagnostic step, because it shows you the actual certificate chain and the actual verify return code rather than the browser’s summarised, deliberately vague warning. A verify code other than 0 almost always means one of three things: the certificate expired (check notAfter in the output), the chain is incomplete because an intermediate certificate wasn’t served alongside the leaf certificate, or the hostname you connected to doesn’t match any name on the certificate’s subject or SAN list. Self-signed certificates fail for a different, simpler reason — there’s no chain to a trusted root at all, which is expected and correct behaviour for testing, not a bug, and the fix for internal-only services is to run your own small certificate authority and trust its root deliberately, not to keep clicking through browser warnings in production. If the handshake completes but the connection then hangs or resets, the problem has usually moved past TLS entirely into the application layer behind it — TLS did its job negotiating the encrypted tunnel, and whatever’s misbehaving now is on the other side of that tunnel.
Mutual TLS: When the Server Checks Your Identity Too
Everything above describes one-way verification: the client checks the server’s identity, but the server just accepts whichever client connects. Mutual TLS flips part of that around, requiring the client to present its own certificate during the handshake so the server can verify the client’s identity cryptographically as well, before any application-level authentication (a password, a token) even happens. This is increasingly common between internal services in a homelab or small production setup specifically because it replaces a shared password or API key — something that can be copied, leaked in a log, or reused across services — with a certificate tied to a specific issued identity that can be revoked individually without rotating a secret every other service also depends on. Setting it up yourself is a genuinely approachable weekend project once the one-way handshake above makes sense, and running your own small certificate authority is the natural starting point rather than trying to bootstrap mutual TLS against a public CA that was never designed to issue client certificates for internal use.
Is It Worth Understanding This?
Yes, particularly if you run anything that terminates TLS yourself — a reverse proxy, an internal service, a homelab dashboard — because the difference between “I clicked through the warning” and “I understand exactly which of the three verify failures I’m looking at” is the difference between a five-minute fix and re-clicking through the same warning for months. The maths underneath is genuinely elegant, but you don’t need the maths to get the practical benefit; you need the shape of the negotiation and which log line tells you which of the two problems — identity or encryption — actually failed. For a related piece on why “maths and a trust fall” gets you the rest of the way to https as a whole, see https is just maths and a trust fall, and for what happens once the encrypted tunnel is up but the promise inside it still has limits, how end-to-end encryption ends is the natural next read.
Related Content
