Caching Is Just Lying on Purpose

Phil Karlton’s line that there are only two hard problems in computer science — cache invalidation and naming things — gets quoted so often it’s become wallpaper, repeated without anyone stopping to explain why invalidation is actually hard. Here’s the honest version: a cache is a copy of an answer you computed earlier, served up as though you just computed it again. That’s a lie, told deliberately, for a good reason — recomputing the real answer is expensive and the old answer is probably still correct. The entire discipline of caching is the discipline of managing exactly how big a lie you’re willing to tell and for how long, and almost every cache bug I’ve ever chased down turned out to be a case where the lie outlived its expiry date without anyone noticing.

That framing sounds glib but it’s genuinely how I debug cache problems now. When a user reports seeing stale data, the question isn’t “is the cache broken” — caches don’t break, they do exactly what you told them to do. The question is “which lie did we tell, when did we tell it, and did we forget to update the user when the truth changed.” Every caching bug traces back to a gap between those three things.

That framing also explains why caching problems resist generic advice. A TTL of thirty seconds is reckless for a page that rarely changes and perfectly safe for a stock ticker nobody expects to be instant; a five-minute TTL is invisible on a blog post and a serious bug on a shopping cart total. There’s no universal “correct” cache duration, because the question was never how long is safe in the abstract — it’s how much disagreement between the cached value and reality your users, or your downstream systems, can tolerate before the lie stops being harmless and starts being a defect.

Why the lie is worth telling

The case for caching is arithmetic. A database query that joins three tables and aggregates a few thousand rows might take 40 milliseconds. The same answer served from an in-memory key-value store takes under a millisecond, because you’ve skipped the query planner, the disk (or page cache) read, the join, and the aggregation, and replaced all of it with a hash-table lookup. Multiply that saving by every request hitting a popular page and the difference between “cached” and “not cached” is the difference between a database that copes with your traffic and one that falls over during a product launch. Caching earns its complexity budget precisely because computation is expensive and memory lookups are cheap, and that gap has only widened as datasets have grown faster than CPU speed per core has.

The cost of the lie is that the cached value and the real value can now disagree, and the whole discipline of cache invalidation is about bounding how long that disagreement is allowed to last and how badly it’s allowed to matter. Get the bound wrong in one direction and you’ve built a cache that never actually saves you anything, because it invalidates so aggressively it might as well not exist. Get it wrong in the other direction and users see data that was true five minutes, five hours, or five days ago, presented with the same confidence as data that’s true right now.

The three ways to bound a lie

Time-to-live is the simplest bound, and it’s honest about what it’s doing: the cached value expires after a fixed duration regardless of whether the underlying data actually changed. SETEX product:123 300 '{"price": 19.99}' tells Redis to serve that price for up to five minutes before forcing a fresh lookup. TTL is a good default because it requires no coordination with whatever wrote the original data — the cache just quietly stops trusting itself after a while, which is a robust failure mode even if the writer never tells it anything changed.

1
2
3
4
5
6
7

# redis-cli session showing a TTL-bound cache entry
127.0.0.1:6379> SETEX product:123 300 '{"price": 19.99}'
OK
127.0.0.1:6379> TTL product:123
(integer) 287
127.0.0.1:6379> GET product:123
"{\"price\": 19.99}"

Write-through invalidation is the second bound, and it’s the honest opposite of TTL: rather than letting the lie expire on a timer, you explicitly delete or update the cached entry the instant the underlying data changes. This is more accurate — there’s no window where the cache is stale — but it requires every code path that writes the real data to remember to also touch the cache, and the moment one write path forgets, you have a cache entry that’s wrong indefinitely, because nothing is going to expire it. I’ve inherited more than one system where a background job updated a database column directly, bypassing the application layer that normally invalidated the cache, and the resulting stale reads took days to diagnose because nobody thought to check whether that particular write path remembered the cache existed.

Event-driven invalidation is the third and most robust, where the cache subscribes to a change stream (database triggers, a message queue, change-data-capture) rather than relying on every writer remembering to invalidate manually. It’s more infrastructure, but it closes the “forgot to invalidate” failure mode structurally rather than by discipline, because the invalidation is driven by the fact of the change itself rather than by someone remembering to call a function.

The stampede is the lie collapsing all at once

The failure mode that catches people who’ve otherwise got invalidation right is the cache stampede: a hot key expires, and the next thousand requests that would have hit the cache all miss simultaneously, all fall through to the database at once, and the database — which the cache existed specifically to protect — gets hit with a thundering herd it was never sized for. This is the mechanism behind more “why did the database fall over at exactly 3pm” incidents than almost anything else, because the timing is deterministic: whatever TTL you set, expiry happens for every client at once if the key was set once and read by everyone.

The standard fix is probabilistic early expiration — recompute the value slightly before it actually expires, with the probability of recomputing increasing as the TTL approaches zero, so the herd gets spread out across many individual refreshes instead of one synchronised stampede. A simpler and more common fix is a lock: the first request to see a miss acquires a short-lived lock, recomputes the value, and every other concurrent request either waits briefly for that value or serves the (technically expired, but still recent) stale value while the recompute is in flight rather than all hitting the database independently. Either approach turns one large, synchronised lie-collapse into many small, staggered ones, which is exactly what protects the origin system the cache was shielding. I’ve seen a genuinely well-designed cache layer take down a database purely because every key in a large batch happened to share the same TTL from a bulk-import job, so thousands of unrelated keys all expired within the same second — the fix there had nothing to do with the caching logic itself and everything to do with adding a small random jitter to the TTL at write time, so expirations spread across a window instead of landing on the same clock tick.

Cache-aside versus read-through, and who’s telling the lie

There’s a design choice underneath all of this that determines who’s responsible for the lie: cache-aside, where your application code checks the cache first, falls through to the database on a miss, and explicitly writes the result back into the cache itself, versus read-through, where the cache sits in front of the database as a proxy and handles the fallback and repopulation transparently. Cache-aside is the pattern most teams reach for first because it needs no extra infrastructure — a Redis client and a few lines of “check, miss, fetch, set” logic — but it means every code path that reads that data has to remember to implement the pattern correctly, and a new engineer adding a new query path can easily forget the cache exists at all, quietly bypassing it and hitting the database on every request without anyone noticing until a load test reveals it.

Read-through pushes that responsibility into the cache layer itself, so the application only ever talks to the cache and never has to remember the fallback logic, at the cost of a piece of infrastructure that needs to understand your data source well enough to repopulate itself. This is roughly the same trade-off that shows up in what eventual consistency is actually promising you — the real question is who’s accountable for bounding the staleness and how visible that boundary is to the rest of the system. A cache-aside pattern scattered across a dozen call sites, each with a slightly different TTL and a slightly different key format, is a much harder system to reason about than a single read-through layer with one invalidation policy, even though the two approaches can serve identical data under identical load in the common case.

Troubleshooting a cache that’s lying badly

The first thing I check when a cache is causing visible problems is whether the TTL actually matches how often the underlying data changes. A five-minute TTL on data that updates every thirty seconds guarantees users see stale prices, stock counts or statuses most of the time they look — the lie is simply too big for the use case, and the fix is either a shorter TTL or a switch to write-through invalidation for that specific key.

The second is checking for cache key collisions from insufficiently specific keys — caching a page fragment keyed only on a product ID when the actual output also depends on the user’s currency or locale will serve the wrong currency to some fraction of users, and it’s a bug that only shows up for a subset of traffic, which makes it maddening to reproduce from a single test account. The fix is almost always to fold every variable the output actually depends on into the cache key itself — product:123:currency:GBP:locale:en-GB rather than product:123 — even though it multiplies the number of distinct entries the cache has to hold and can push memory usage up faster than a flat per-item key would.

The third, and the one I’ve personally lost the most hours to, is a cache that’s silently never being invalidated at all because the invalidation call is wrapped in a try/catch that swallows errors. If your invalidation logic can fail (network blip to the cache server, wrong key format after a schema change) and that failure is caught and logged rather than surfaced, you can run for months with invalidation silently broken while TTL alone masks the problem, until the day someone lowers the TTL for unrelated reasons and the staleness becomes obvious. Log cache invalidation failures loudly, because a cache that’s failing to expire correctly, working purely on TTL, is a much bigger lie than the one you signed up to tell.

Is it worth the complexity

Almost always, for read-heavy paths where the same computation gets requested repeatedly and the underlying data doesn’t need to be instantaneous — product listings, rendered page fragments, search results, aggregated statistics. It’s much harder to justify for data where staleness is genuinely dangerous rather than just slightly wrong: account balances, inventory counts near zero, anything feeding a financial transaction, where the cost of serving a confidently wrong answer outweighs the milliseconds saved. In those cases the honest move is either no cache at all, or a very short TTL paired with explicit invalidation and a monitoring alert on invalidation failures, because the entire value of caching depends on the lie being small enough, and short-lived enough, that nobody gets hurt by believing it. If you want the mechanism at internet scale rather than inside one service, a CDN is the same trick played across the whole planet, and Redis as a cache you actually understand is worth reading before reaching for it as a default, because the discipline is identical regardless of which layer you’re applying it at — bound the lie, and know exactly when it stops being true.