Redis as a Cache You Actually Understand

Half the self-hosted apps you deploy will ask for a Redis instance in their docker-compose.yml, and most people give it to them the way you leave a light on in a room you never enter: it is there, it seems to help, and you have no idea what it is doing. Then one day the app starts serving stale data, or Redis eats all your memory and gets killed, and you discover that the thing you have been running for a year is a database you never learned to read. This is a shame, because Redis is one of the most comprehensible pieces of infrastructure in the whole self-hosting toolbox once you spend an afternoon with it, and understanding it turns caching from a black box into a tool you can reason about.

The goal here is a working mental model of Redis as a cache — what it stores, how items leave it, and which handful of settings decide whether it makes your homelab faster or quietly starts lying to it. I will keep the focus on caching specifically, which is what most people actually use it for, and flag the moment where “cache” quietly becomes “database” and the rules change under you.

Why a cache exists at all

Before any Redis config, the question worth answering is what a cache is for, because a cache added without a reason is pure liability — another process to run, another source of stale data, another thing to debug. A cache earns its place by making a repeated expensive operation cheap. Your app renders a dashboard that runs six database queries and takes 300 milliseconds; those queries return the same answer for the next five minutes because the underlying data barely changes; so you compute the answer once, stash it somewhere fast, and serve the next hundred requests from the stash in under a millisecond each. The database does one-hundredth of the work and the user waits one-three-hundredth of the time.

Redis is good at the “somewhere fast” part because it keeps everything in RAM and reaches for it through simple key lookups, so a read is a memory access and a hash, measured in microseconds. That speed is also the constraint that shapes everything else about running it: RAM is finite and expensive, so a cache is fundamentally an exercise in deciding what to keep and what to throw away. Almost every Redis-as-cache problem you will ever hit is really a disagreement between you and Redis about what should have been thrown away and when.

What Redis actually is

Strip away the buzzwords and Redis is an in-memory key-value store where the values can be richer than plain strings — hashes, lists, sets, sorted sets, counters. For caching you will mostly use plain string values holding a serialised blob (JSON, a rendered fragment, a query result), keyed by something you can reconstruct, like user:42:dashboard. Two properties matter more than any feature list.

First, Redis is effectively single-threaded for command execution. One command runs to completion before the next begins, which is why a single slow command — a KEYS * across a million keys, a giant sorted-set operation — stalls everything, and why you will hear people warn you off certain commands in production. It also means Redis has no locking complexity to reason about; operations are atomic because they simply cannot overlap.

Second, every key can carry a time-to-live, an expiry after which Redis deletes it automatically. TTLs are the beating heart of caching. They are how you say “this answer is good for five minutes, then forget it and let the app recompute”, and getting them right is most of the skill.

The pattern: cache-aside

There are several caching strategies, and the one you almost certainly want, the one nearly every app implements, is cache-aside (also called lazy loading). The application owns the logic here, and it goes like this: on a read, ask Redis first; if the value is there (a cache hit) use it; if it is not (a cache miss) fetch from the real database, store the result in Redis with a TTL, and return it. In pseudocode:

1
2
3
4
5
6
7
8
9

def get_dashboard(user_id):
    key = f"user:{user_id}:dashboard"
    cached = redis.get(key)
    if cached is not None:
        return json.loads(cached)          # hit: microseconds

    data = db.query_expensive_dashboard(user_id)   # miss: the slow path
    redis.set(key, json.dumps(data), ex=300)        # cache for 300 seconds
    return data

The ex=300 is the whole game. It means each cached dashboard lives at most five minutes, so the worst-case staleness is five minutes and the database gets hit at most once every five minutes per user regardless of how many times they refresh. Shorten the TTL for data that must be fresh; lengthen it for data that rarely changes. This single number is the dial that trades staleness against database load, and tuning it per cache key is the everyday work of running a cache well.

Cache-aside has one famous failure worth naming now: the cache stampede (or thundering herd). When a popular key expires, every request that arrives in the gap before it is repopulated misses simultaneously, and they all hammer the database at once to recompute the same value. For a homelab this is rarely fatal, but if you see periodic database spikes lining up with TTL boundaries, that is the cause, and the fix is to add a little randomness to your TTLs so keys do not all expire on the same tick.

The setting everyone forgets: maxmemory

Here is the configuration mistake I see most often, and it is the one that gets Redis killed. By default, some Redis builds set no memory limit and no eviction policy, so the cache grows without bound until the host runs out of RAM and the kernel’s OOM killer shoots the process. A cache should never do this. A cache is supposed to have a fixed budget and throw away its least useful entries when full. You have to tell Redis that, in redis.conf:

1
2
3

# redis.conf — give Redis a fixed memory budget so it behaves like a cache.
maxmemory 512mb
maxmemory-policy allkeys-lru

maxmemory caps the RAM Redis will use. maxmemory-policy decides what happens when that cap is reached, and the choice matters:

• allkeys-lru evicts the least-recently-used key across the whole dataset. This is the right default for a pure cache — Redis approximates “throw away what nobody has touched lately”, which is exactly what you want.
• allkeys-lfu evicts the least-frequently-used key, which handles the case where a key is touched once in a burst and then never again better than LRU does.
• volatile-lru only evicts keys that have a TTL set, leaving keys without one untouched — useful when you mix cache entries and data you must not lose in the same instance.
• noeviction refuses new writes when full and returns errors. This is the default in many setups and it is the trap: an unbounded, noeviction Redis is a time bomb that either fills the disk (with persistence on) or the RAM.

Set maxmemory to something the host can comfortably spare and pick allkeys-lru for a cache, and Redis stops being a memory risk and starts being a well-behaved fixed-size cache that quietly drops cold data to make room for hot data.

Cache or database? Persistence changes the rules

Redis can write its data to disk, which is where the “cache versus database” line blurs and people get confused about what guarantees they have. There are two mechanisms. RDB takes point-in-time snapshots of the whole dataset every so often — compact, fast to load, but you lose everything since the last snapshot on a crash. AOF (append-only file) logs every write, so it can rebuild the exact state, at the cost of a larger file and slightly slower writes.

For a pure cache, the correct amount of persistence is often none. If Redis restarts and comes back empty, cache-aside simply repopulates it from the database on the next requests; a cold cache just means a slower few seconds while it refills, and nothing is lost. Turning persistence off makes Redis faster and removes a whole class of disk-related problems. The instant your app stores something in Redis that exists nowhere else — session data, a job queue, a rate-limit counter you cannot regenerate — it has stopped being a cache and become your database, and now you need persistence, and you need to back it up like any other database. Know which one you are running. The failure I have watched people walk into is treating a Redis full of irreplaceable session data as a disposable cache, then wondering why a restart logged everyone out or dropped a queue of pending jobs.

Where a cache does and does not belong

A cache is the correct answer when the same expensive read repeats far more often than the underlying data changes. It is the wrong answer, or at least a premature one, when your database is slow for reasons a cache only papers over. If a query is slow because Postgres is running on defaults that ignore your hardware, the honest fix is tuning Postgres so it uses the machine you gave it, and a missing index will beat any cache you bolt on top. Reach for Redis once the database is doing its job well and you still have genuinely repeated expensive work to eliminate.

Caching also sits alongside the other techniques for taking load off a database rather than replacing them. When the pressure is too many short-lived connections rather than too many repeated reads, the tool is connection pooling with something like PgBouncer, and it is worth knowing which problem you actually have before adding either. A cache fixes repeated reads; a pooler fixes repeated connections; they solve different bottlenecks and a homelab rarely needs both at once.

Troubleshooting

Redis keeps getting OOM-killed. No maxmemory set, or maxmemory-policy noeviction. Set a limit and an LRU policy as above. Confirm with redis-cli config get maxmemory and watch redis-cli info memory — the used_memory_human figure should stabilise near your cap rather than climbing past it.

The app serves stale data. Your TTL is too long for how often that data changes, or the app updates the database and forgets to invalidate the cached copy. For write-through freshness, delete the relevant key when the underlying row changes: redis.delete(key) right after the database write, so the next read misses and repopulates.

A low cache hit rate. Check keyspace_hits and keyspace_misses in redis-cli info stats. A hit rate below, say, 80% means your TTLs expire faster than the data is reused, your keys are too granular to be reused, or memory pressure is evicting entries before they are read again. Raise TTLs, coarsen keys, or raise maxmemory.

Everything periodically freezes for a moment. Something is running an O(n) command against a large keyspace — KEYS * is the usual culprit, and it blocks the single command thread. Replace it with SCAN, which iterates in small batches without stalling other commands.

A restart lost data you needed. You were treating a persistence-less cache as durable storage. Decide honestly whether that data is a cache (fine to lose, no persistence needed) or a database (enable AOF and back it up).

Is it worth it?

For a genuine repeated-expensive-read problem, Redis as a cache is one of the highest-leverage things you can add to a homelab: a few lines of cache-aside logic, a sensible maxmemory with allkeys-lru, a well-chosen TTL, and an app that was sluggish becomes instant while the database relaxes. The whole thing is comprehensible in an afternoon, and once you hold the model — cache-aside for the pattern, TTL for freshness, maxmemory plus eviction for the budget, persistence only when it has secretly become a database — Redis stops being the mystery service in the compose file and becomes a tool you can actually reason about.

The failure mode to avoid is adding it out of cargo-cult habit. If nothing in your setup does repeated expensive reads, a cache adds staleness bugs and an extra daemon in exchange for nothing. Add Redis when you can name the expensive operation it will eliminate, size it, give it a TTL you have thought about, and decide up front whether it is allowed to forget everything on a restart. Answer those questions and it will serve you well for years without ever asking for your attention again.