The Computing Series

In Practice: Reading a Cache Through Four Lenses

A distributed cache sits between the application tier and the database.

Flow (MM4): Read requests flow to the cache first. Cache hits return in microseconds. Cache misses add a full database round-trip. The cache’s hit rate determines the effective latency distribution of all read requests.

State (MM5): The cache holds state. That state can become stale when the database is updated. The TTL is the policy for how long stale state is tolerated.

Feedback (MM7): When the cache is healthy, read load on the database is low. When the cache is overloaded or fails, read load on the database spikes. The database degrades. The cache misses increase further. The feedback loop is positive — amplifying the problem.

Redundancy (MM9): A single-node cache is a SPOF for read performance. A clustered cache is redundant — but now requires consistent hashing (T1) to route requests to the right node, and introduces the question of what happens when a node fails and its keys must be served cold.

Four lenses. Four different problems. All present in the same component. This is why fluency with the mental models matters — any one of them alone misses something.

Read in the book →