Postgres can model a leaderboard. A table of (user_id, score) with an index. It works. It is also 100× slower than a Redis Sorted Set built for exactly this query.
Both answers are correct. Neither engineer is wrong. They are sitting at different points on the same dial: AT6 — Generality vs Specialisation.
This article dissects that dial. The general-purpose tool handles all cases. The specialised tool handles one case optimally. Choosing between them is a decision most teams make implicitly, then pay for later.
The dial
A general-purpose component handles many use cases. A specialised component handles one use case optimally. Postgres is general-purpose. An in-memory sorted set is specialised.
Using Postgres for a leaderboard works. Using Redis Sorted Sets for the same leaderboard is 100× faster. Same functional answer. Different order of magnitude on the metric that matters.
The examples appear at every layer of a system. SQL database vs a specialised time-series database for metrics storage. HTTP REST API vs gRPC for high-throughput internal service communication. A general task queue vs a specialised workflow engine for complex multi-step processes.
Each pair is the same dial. One side handles the long tail of requirements. The other side is built for one access pattern and dominates on that pattern alone. Neither side is superior in the abstract. The dial is set correctly only against a specific workload.
Pole 1 — Generality
The general-purpose pole assumes you do not yet know what your access patterns are. Postgres accepts arbitrary queries. It joins, filters, aggregates, indexes any column. If 90% of operations are arbitrary queries on relational data, the general-purpose SQL database is correct.
The benefit is optionality. A new product requirement appears. A new report is needed. The database already supports it. No new system to procure, learn, or operate.
The cost is efficiency on any one pattern. A query the database was not designed to accelerate runs at general-purpose speed. That is fine when speed is not the constraint. It stops being fine when a specific query pattern moves onto the critical path.
Pole 2 — Specialisation
The specialised pole assumes you know the access pattern and it dominates. A Redis Sorted Set is built for one operation: maintain a set of members ordered by score, retrieve top-N in log time. Nothing else. That single-purpose design is why it is 100× faster than the general-purpose alternative.
Where this pole wins: SQL database vs specialised time-series database for metrics storage. A time-series workload is append-heavy, range-query-heavy, and predictable. A time-series database is built for exactly that shape. It compresses better, ingests faster, and answers range queries faster than a general SQL store handling the same volume.
The rule for setting the dial toward specialisation: optimise for the common case. If 90% of operations are range queries on time-series data, a specialised time-series database is justified.
The setting rule
"Optimise for the common case."
Two words matter in that sentence: common and justified.
Common means the pattern dominates. If range queries are 90% of the workload, they define the workload. If they are 5%, they do not. The general-purpose tool absorbs 5% inefficiency without notice.
Justified means the performance benefit exceeds the cost of adopting the specialised system. That cost is not zero. Every specialised component is another system to operate, monitor, and hire expertise for. The performance benefit must exceed the operational cost.
Premature specialisation adds operational complexity for a performance gain that may never be needed. That is the failure mode of the specialised pole.
The failure mode — FM3
FM3 — Unbounded Resource Consumption is what you pay when the dial is set incorrectly for the access pattern.
The direction of failure depends on which pole is wrong.
Set toward generality when the pattern demands specialisation: the general tool consumes resources proportional to a query shape it was not built for. Memory grows. CPU grows. Connections grow. The system does not fail cleanly — it degrades under the load a specialised tool would have absorbed easily. A leaderboard served by Postgres at scale is this failure. The query works. It just consumes 100× the resources.
Set toward specialisation when the pattern is not yet stable: the specialised tool cannot answer the next query the product needs. Now you have two systems — the specialised one and the general one — plus the plumbing to keep them in sync. Operational surface expands. Every new access pattern forces a decision about which store owns it.
Both directions of the wrong choice consume resources without bound as scale grows. That is FM3. The signal is the same on either side: resource consumption grows faster than request volume.
Where this appears in a real design
A social feed makes this decision explicitly. A relational database can model a feed — a table of (user_id, post_id, timestamp) rows. It works at small scale.
But fetching the top 50 posts for a user from that table, sorted by timestamp, at 10 million reads per day, is a read pattern that a general-purpose SQL database handles less efficiently than a sorted-set store. Redis ZSET — user_id mapped to a sorted set of post references — exists precisely for this query pattern.
The dial is set toward specialisation. The reason is the common case: top-N ordered reads on a per-user set, at high volume.
The cost is named: Redis is another operational system. It must be monitored, backed up, and tuned separately from the primary database.
That is the shape of every AT6 decision. Name the common case. Name the operational cost. Compare.
The same shape appears in the other examples. A time-series metrics workload sitting inside a general SQL store: the common case is append plus range scan, and the specialised time-series database is justified. A high-throughput internal RPC path sitting on HTTP REST: the common case is short, frequent, machine-to-machine calls, and gRPC is justified. A multi-step process orchestrated inside a general task queue: the common case is stateful workflow with retries and branches, and a specialised workflow engine is justified.
In each case the general tool got the product to a working state. It is not wrong. It is what let the team ship before the access pattern was known. The specialisation decision only becomes correct once the pattern is known and dominant.
The signal that tells you this applies to your system
Your general-purpose database is on the critical path. One query pattern accounts for a disproportionate share of its load. That query pattern is not what the database was optimised for — it is a leaderboard, a top-N, a range scan on time-series data, a graph traversal.
The database still answers the query. It just consumes far more resources than the shape of the data would require if the right tool held it.
When you see this pattern, AT6 is the decision waiting for you. The general tool bought you time. The specialised tool is what the access pattern now demands. FM3 is the cost of waiting too long to move.
The harder question — the one this article does not answer — is when to move. Every specialised system you add is operational debt. Every one you delay is resource debt. The dial has a correct setting at every point in a product's life, and the correct setting changes as the product grows.
The full framework treatment — compression blocks, three-level exercises, and the complete AT/FM mapping — is in the Reference Book, Chapter 8 (Architecture Tradeoffs). Free chapter available at computingseries.com/books/ref.