In 2007, Amazon published the Dynamo paper. The company had built a shopping cart service and kept running into the same problem: a relational database, no matter how well tuned, eventually became the bottleneck. Worse, the database’s strong consistency guarantee meant that a network partition — a routine event at Amazon’s scale — could take the entire cart service down. The paper described a different trade: give up strong consistency, accept eventual consistency, and in return get a system that stays available even when nodes fail and network partitions occur. Dynamo became the blueprint for a generation of distributed key-value stores.
The distributed key-value store is the foundational read/write primitive of large-scale systems. Understanding how it works — and what it costs — is prerequisite knowledge for every other system in this book.