How Uber Serves 150 Million Reads Per Second From Cache

Sign up for a free trial of Attio, the agentic CRM.

Ask Attio to build a daily workflow that surfaces the deals that need your attention today, like anything with a stage change, a recent reply, or a new signal in the last 24 hours.

Review your pipeline in Claude, synced live from Attio via MCP.

That's it.

Try Attio Now

Slow billing compounds fast. The Tabs Billing Lag Calculator shows SaaS finance teams exactly what it's costing them — in two minutes.

Calculate your billing lag

150 million reads per second. That's what Uber's caching layer handles during peak hours. And it does it with a 99.9% cache hit rate while keeping data consistent across the board. Most companies struggle to keep a single Redis cluster healthy. Uber built a system that makes their database almost invisible.

Every time you open the Uber app, dozens of backend services fire read requests. Rider location. Driver availability. Surge pricing. Menu items. Order history. Payment methods. All of it sits in Docstore, Uber's distributed database built on top of MySQL.

At Uber's scale, even a well tuned MySQL cluster starts sweating. The storage engine nodes that serve reads are expensive to run and slow to scale horizontally. Adding more replicas means more replication lag, more operational burden, and more chances for things to go wrong.

So instead of scaling the database, Uber put a cache in front of it. But not a separate cache you bolt on from the outside. They embedded the cache directly inside their database's query engine layer. That's the key insight that makes CacheFront different from what you'd build with a standalone Redis cluster.

Most caching architectures look like this: your service talks to a cache, and if the cache misses, your service talks to the database. Two separate systems. Two separate failure modes. Two separate teams worrying about consistency.

Uber did something smarter. CacheFront lives inside the query engine layer of Docstore. The query engine is the stateless component that handles query planning, routing, sharding, and request validation. Since every read and write already flows through this layer, it's the natural place to intercept and cache.

This design gives CacheFront a superpower: it can see both reads AND writes. When a write completes, the query engine already knows which rows changed. It doesn't need to wait for some external system to detect the change. It can invalidate the cache immediately, right there in the same process.

Compare that to a typical setup where your application writes to the database, and then maybe, eventually, a CDC pipeline notices the change and invalidates the cache. That gap between "data changed" and "cache updated" is where stale reads live. Uber's design shrinks that gap to nearly zero.

Here's what happens when a service reads from Docstore with CacheFront enabled.

The query engine receives the read request and checks Redis first. If all requested rows exist in the cache (a cache hit), they're streamed back immediately. The database never even knows a request happened. At 99.9% hit rates, this is what happens for the vast majority of traffic.

If some or all rows are missing from cache (a cache miss), the query engine fetches them from the storage engine, writes them to Redis asynchronously, and streams the combined result back to the caller.

One thing that's easy to miss: CacheFront doesn't just cache "found" rows. It also caches negative results. If a row doesn't exist, that "not found" result gets cached too. Without negative caching, a service repeatedly querying for a nonexistent row would hammer the database every single time. At Uber's scale, that would be catastrophic.

Cache invalidation is famously one of the two hard problems in computer science (the other being naming things). Uber doesn't solve it with one mechanism. They use three, running simultaneously, each covering the gaps of the others.

Layer 1: Direct Invalidation. When a write transaction completes, the storage engine returns the affected row keys and the commit timestamp. A callback in the query engine immediately writes invalidation markers to Redis for those keys. This is the fastest path. The cache is updated before the write response reaches the calling service.

Layer 2: Flux CDC Pipeline. Uber's change data capture system (Flux) tails the MySQL binlog and publishes change events. A consumer picks up these events and invalidates the corresponding cache entries in Redis. This runs asynchronously with sub second delays. It acts as a safety net: if the direct invalidation callback fails (maybe Redis was briefly unreachable), Flux catches the change a moment later.

Layer 3: TTL Expiration. Every cached entry has a time to live. Originally set at 5 minutes, Uber later extended this to 24 hours after proving the first two layers were reliable. TTL is the fallback of last resort. If both direct invalidation and Flux somehow miss a change, the stale entry will expire on its own.

The original CacheFront (which served 40 million reads per second) had a nasty consistency bug. Here's the scenario that broke it.

A service writes a row. The old value sits in cache. The direct invalidation fires and clears the cache. Good so far. But now another read request arrives before Flux processes the change. This read hits a cache miss, goes to a MySQL follower replica, and that replica hasn't received the write yet because of replication lag. So the read pulls the OLD value and caches it. Now you've got stale data sitting in cache, and it won't get fixed until TTL expiration.

I've seen this exact pattern at two other companies. Replication lag plus cache refill is a consistency land mine that almost every team discovers the hard way.

Uber fixed it with two changes to the storage engine.

Soft deletes. Instead of actually removing rows on DELETE, the storage engine sets a tombstone flag. The row stays in the table, marked as deleted. This means every change to a row, including deletions, produces a new version that can be tracked and compared. Without soft deletes, you can't tell the difference between "this row was deleted" and "this row never existed."

Monotonic timestamps. Every transaction gets a strictly monotonic timestamp at microsecond precision. When CacheFront writes a value to Redis, it includes this timestamp. When Flux tries to update the same key later, a Lua script in Redis compares timestamps. If the Flux update carries an older timestamp, Redis rejects it. No more race conditions between the direct path and the CDC path.

How do you know your cache is actually consistent? You measure it.

Uber built Cache Inspector, a monitoring system that mirrors the Flux pipeline with a one minute delay. It compares cached values against the binlog to detect mismatches. It exports staleness metrics, mismatch rates, and histograms.

When they ran Cache Inspector on production traffic, the results showed negligible stale entries. That's what gave them the confidence to extend TTLs from 5 minutes to 24 hours. Longer TTLs mean higher hit rates. Higher hit rates mean fewer database reads. Fewer database reads mean lower costs.

The numbers are staggering. One large Uber customer serving 6 million reads per second needed only 3,000 Redis cores versus roughly 60,000 storage engine cores without caching. That's a 95% cost reduction.

You don't need Uber's scale to use these ideas. Most of these patterns work at any size.

The integrated cache pattern. If you control both your API layer and your data layer, putting cache logic in the API layer (not in a separate service) gives you visibility into both reads and writes. You don't need a separate CDC pipeline to detect changes. Netflix uses a similar approach with their EVCache system, embedding cache logic close to the data path.

Multi layer invalidation. Don't rely on a single invalidation mechanism. At minimum, use direct invalidation on writes plus TTL as a safety net. If you have CDC (like Debezium for PostgreSQL), add that as a second async path. Three layers is better than one.

Negative caching. Cache your "not found" results. At Google, they found that up to 30% of some query workloads were for nonexistent keys. Caching those results cut database load dramatically.

Monotonic versioning. If you have race conditions between two systems updating the same cache key, add a version number or timestamp. Use a Redis Lua script to do compare and set atomically. This is the same pattern DynamoDB uses for optimistic locking.

Follow me on Youtube · LinkedIn · X · Instagram to stay updated.

See you in the next one!
Scortier, Signing Off!