The pattern index
Thirty-one problems, roughly twenty ideas. Once you can name the pattern, recognise where it applies, and state its cost, most designs become composition rather than invention.
Each entry gives you: what it is, where it appears on this site, and the sentence to say.
Distribution and placement
1. Consistent hashing with virtual nodes
Map nodes and keys onto a ring; a key belongs to the first node clockwise. Adding a node moves 1/N keys instead of all of them. Virtual nodes (128–256 per host) even out the arc lengths.
Appears in: distributed cache · web crawler · Google Docs · ChatGPT routing
"Modulo hashing turns a capacity addition into an outage — every key remaps at once. Consistent hashing with 256 vnodes per host bounds it to 1/N and keeps load within a few percent of uniform."
Cost to name: it distributes keys, not load. A single hot key still lands on one node.
2. Partition by the entity that needs serialising
Choose the partition key so that everything requiring a total order lands in one place. Then you need no distributed locks at all.
Appears in: Tinder pair keys · web crawler domains · metrics by series hash · Ticketmaster hash tags
"I make the partition key order-independent over both user IDs, so A→B and B→A land on the same shard and the storage engine serialises them for me — the race disappears rather than being defended against."
3. Actor per entity
One single-threaded mailbox owns one entity's state. Ordering, invariants, and tricky business logic become trivially correct because nothing interleaves.
Appears in: online auction · Google Docs · online chess · Facebook Live comments hub
"The mailbox is the lock. I get linearizability per entity with no database contention, and the tricky rules become a pure function over in-memory state that I can unit-test exhaustively."
Cost to name: the actor is a single point of throughput and failure for that entity; you need rehydration from a log and a fencing token.
Caching
4. Tiered cache with admission control
L1 in-process (W-TinyLFU) → L2 distributed → origin, with an admission policy so a scan cannot evict the hot set.
Appears in: distributed cache · Bitly · Facebook news feed · full treatment in LRU vs W-TinyLFU
"Plain LRU isn't scan-resistant. W-TinyLFU keeps a Count-Min Sketch of recent frequency and refuses to admit a candidate less frequent than the victim it would evict."
5. The near-cache fixes hot keys; sharding does not
A single key hotter than one machine cannot be fixed by adding nodes. An in-process cache with a 1-second TTL turns 2M requests/sec into one per app instance.
Appears in: distributed cache · Facebook news feed · Instagram counters
"Consistent hashing distributes keys, not load. For a viral key the only real fix is caching it in the app process, or replicating it across N salted keys."
6. Stampede prevention: singleflight, early expiry, stale-while-revalidate
One hot key expiring must not produce 50,000 simultaneous origin queries.
Appears in: distributed cache · Bitly · Yelp
"Singleflight first — it's ten lines and coalesces concurrent misses. Then probabilistic early expiry so one reader refreshes before the cliff, and serve-stale-while-revalidate so nobody ever waits."
7. Cache the inputs when you cannot cache the answer
When results are per-viewer (privacy, personalisation), cache every shared component instead: posting lists, hydration, features.
Appears in: Facebook post search · Yelp · Facebook news feed
"Web search caches answers; social search can't, because the answer depends on who's asking. So I cache every input to the answer and leave only a bitmap intersection per request."
Read/write split
8. Hybrid fan-out
Push to precomputed feeds for the many; pull and merge at read time for the few with enormous audiences.
Appears in: Instagram · Facebook news feed · Dropbox notifications
"400M followers at 100k writes/sec is 66 minutes for one post. Above ~25k followers I switch to pull-and-merge, and I only push to followers active in the last 30 days, which removes 70% of the writes for free."
9. Separate the read tier from the write tier entirely
Reads outnumber writes by orders of magnitude and must never contend with them.
Appears in: Ticketmaster seat map · online chess spectators · Google Docs viewers · online auction CQRS
"500k people watching the seat map must not touch the booking core. Availability becomes a 6 KB bitmap on the CDN plus WebSocket deltas — two seconds stale, and I'd say so in the UI."
10. Fan-out tree
One hub cannot write a million sockets. Hub → regional relays → leaf connections, encoding each frame once per relay.
Appears in: Facebook Live comments · online auction · Robinhood market data · online chess
"Coalesce at the source first — nobody can read 50 updates a second — then distribute through a tree so the hub sends ten messages and the leaves do the fan-out."
Correctness
11. Idempotency keys
Deterministic key, claimed by a unique constraint, with the response stored for replay. See the full treatment in idempotency.
Appears in: payments · job scheduler · notifications · Robinhood · Ticketmaster
"Exactly-once delivery is impossible, so I use at-least-once delivery plus an idempotent consumer, which gives an exactly-once effect. The key is deterministic and the database's unique constraint does the mutual exclusion."
12. Fencing tokens
A monotonic term number attached to every action, checked by the resource. Stops a paused-then-resumed leader from acting after it lost authority.
Appears in: job scheduler · online auction close timers · Google Docs
"A lock gives mutual exclusion only while my process is healthy. A GC-paused process is exactly the case where it isn't, so the resource has to reject any token older than the highest it has seen."
13. Atomic check-and-act in one operation
Never GET then SET. Push the whole decision into one Lua script, one conditional write, or one INSERT ... ON CONFLICT.
Appears in: rate limiter · Ticketmaster holds · Bitly custom aliases · notification frequency caps
"Check-then-act races under concurrency, and it fails hardest exactly when load is highest. Redis is single-threaded per shard, so a Lua script is atomic without any distributed locking."
14. Double-entry ledger in integer minor units
Append-only, balanced legs per transfer, balances as projections. Never a mutable balance column, never floats.
Appears in: payments · Robinhood · Uber fares · online auction escrow
"A mutable balance column has no history, contends on writes, and has no structural invariant against a bug creating money. Double-entry with a balance check enforced per transfer makes the invariant the database's job."
15. Saga with compensations
Multi-step money movements across systems that share no transaction. Every step has an explicit compensating action, retried until it succeeds.
Appears in: payments · Ticketmaster checkout · local delivery orders
"Compensation isn't rollback — a refund is a new transaction with its own fee and its own ledger entries. And a compensation that fails is an incident, not a retry-and-forget."
16. Transactional outbox
Write state and enqueue the event in one local transaction; a relay publishes afterwards. Eliminates divergence between your database and your event stream.
Appears in: payments · local delivery · online auction
"Writing to the database and publishing to Kafka as two operations means a crash between them leaves the two permanently disagreeing — about money. One commit, then a relay."
Scale through approximation
17. Probabilistic data structures
Bloom filters for membership, HyperLogLog for cardinality, Count-Min Sketch for frequency, t-digest/DDSketch for percentiles. All mergeable, all bounded-memory.
Appears in: YouTube Top K · web crawler · Tinder · metrics · Bitly analytics
"Exact counting is 115 TB; sketches do it in 16 GB. And they're mergeable, which is what makes distributed aggregation work at all."
Cost to name: know the error bound and what a false positive does. A Bloom filter in Tinder would permanently hide a profile, which is why Roaring bitmaps are better there.
18. Approximate to nominate, exact to publish
Use sketches to narrow billions of candidates to thousands, then compute the published answer exactly over that bounded set.
Appears in: YouTube Top K · ad click aggregator · Strava segment matching
"Approximation where the input is unbounded, exactness where the output is public. The sketch picks 2,000 candidates; the OLAP store counts those 2,000 exactly."
19. Two-stage retrieval and ranking
Cheap heuristics narrow thousands to hundreds; an expensive model scores only the survivors.
Appears in: Instagram · Facebook news feed · Facebook post search · Yelp
"10,000 candidates → 500 by a two-tower dot product → 50 by the heavy model → 20 after policy re-ranking. The expensive model only ever sees fifty items."
20. Spatial pruning before geometry
An R-tree or cell index reduces millions of candidates to tens before any expensive computation.
Appears in: Strava · Yelp · Uber · Tinder
"Bounding-box pruning takes 30 million segment comparisons down to about 200 — a 150,000× reduction — and only then do I run Fréchet distance on the survivors."
Time and streams
21. Event time, watermarks, bounded lateness
Window by when the event happened, not when it arrived; allow a bounded revision period; compensate beyond it rather than rewriting history.
Appears in: ad click aggregator · YouTube Top K · metrics · Bitly
"Within the lateness window I revise the window; beyond it I book a dated correction, because rewriting a settled billing period breaks the audit."
22. Lazy evaluation instead of background sweeps
Never iterate millions of counters on a timer. Store the last-update time and compute the decay or refill when the value is touched.
Appears in: rate limiter token buckets · YouTube Top K decay · Uber TTL'd positions
"Lazy refill: no cron, no sweep. The bucket computes what would have accrued since its last touch, and idle keys expire themselves."
23. Hierarchical timing wheels
O(1) insert and O(1) tick for millions of timers, versus O(log n) for a heap. Coarser wheels cascade into finer ones.
Appears in: job scheduler · online auction close timers
"A near-term wheel in memory for the next 60 seconds, backed by time-bucketed rows on disk for everything beyond. Crash recovery is just reloading the current bucket."
24. Deterministic jitter
Spread scheduled work across a window using a hash of the entity ID, so the same item always lands on the same offset but the population is smeared.
Appears in: job scheduler · notifications quiet hours · metrics alert evaluation · Robinhood market open
"Everyone writes
0 * * * *, so 60% of jobs fire in the same second. Hash the job ID into a 60-second offset — the schedule stays evenly spaced per job, and the herd disappears."
Resilience
25. Fail open or fail closed — decide per component
Protective components must have an explicit policy, and the policy differs by what they protect.
Appears in: rate limiter · ad budget enforcement · Facebook news feed leaves
"Fail open when the component protects capacity, fail closed when it protects money. Rate limiting fails open; budget enforcement fails closed."
26. Deadline plus partial results
Fan out with a hard budget and return whatever arrived. A degraded answer beats no answer.
Appears in: Facebook news feed · Facebook post search · Yelp
"With five leaves each at p99 150ms, waiting for all of them puts me at the 99.8th percentile of a single leaf. I deadline each one, circuit-break the sick ones, and track degraded-response rate as an SLI so quality loss is visible."
27. Retry with full jitter, circuit breakers, and a retry budget
All three. Backoff alone still produces synchronised waves; a budget is the backstop when someone misconfigures the backoff.
Appears in: job scheduler · notifications · Facebook Live comments reconnects
"Full jitter, not exponential-plus-noise — the randomness is what decorrelates the wave. Then a circuit breaker so 50,000 retries become one probe, and a retry budget capped at 10% of normal traffic as the backstop."
28. Admission control at the edge
Keep the herd outside the building. Rejecting cheaply, early, is the only thing that works at 40× normal load.
Appears in: Ticketmaster waiting room · ChatGPT · local delivery · metrics cardinality limits
"You can't autoscale 40× in sixty seconds, and you can't fix contention inside the application. A signed admission token at the edge with a drip rate governed by the booking service's own health is the only mechanism that works."
29. Physical isolation beats logical priority
A priority field in one queue only reorders; the low-priority backlog still consumes the shared resource. Separate topics, workers, and connection pools.
Appears in: notifications · LeetCode contests · ChatGPT tiers
"An OTP behind a 100-million-message marketing campaign is a 30-minute wait. Priority within one queue doesn't isolate — the lanes need separate topics, separate workers, and separate provider connections."
30. Degrade the read path to protect the write path
When you must shed, shed the thing that is merely annoying, not the thing that causes harm.
Appears in: Robinhood market open · Facebook Live comments · Bitly
"A user who can't see a chart is annoyed. A user who can't cancel an order during a crash is harmed. Charts get shed first, and order cancellation never does."
Using the index
In an interview, the pattern name alone is worth little — anyone can say "consistent hashing." What earns credit is the three-part move:
- Name the failure the pattern addresses, concretely, with a number if you have one.
- Name the pattern and how it works in one sentence.
- Name what it costs, and why you accept that cost here.
That third step is the one most candidates skip, and it is the one that distinguishes someone who has used these things from someone who has read about them.