{"id":"cache","title":"Caching","sections":[{"id":"overview","title":"Overview","html":"<p>This chapter answers eight questions:</p>\n<ul>\n<li>What is a cache, and why does it make pages faster and databases less busy?</li>\n<li>Where can a cache live, from the user's browser down to the database?</li>\n<li>What should you cache, and why are whole objects easier to keep correct than query results?</li>\n<li>When does data get into the cache and back out of it? The common strategies are cache-aside, read-through, write-through, write-behind and refresh-ahead.</li>\n<li>What happens when the cache runs out of memory?</li>\n<li>How do caches fail under heavy load, and how do you stop that from happening?</li>\n<li>How do Redis and Memcached work, and how do you run Redis on more than one machine?</li>\n<li>How do you keep the cache consistent with the database?</li>\n</ul>\n<p>We'll keep using <strong>Crumb</strong>, the recipe site from the <a href=\"/learn/scalability\">Scalability</a> chapter. Crumb already runs several app servers behind a load balancer, with a Redis cache in front of its database. <a href=\"/learn/scalability#caching\">The caching section of the Scalability chapter</a> explained why Crumb added that cache. This chapter covers how to run one well.</p>","headings":[]},{"id":"what-a-cache-is","title":"What a cache is","html":"<h3 id=\"what-a-cache-is\">What a cache is</h3>\n<p>A <strong>cache</strong> is a store that keeps copies of results you have already computed or fetched. When the same request comes again, you return the saved copy instead of doing the work a second time. A few terms come up constantly:</p>\n<ul>\n<li>A <strong>cache hit</strong> is a request the cache can answer from a saved copy.</li>\n<li>A <strong>cache miss</strong> is a request the cache can't answer, so the work has to be done the slow way.</li>\n<li>The <strong>hit ratio</strong> is the share of requests that are hits. A cache with a 95% hit ratio sends only 5 requests in 100 to the slow system behind it.</li>\n<li>An <strong>entry</strong> is one saved copy, stored under a <strong>key</strong> such as <code>recipe:812</code>.</li>\n</ul>\n<p>Ricky Ho's <a href=\"http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html\" target=\"_blank\" rel=\"noopener noreferrer\">2010 catalog of scalable design patterns</a> calls this the <strong>result cache</strong> pattern. A dispatcher receives each request. It \"will first lookup if the request has been made before and try to find the previous result to return,\" which saves running the request again.</p>\n<p><strong>Crumb's recipe page.</strong> Showing recipe 812 takes five database queries: the recipe, its ingredients, its photos, its ratings and its author. Together they take about 40 ms. If Crumb stores the finished result under <code>recipe:812</code>, the next visitor gets it from memory in about a millisecond, and the database does no work at all.</p>\n<p>So a cache does two things. <strong>Pages load faster</strong>, because a hit skips the slow work. And <strong>servers and databases do less work</strong>, because hits never reach them.</p>\n<h4 id=\"caches-smooth-out-uneven-load\">Caches smooth out uneven load</h4>\n<p>Databases work best when reads and writes are spread evenly across their partitions. (A <strong>partition</strong> is one slice of the data, often on its own machine. <a href=\"/learn/relational-databases#sharding-splitting-one-table-across-databases\">Relational databases</a> covers how data gets split this way.) Popular items break that balance. If one item gets most of the traffic, the partition that holds it becomes a bottleneck while the others sit idle.</p>\n<p>Crumb sees this when a food influencer posts a link to recipe 812, and that one recipe gets thousands of views a second. Without a cache, all of them land on the one database shard that stores it. With a cache in front, almost all of them are hits, and the shard sees about one query each time the cached copy expires. The cache absorbs the uneven load and the spike.</p>","headings":[{"id":"what-a-cache-is","title":"What a cache is","depth":3},{"id":"caches-smooth-out-uneven-load","title":"Caches smooth out uneven load","depth":4}]},{"id":"where-caches-live","title":"Where caches live","html":"<h3 id=\"where-caches-live\">Where caches live</h3>\n<p>A request can be answered from a cache at several points on its way to the database. Each layer catches some requests so the layers behind it see fewer.</p>\n<figure class=\"diagram\" data-text=\"Browser cache -> CDN edge; CDN edge -> Reverse proxy cache; Reverse proxy cache -> App server; App server -> Redis or Memcached (1. check); App server -> Database buffer cache (2. on a miss)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 -15 800 256\" width=\"800\" role=\"img\" aria-label=\"A request passes several caches before it reaches the database\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"151\" y1=\"42\" x2=\"167\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"311\" y1=\"42\" x2=\"327\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"471\" y1=\"42\" x2=\"487\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"560\" y1=\"69\" x2=\"560\" y2=\"162\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"631\" y1=\"42\" x2=\"647\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-client\" x=\"12\" y=\"16\" width=\"136\" height=\"52\" rx=\"26\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"38.5\">Browser</tspan><tspan x=\"80\" y=\"54.5\">cache</tspan></text><rect class=\"dg-node dg-box\" x=\"172\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"240\" y=\"46.5\">CDN edge</tspan></text><rect class=\"dg-node dg-box\" x=\"332\" y=\"16\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"400\" y=\"38.5\">Reverse proxy</tspan><tspan x=\"400\" y=\"54.5\">cache</tspan></text><rect class=\"dg-node dg-box\" x=\"492\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"46.5\">App server</tspan></text><path class=\"dg-node dg-store\" d=\"M492,174 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M492,174 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"197.5\">Redis or</tspan><tspan x=\"560\" y=\"213.5\">Memcached</tspan></text><path class=\"dg-node dg-store\" d=\"M652,18 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M652,18 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"720\" y=\"41.5\">Database</tspan><tspan x=\"720\" y=\"57.5\">buffer cache</tspan></text><rect class=\"dg-edge-bg\" x=\"530.2\" y=\"106.5\" width=\"59.6\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"560\" y=\"119.5\" text-anchor=\"middle\">1. check</text><rect class=\"dg-edge-bg\" x=\"596.8\" y=\"-11\" width=\"84.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"639\" y=\"2\" text-anchor=\"middle\">2. on a miss</text></svg></div><figcaption>A request passes several caches before it reaches the database</figcaption></figure>\n<h4 id=\"client-caching\">Client caching</h4>\n<p>Caches can live on the client side, on the server side (for example inside a reverse proxy), or in a separate cache layer. The first one is on the user's own device. The browser keeps copies of images, scripts and pages, and the operating system keeps recent DNS answers. A request answered there never crosses the network. The server controls what the browser may keep, and for how long, with HTTP headers such as <code>Cache-Control</code>, which <a href=\"/learn/dns-and-cdn#http-caching-headers\">DNS and CDNs</a> explains.</p>\n<h4 id=\"cdn-caching\">CDN caching</h4>\n<p>A <strong>content delivery network (CDN)</strong> is a set of servers spread around the world that keep copies of your content close to users. That makes a CDN a kind of cache. Crumb's recipe photos and JavaScript come from a CDN server near each user. <a href=\"/learn/dns-and-cdn#content-delivery-networks\">DNS and CDNs</a> covers how CDNs work.</p>\n<h4 id=\"web-server-caching\">Web server caching</h4>\n<p>A <strong>reverse proxy</strong> is a server that sits in front of your app servers and receives requests on their behalf (<a href=\"/learn/reverse-proxy-and-api-gateway#forward-proxy-vs-reverse-proxy\">Reverse proxies and API gateways</a> explains them). Reverse proxies and dedicated HTTP caches can store responses and serve both static and dynamic content directly. Varnish Cache is a well-known example: \"a caching HTTP reverse proxy\", in its project's words. It was <a href=\"https://vinyl-cache.org/intro/index.html\" target=\"_blank\" rel=\"noopener noreferrer\">renamed Vinyl Cache</a> between its September 2025 and March 2026 releases. Web servers can also cache responses, so a repeated request is answered without contacting an application server.</p>\n<p>At Crumb, the logged-out version of a recipe page is the same for everyone. If the reverse proxy keeps it for 60 seconds, the app servers build that page at most once a minute.</p>\n<h4 id=\"database-caching\">Database caching</h4>\n<p>Your database already caches. It keeps recently used data pages in memory so it doesn't have to read them from disk every time. The default settings suit a generic workload, and tuning them for your own usage pattern can make a real difference.</p>\n<p>PostgreSQL is a good example. Its main cache setting, <code>shared_buffers</code>, defaults to \"typically 128 megabytes\". The <a href=\"https://www.postgresql.org/docs/current/runtime-config-resource.html\" target=\"_blank\" rel=\"noopener noreferrer\">PostgreSQL documentation</a> says that on a dedicated database server with 1 GB or more of RAM, \"a reasonable starting value for <code>shared_buffers</code> is 25% of the memory in your system.\" It adds that PostgreSQL \"also relies on the operating system cache\", so going above about 40% of RAM is unlikely to help.</p>\n<p>Not every built-in cache worked out. <a href=\"/learn/relational-databases#the-query-cache-and-why-its-gone\">Relational databases</a> tells how MySQL removed its query cache because it didn't scale on many-core machines.</p>\n<h4 id=\"application-caching\">Application caching</h4>\n<p><strong>Application caching</strong> means your code stores results in a cache that you control. The usual tools are <strong>Memcached</strong> and <strong>Redis</strong>. Both are <strong>in-memory key-value stores</strong>: you store a value under a key, and you fetch it back by the same key. They sit between your application and your database.</p>\n<p>They are fast because they keep data in RAM. A typical database stores its data on disk, and reading memory is much faster than reading disk. Will Larson's <a href=\"https://lethain.com/introduction-to-architecting-systems-for-scale/\" target=\"_blank\" rel=\"noopener noreferrer\">introduction to architecting systems for scale</a> puts it this way: \"accesses to RAM are orders of magnitude faster than those to disk.\"</p>\n<p>RAM is also much smaller and more expensive than disk, so a cache can't hold everything. It needs a rule for what to remove when it is full. The most common rule is <strong>least recently used (LRU)</strong>: throw out the entry that has gone longest without being read. This removes \"cold\" entries that nobody is asking for and keeps \"hot\" entries in memory. <a href=\"/learn/cache#when-the-cache-is-full\">When the cache is full</a> covers these rules in detail.</p>\n<p>Redis does more than Memcached. It has:</p>\n<ul>\n<li><strong>A persistence option</strong>, so it can save its data to disk and reload it after a restart.</li>\n<li><strong>Built-in data structures</strong> such as lists, sets and sorted sets, with commands that work on them inside the server.</li>\n</ul>\n<p><a href=\"/learn/cache#redis-internals-and-memcached\">Redis internals and Memcached</a> compares the two in depth.</p>","headings":[{"id":"where-caches-live","title":"Where caches live","depth":3},{"id":"client-caching","title":"Client caching","depth":4},{"id":"cdn-caching","title":"CDN caching","depth":4},{"id":"web-server-caching","title":"Web server caching","depth":4},{"id":"database-caching","title":"Database caching","depth":4},{"id":"application-caching","title":"Application caching","depth":4}]},{"id":"what-to-cache","title":"What to cache","html":"<h3 id=\"what-to-cache\">What to cache</h3>\n<p>Application caches usually hold one of two kinds of thing: the results of <strong>database queries</strong>, or <strong>objects</strong> your code has assembled. Within those, you can cache at four levels:</p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th>Level</th><th>What you store</th><th>Crumb example</th></tr></thead><tbody><tr><td>Row</td><td>One row from one table</td><td>The <code>recipes</code> row for recipe 812</td></tr><tr><td>Query</td><td>The full result of one query</td><td>The 20 rows returned by \"top Thai recipes\"</td></tr><tr><td>Object</td><td>A complete, serializable object built from several queries</td><td>Recipe 812 with its ingredients, photos, ratings and author</td></tr><tr><td>Rendered HTML</td><td>A finished page or page fragment</td><td>The HTML for the recipe 812 page</td></tr></tbody></table></div>\n<p>(A <strong>serializable</strong> object is one you can turn into bytes, for example as JSON, and turn back into the same object later.)</p>\n<p>Avoid caching in files on each app server's disk. If one server writes <code>recipe-812.html</code> to its own disk, the other servers can't see it, and a new server starts with nothing. That makes <a href=\"/learn/scalability#scaling-out-clones-behind-a-load-balancer\">cloning servers</a> and autoscaling harder. Keep the cache in a shared store such as Redis or Memcached.</p>\n<h4 id=\"caching-at-the-database-query-level\">Caching at the database query level</h4>\n<p>The simplest approach is to cache every query result. Hash the SQL text to make a key, and store the result under it. The next time the same query runs, look up the hash first.</p>\n<p>This is easy to add, but hard to keep correct, because you can't easily tell which cached results to delete when data changes:</p>\n<ul>\n<li><strong>Complex queries are hard to delete.</strong> A query that joins four tables could be affected by a change to any of them.</li>\n<li><strong>One small change can affect many entries.</strong> If a single recipe's rating changes, every cached query that might include it is now wrong: \"top Thai recipes\", \"top-rated this week\", and hundreds more. You can't cheaply find them all, so you either delete too much or leave stale results behind.</li>\n</ul>\n<h4 id=\"caching-at-the-object-level\">Caching at the object level</h4>\n<p>The other approach treats your data as objects, the way your application code already does. Your application runs the queries, assembles the results into a class instance or data structure, and caches that whole object. Crumb caches recipe 812 as one object under <code>recipe:812</code>.</p>\n<p>This has two advantages:</p>\n<ul>\n<li><strong>When the underlying data changes, you delete one key.</strong> If someone edits or rates recipe 812, Crumb deletes <code>recipe:812</code>, and the next request rebuilds it.</li>\n<li><strong>It allows asynchronous processing.</strong> Background workers can assemble objects ahead of time, and other workers can consume the latest cached object instead of querying the database. <a href=\"/learn/queues-and-async\">Queues and async</a> covers background workers.</li>\n</ul>\n<p>Good candidates for object caching include:</p>\n<ul>\n<li>user sessions,</li>\n<li>fully rendered web pages,</li>\n<li>activity streams, such as \"recent recipes from people you follow\",</li>\n<li>user graph data, such as who follows whom.</li>\n</ul>","headings":[{"id":"what-to-cache","title":"What to cache","depth":3},{"id":"caching-at-the-database-query-level","title":"Caching at the database query level","depth":4},{"id":"caching-at-the-object-level","title":"Caching at the object level","depth":4}]},{"id":"when-to-update-the-cache","title":"When to update the cache","html":"<h3 id=\"when-to-update-the-cache\">When to update the cache</h3>\n<p>A cache holds a limited amount of data, and that data has to stay close to what the database says. So every cache needs answers to two questions: how does data get into the cache, and what happens to the cached copy when the data changes? The common strategies answer those questions differently.</p>\n<p>We'll call the code that talks to the cache \"the application\" and the system of record \"the database\". (The <strong>system of record</strong> is the store whose copy counts as the truth.)</p>\n<h4 id=\"cache-aside\">Cache-aside</h4>\n<p>In <strong>cache-aside</strong>, the application does all the work. The cache never talks to the database. To read, the application:</p>\n<ol>\n<li>looks for the entry in the cache, and gets a miss,</li>\n<li>loads the entry from the database,</li>\n<li>adds the entry to the cache,</li>\n<li>returns the entry.</li>\n</ol>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> get_recipe</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(recipe_id):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    key </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> f</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"recipe:</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">{</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">recipe_id</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">}</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    cached </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> cache.get(key)                  </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># trip 1: check the cache</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> cached </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">is</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> not</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">        return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> json.loads(cached)            </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># hit: done</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    recipe </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> db.load_recipe(recipe_id)       </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># trip 2: read the database</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    if</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> recipe </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">is</span><span style=\"color:#D73A49;--shiki-dark:#F97583\"> not</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\"> None</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">:</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">        cache.set(key, json.dumps(recipe), </span><span style=\"color:#E36209;--shiki-dark:#FFAB70\">ex</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">3600</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">)  </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># trip 3: fill the cache</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> recipe</span></span></code></pre>\n<p>To write, the application updates the database and then deletes the cached copy, so the next read loads the new value. <a href=\"/learn/cache#keeping-the-cache-consistent-with-the-database\">Keeping the cache consistent with the database</a> explains why deleting is safer than updating.</p>\n<p>Memcached is generally used this way. Cache-aside is also called <strong>lazy loading</strong>, because data enters the cache only when someone asks for it. The <a href=\"https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/Strategies.html\" target=\"_blank\" rel=\"noopener noreferrer\">AWS ElastiCache caching strategies guide</a> lists its benefits. Only requested data is cached, so the cache doesn't fill up with data nobody reads. Reads after the first miss are fast. And a failed cache node isn't fatal: the application keeps working, more slowly, and refills the new node as requests arrive.</p>\n<p>Its disadvantages:</p>\n<ul>\n<li><strong>Each miss costs three trips:</strong> the cache check, the database query and the cache write. That can add a noticeable delay to the first request.</li>\n<li><strong>Data can go stale.</strong> If the database changes and nobody deletes the cached copy, for example because another program wrote to the database, readers keep getting the old value. A <strong>time to live (TTL)</strong> limits the damage. It is an expiry time on each entry, after which the next read fetches a fresh copy. Write-through, below, is another fix.</li>\n<li><strong>A replaced node starts empty.</strong> When a cache node fails, every request to its replacement misses until it warms up, so latency goes up in the meantime.</li>\n</ul>\n<h4 id=\"read-through\">Read-through</h4>\n<p><strong>Read-through</strong> moves the loading step into the cache itself. The application only ever asks the cache. On a miss, the cache loads the entry from the database, keeps a copy and returns it, as Oracle's <a href=\"https://docs.oracle.com/en/middleware/standalone/coherence/14.1.1.2206/develop-applications/caching-data-sources.html\" target=\"_blank\" rel=\"noopener noreferrer\">Coherence documentation</a> describes. Reads behave like cache-aside, and a miss costs the same. The difference is where the code lives: one loader inside the caching layer instead of lookup logic repeated across the application. Plain Redis and Memcached don't load from your database on their own, so read-through usually comes from a caching library or a data grid product.</p>\n<h4 id=\"write-through\">Write-through</h4>\n<p>In <strong>write-through</strong>, the application treats the cache as its main data store. It reads from the cache and writes to the cache, and the cache is responsible for reading from and writing to the database. A write goes like this:</p>\n<ol>\n<li>The application adds or updates the entry in the cache.</li>\n<li>The cache writes the entry to the database, and waits for the write to finish.</li>\n<li>The write returns.</li>\n</ol>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># Inside the caching layer. The application just calls save_recipe(812, {...}).</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> save_recipe</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(recipe_id, fields):</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    recipe </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> db.update_recipe(recipe_id, fields)          </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># synchronous database write</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    cache.set(</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">f</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"recipe:</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">{</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">recipe_id</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">}</span><span style=\"color:#032F62;--shiki-dark:#9ECBFF\">\"</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, json.dumps(recipe))  </span><span style=\"color:#6A737D;--shiki-dark:#6A737D\"># cache now matches the database</span></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">    return</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> recipe</span></span></code></pre>\n<p>Each write is slower, because it waits for the database. Reads of just-written data are fast, because it's already in the cache. The AWS guide notes that \"end users are generally more tolerant of latency when updating data than when retrieving data\", so a Crumb user saving a recipe won't mind an extra 15 ms. And the cached data doesn't go stale, as long as every write goes through the cache.</p>\n<p>Its disadvantages:</p>\n<ul>\n<li><strong>A new node starts empty and stays that way.</strong> A node added after a failure or to scale out holds nothing until each entry is written again, so a recipe nobody edits never gets in. Combining write-through with cache-aside fixes this: reads fill in anything missing.</li>\n<li><strong>Most data written may never be read.</strong> AWS calls this \"cache churn\". A TTL limits the wasted memory, because unread entries expire.</li>\n</ul>\n<h4 id=\"write-behind-write-back\">Write-behind (write-back)</h4>\n<p><strong>Write-behind</strong>, also called <strong>write-back</strong>, makes writes fast by delaying the database write:</p>\n<ol>\n<li>The application adds or updates the entry in the cache, and the write returns at once.</li>\n<li>The cache writes the entry to the database later, asynchronously (in the background, without the application waiting for it).</li>\n</ol>\n<p>This improves write performance. It can also save database work, because, in the Coherence documentation's words, \"multiple changes to the same object within the write-behind interval are 'coalesced' and only written once\". If Crumb counts views and recipe 812 gets 5,000 in ten seconds, write-behind turns them into one database write of the final count.</p>\n<p>Its disadvantages:</p>\n<ul>\n<li><strong>Data can be lost.</strong> If the cache goes down before its queued writes reach the database, those writes are gone, even though the application was told they succeeded. Until then the cache is effectively the system of record.</li>\n<li><strong>It is more complex</strong> than cache-aside or write-through. You need a reliable queue of pending writes, retries, and a plan for writes that arrive out of order. If another program writes the same rows directly, a queued write can later overwrite its newer value.</li>\n</ul>\n<h4 id=\"refresh-ahead\">Refresh-ahead</h4>\n<p><strong>Refresh-ahead</strong> reloads popular entries before they expire. You configure the cache to refresh any recently accessed entry automatically when it gets close to its expiry time, while readers keep getting the current copy.</p>\n<p>The Coherence documentation gives an example. Entries expire after 60 seconds, and the refresh-ahead factor is 0.5. A read of an entry between 30 and 60 seconds old returns the cached value at once and schedules a background reload. A read after 60 seconds waits for a normal load.</p>\n<p>Refresh-ahead can give lower latency than read-through, because readers of popular entries never wait for a miss. The <a href=\"https://docs.oracle.com/en/middleware/standalone/coherence/14.1.1.2206/develop-applications/caching-data-sources.html\" target=\"_blank\" rel=\"noopener noreferrer\">Coherence documentation</a> says this holds \"only if the cache can accurately predict which cache items are likely to be needed in the future.\"</p>\n<p>Its disadvantage: wrong predictions reload entries nobody will read. That sends extra queries to the database, and performance can end up worse than with no refresh-ahead at all. <a href=\"/learn/cache#fix-2-probabilistic-early-expiration-xfetch\">XFetch</a>, later in this chapter, is a refresh-ahead method that uses probability to decide when to refresh.</p>\n<h4 id=\"trying-the-strategies\">Trying the strategies</h4>\n<p>The diagram below holds one record, <code>user:42</code>, with a cache in front of a database. Pick cache-aside, write-through or write-behind, then read and write it. In write-through mode, reads also go through the cache, as in read-through. The timings are illustrative: 1 ms for the cache, 10 ms for a database read and 15 ms for a database write.</p>\n<div class=\"widget-slot\" data-widget=\"cache-strategies\"></div>\n<p>Some things to try:</p>\n<ul>\n<li><strong>Cache-aside:</strong> read, then press <em>Another service writes the database</em>. Reads stay stale until you press <em>Entry expires</em>.</li>\n<li><strong>Write-behind:</strong> write twice quickly, and the two writes become one database write. Then write again and press <em>Cache node restarts</em> before the queued write runs. That write is lost.</li>\n</ul>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th>Strategy</th><th>Who loads data on a miss</th><th>When the database is written</th><th>Main risk</th></tr></thead><tbody><tr><td>Cache-aside</td><td>The application</td><td>By the application, before deleting the cached copy</td><td>Stale data, slow first read</td></tr><tr><td>Read-through</td><td>The cache</td><td>Depends on the write strategy paired with it</td><td>Slow first read</td></tr><tr><td>Write-through</td><td>The cache (usually paired with read-through)</td><td>By the cache, before the write returns</td><td>Slower writes, cache full of unread data</td></tr><tr><td>Write-behind</td><td>The cache</td><td>By the cache, later</td><td>Lost writes if the cache fails</td></tr><tr><td>Refresh-ahead</td><td>The cache, before expiry</td><td>Depends on the write strategy paired with it</td><td>Wasted reloads if it guesses wrong</td></tr></tbody></table></div>","headings":[{"id":"when-to-update-the-cache","title":"When to update the cache","depth":3},{"id":"cache-aside","title":"Cache-aside","depth":4},{"id":"read-through","title":"Read-through","depth":4},{"id":"write-through","title":"Write-through","depth":4},{"id":"write-behind-write-back","title":"Write-behind (write-back)","depth":4},{"id":"refresh-ahead","title":"Refresh-ahead","depth":4},{"id":"trying-the-strategies","title":"Trying the strategies","depth":4}]},{"id":"when-the-cache-is-full","title":"When the cache is full","html":"<h3 id=\"when-the-cache-is-full\">When the cache is full</h3>\n<p>Entries leave a cache in two ways. With <strong>expiry</strong>, an entry's TTL runs out. With <strong>eviction</strong>, the cache is out of memory and removes entries to make room. The <strong>eviction policy</strong> decides which entries go:</p>\n<ul>\n<li><strong>Least recently used (LRU):</strong> remove the entry that has gone longest without being read.</li>\n<li><strong>Least frequently used (LFU):</strong> remove the entry that is read least often.</li>\n<li><strong>Random:</strong> remove any entry.</li>\n</ul>\n<p>LRU suits most caches, because a small set of hot keys usually gets most of the reads. At Crumb, this week's popular recipes stay in memory, and recipes nobody has opened for months get evicted.</p>\n<p><strong>Redis.</strong> By default, Redis on a 64-bit system has no memory limit, so it doesn't evict anything. To use it as a cache, you set <code>maxmemory</code> and choose a <code>maxmemory-policy</code>. The <a href=\"https://redis.io/docs/latest/develop/reference/eviction/\" target=\"_blank\" rel=\"noopener noreferrer\">Redis eviction documentation</a> lists the options. A few of them:</p>\n<ul>\n<li><code>noeviction</code> keeps every key and returns an error when a command tries to add more data.</li>\n<li><code>allkeys-lru</code> evicts the least recently used keys. The docs call it \"a good default option if you have no reason to prefer any others.\"</li>\n<li><code>allkeys-lfu</code> evicts the least frequently used keys (since Redis 4.0).</li>\n<li><code>volatile-lru</code> and the other <code>volatile-</code> policies only evict keys that have a TTL, which suits an instance that also holds data you must keep.</li>\n</ul>\n<p>Redis approximates LRU to save memory. It samples a few keys at random (5 by default, set with <code>maxmemory-samples</code>) and evicts the one unused for longest. The docs call this \"virtually equivalent\" to true LRU for applications.</p>\n<p><strong>Memcached.</strong> Memcached divides memory into slab classes by item size, and <a href=\"https://github.com/memcached/memcached/wiki/UserInternals\" target=\"_blank\" rel=\"noopener noreferrer\">each class has \"its own LRU\"</a>. When a class needs room, Memcached looks at the tail of that LRU for an already expired item, and evicts an unexpired one only if it finds none.</p>\n<p>Redis's <code>INFO stats</code> reports <code>keyspace_hits</code>, <code>keyspace_misses</code> and <code>evicted_keys</code>. A low hit ratio with many evictions means the cache is too small or the policy is evicting the wrong keys.</p>","headings":[{"id":"when-the-cache-is-full","title":"When the cache is full","depth":3}]},{"id":"the-costs-of-caching","title":"The costs of caching","html":"<h3 id=\"the-costs-of-caching\">The costs of caching</h3>\n<p>A cache isn't free. Before adding one, weigh three costs:</p>\n<ul>\n<li><strong>You have to keep it consistent with the database.</strong> The database is the source of truth. When data changes there, cached copies have to be updated or removed. This is called <strong>cache invalidation</strong>.</li>\n<li><strong>Invalidation is hard.</strong> Deciding when to update the cache, on every path that changes data, adds real complexity. Stale data, races between writers and lost deletes all come from this.</li>\n<li><strong>You have to change the application.</strong> You add Redis or Memcached, run and monitor it, and write code that uses it.</li>\n</ul>\n<p>A cache also becomes a dependency. If the database copes only because the cache absorbs most reads, a cache failure can take the database down too. <a href=\"/learn/scalability#caching\">Scalability</a> tells how this happened to Facebook in 2010.</p>","headings":[{"id":"the-costs-of-caching","title":"The costs of caching","depth":3}]},{"id":"failure-modes-and-how-to-prevent-them","title":"Failure modes and how to prevent them","html":"<h3 id=\"failure-modes-and-how-to-prevent-them\">Failure modes and how to prevent them</h3>\n<p>At 50,000 requests a second, small cache problems become large database problems. Three failure modes come up again and again. All of them end with requests the cache was meant to absorb reaching the database faster than it can serve them.</p>\n<figure class=\"diagram\" data-text=\"Stampede one hot key expires -> Database overloaded (same query × N); Avalanche many keys expire or cache restarts -> Database overloaded (misses on every key); Penetration keys that don&#x27;t exist -> Database overloaded (every lookup)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 0 640 265\" width=\"640\" role=\"img\" aria-label=\"Three ways load gets past the cache\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge dg-dashed\" x1=\"129.3\" y1=\"79\" x2=\"272\" y2=\"186\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"320\" y1=\"79\" x2=\"320\" y2=\"186\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"510.7\" y1=\"79\" x2=\"368\" y2=\"186\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-box\" x=\"12\" y=\"8\" width=\"136\" height=\"68\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"30.5\">Stampede</tspan><tspan x=\"80\" y=\"46.5\">one hot key</tspan><tspan x=\"80\" y=\"62.5\">expires</tspan></text><rect class=\"dg-node dg-box\" x=\"252\" y=\"8\" width=\"136\" height=\"68\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"30.5\">Avalanche</tspan><tspan x=\"320\" y=\"46.5\">many keys expire</tspan><tspan x=\"320\" y=\"62.5\">or cache restarts</tspan></text><rect class=\"dg-node dg-box\" x=\"492\" y=\"8\" width=\"136\" height=\"68\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"30.5\">Penetration</tspan><tspan x=\"560\" y=\"46.5\">keys that</tspan><tspan x=\"560\" y=\"62.5\">don't exist</tspan></text><path class=\"dg-node dg-store\" d=\"M252,198 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M252,198 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"221.5\">Database</tspan><tspan x=\"320\" y=\"237.5\">overloaded</tspan></text><rect class=\"dg-edge-bg\" x=\"152.3\" y=\"123.5\" width=\"96.8\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"200.7\" y=\"136.5\" text-anchor=\"middle\">same query × N</text><rect class=\"dg-edge-bg\" x=\"256.1\" y=\"123.5\" width=\"127.8\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"320\" y=\"136.5\" text-anchor=\"middle\">misses on every key</text><rect class=\"dg-edge-bg\" x=\"397.1\" y=\"123.5\" width=\"84.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"439.3\" y=\"136.5\" text-anchor=\"middle\">every lookup</text></svg></div><figcaption>Three ways load gets past the cache</figcaption></figure>\n<h4 id=\"cache-stampede-thundering-herd\">Cache stampede (thundering herd)</h4>\n<p>A <strong>cache stampede</strong>, also called a <strong>thundering herd</strong>, starts when a very popular key expires or is evicted. Every request before someone puts the value back is a miss, and every miss runs the same query. As an illustration, say recipe 812's key gets 10,000 reads a second and takes 200 ms to rebuild:</p>\n<ul>\n<li>While the key is missing, 10,000 × 0.2 s = <strong>2,000 requests</strong> miss it.</li>\n<li>Each of them runs the same expensive query.</li>\n<li>The database's connection pool fills up and queries queue. The rebuild now takes longer than 200 ms, so the key stays missing longer and even more requests miss. Without a fix, this feeds on itself.</li>\n</ul>\n<h5 id=\"fix-1-a-lock-so-only-one-request-rebuilds\">Fix 1: a lock, so only one request rebuilds</h5>\n<p>On a miss, a request first tries to take a short-lived lock for that key. In Redis that is one command: <code>SET lock:&#x3C;key> &#x3C;random-token> NX PX 5000</code>. It succeeds only if the lock doesn't exist yet, and the lock expires after 5 seconds in case its holder dies.</p>\n<ul>\n<li><strong>The request that gets the lock</strong> queries the database, writes the value into the cache and releases the lock. It releases it with a small Lua script that deletes the lock only if it still holds that request's token, so it never deletes a lock that another request has since taken.</li>\n<li><strong>Every other request</strong> waits briefly, say 50 ms, and reads the cache again, or serves a slightly stale copy if you keep one.</li>\n</ul>\n<p>Each miss now costs a lock round trip, but the database sees one query instead of 2,000. Facebook's memcache paper describes a similar idea called <strong>leases</strong>: the cache gives a token to the first client that misses and tells the others to retry shortly.</p>\n<h5 id=\"fix-2-probabilistic-early-expiration-xfetch\">Fix 2: probabilistic early expiration (XFetch)</h5>\n<p>The 2015 paper <em>Optimal Probabilistic Cache Stampede Prevention</em>, by Andrea Vattani, Flavio Chierichetti and Keegan Lowenstein, describes an algorithm called <strong>XFetch</strong>. Each reader decides at random whether to refresh the key early. The chance of refreshing rises as the expiry time gets closer.</p>\n<p>Store two extra values with each entry: <code>delta</code>, how long the last rebuild took, and <code>expiry</code>, when the entry expires. On every read:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span>if now − delta × beta × ln(rand()) ≥ expiry:</span></span>\n<span class=\"line\"><span>    recompute the value, store it with a new expiry and a new delta</span></span>\n<span class=\"line\"><span>else:</span></span>\n<span class=\"line\"><span>    return the cached value</span></span></code></pre>\n<p><code>rand()</code> is a uniform random number in (0, 1], so <code>−delta × beta × ln(rand())</code> is a positive, random \"head start\" that averages <code>delta × beta</code>. In words: refresh early if the head start is larger than the time left before expiry. <code>beta</code> defaults to 1, and larger values refresh earlier.</p>\n<p>Far from expiry, the head start almost never covers the time left, so nobody refreshes. In the last few multiples of <code>delta</code>, the chance climbs quickly, and with many readers one of them very likely refreshes before the key expires. Scaling by <code>delta</code> means a value that takes 2 seconds to rebuild starts refreshing much earlier than one that takes 20 ms.</p>\n<p>XFetch needs no locks. It doesn't guarantee a single refresh, since two readers can win the draw at nearly the same moment. The paper shows that this exponential rule is the best balance between refreshing too early, which wastes work, and too late, which allows a stampede. Teams often add a lock as a backstop.</p>\n<h5 id=\"fix-3-refresh-in-the-background\">Fix 3: refresh in the background</h5>\n<p>For a handful of known hot keys, don't let them expire. A background job rebuilds them on a schedule, so readers never miss. This is refresh-ahead run by your own code. It works for a small, known set of keys and doesn't scale to millions.</p>\n<h4 id=\"cache-avalanche\">Cache avalanche</h4>\n<p>A <strong>cache avalanche</strong> is the same problem across many keys: a large share of the key space misses at once. It has two common causes:</p>\n<ul>\n<li><strong>Keys that expire together.</strong> A batch job warms the cache at 02:00 and gives every key a TTL of exactly one hour. At 03:00 they all expire within the same second.</li>\n<li><strong>A cache node restarting empty.</strong> If a Redis node holding a third of the keys comes back with no data, every request for those keys misses until they are rebuilt.</li>\n</ul>\n<h5 id=\"fix-add-jitter-to-ttls\">Fix: add jitter to TTLs</h5>\n<p>Add a random amount, called <strong>jitter</strong>, to each key's TTL:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">import</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> random</span></span>\n<span class=\"line\"></span>\n<span class=\"line\"><span style=\"color:#D73A49;--shiki-dark:#F97583\">def</span><span style=\"color:#6F42C1;--shiki-dark:#B392F0\"> cache_set_with_jitter</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(redis, key, value, base_ttl</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">3600</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">, jitter_pct</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">0.15</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">):</span></span>\n<span class=\"line\"><span style=\"color:#6A737D;--shiki-dark:#6A737D\">    # ±15% of one hour: somewhere between 3,060 and 4,140 seconds</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    spread </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> base_ttl </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">*</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> jitter_pct</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    ttl </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> base_ttl </span><span style=\"color:#D73A49;--shiki-dark:#F97583\">+</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\"> random.uniform(</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">-</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">spread, spread)</span></span>\n<span class=\"line\"><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">    redis.set(key, value, </span><span style=\"color:#E36209;--shiki-dark:#FFAB70\">ex</span><span style=\"color:#D73A49;--shiki-dark:#F97583\">=</span><span style=\"color:#005CC5;--shiki-dark:#79B8FF\">int</span><span style=\"color:#24292E;--shiki-dark:#E1E4E8\">(ttl))</span></span></code></pre>\n<p>The keys now expire over about 18 minutes, and the database sees a steady trickle of rebuilds.</p>\n<p>Jitter doesn't help with a restart. For that, use replicas that keep their data (<a href=\"/learn/cache#redis-sentinel-vs-redis-cluster\">Sentinel and Cluster</a> below), warm a new node before it takes traffic, and put a <a href=\"/learn/reliability-patterns#circuit-breakers-bulkheads-and-load-shedding\">circuit breaker or load shedding</a> in front of the database to turn away a burst of misses.</p>\n<h4 id=\"cache-penetration\">Cache penetration</h4>\n<p><strong>Cache penetration</strong> happens when clients ask for keys that don't exist in the database, such as <code>GET /users/-9999</code> or a scanner trying random IDs. With plain cache-aside, each request misses, finds no row in the database, and caches nothing, so the next request for the same ID does the same. A client sending 20,000 made-up IDs a second sends 20,000 queries a second straight to the database.</p>\n<h5 id=\"fix-1-cache-the-not-found\">Fix 1: cache the \"not found\"</h5>\n<p>When the database returns nothing, cache a marker such as <code>redis.set(\"user:-9999\", \"__none__\", ex=60)</code>. For the next 60 seconds, requests for that ID get a 404 from the cache. Keep the TTL short, because the ID might be created later. This helps when the same missing IDs are requested again and again. It does little against random IDs, and it fills the cache with markers.</p>\n<h5 id=\"fix-2-a-bloom-filter-in-front-of-the-cache\">Fix 2: a Bloom filter in front of the cache</h5>\n<p>A <strong>Bloom filter</strong> is a compact data structure that answers \"is this key in the set?\" with either \"definitely not\" or \"probably yes\". It is an array of m bits and k hash functions. To add a key, hash it k times and set those k bits to 1. To check a key, hash it k times. If any of those bits is 0, the key was never added. If all are 1, it was probably added, though other keys might have set those bits (a <strong>false positive</strong>). About 9.6 bits per key gives a 1% false positive rate, so 100 million keys fit in roughly 120 MB.</p>\n<p>To use it against penetration, add every ID to the filter when its row is created, and check the filter before the cache. On \"definitely not\", return 404 without touching the cache or the database. On \"probably yes\", carry on as usual. About 1% of requests for missing IDs still get through, which is fine.</p>\n<figure class=\"diagram\" data-text=\"Client GET /users/:id <-> Bloom filter (&#x22;definitely not&#x22;: 404); Bloom filter -> Redis cache (&#x22;probably yes&#x22;); Redis cache -> Database (cache miss)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 -13 640 240\" width=\"640\" role=\"img\" aria-label=\"A Bloom filter turns away IDs that don&#x27;t exist\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"153\" y1=\"42\" x2=\"247\" y2=\"42\" marker-end=\"url(#dg-arrow)\" marker-start=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"391\" y1=\"42\" x2=\"487\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"560\" y1=\"74\" x2=\"560\" y2=\"152\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-client\" x=\"12\" y=\"16\" width=\"136\" height=\"52\" rx=\"26\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"38.5\">Client</tspan><tspan x=\"80\" y=\"54.5\">GET /users/:id</tspan></text><rect class=\"dg-node dg-box\" x=\"252\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"46.5\">Bloom filter</tspan></text><path class=\"dg-node dg-store\" d=\"M492,20 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M492,20 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"49.5\">Redis cache</tspan></text><path class=\"dg-node dg-store\" d=\"M492,164 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M492,164 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"193.5\">Database</tspan></text><rect class=\"dg-edge-bg\" x=\"129.9\" y=\"-6\" width=\"140.2\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"200\" y=\"7\" text-anchor=\"middle\">\"definitely not\": 404</text><rect class=\"dg-edge-bg\" x=\"390.6\" y=\"-9\" width=\"96.8\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"439\" y=\"4\" text-anchor=\"middle\">\"probably yes\"</text><rect class=\"dg-edge-bg\" x=\"524\" y=\"104\" width=\"72\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"560\" y=\"117\" text-anchor=\"middle\">cache miss</text></svg></div><figcaption>A Bloom filter turns away IDs that don't exist</figcaption></figure>\n<p>There are two caveats. A plain Bloom filter can't remove keys, so deleted IDs stay \"probably yes\" until you rebuild it (counting Bloom filters and cuckoo filters support deletes). And the filter must learn about a new ID in the same code path that creates the row, or a new user can briefly get a 404.</p>","headings":[{"id":"failure-modes-and-how-to-prevent-them","title":"Failure modes and how to prevent them","depth":3},{"id":"cache-stampede-thundering-herd","title":"Cache stampede (thundering herd)","depth":4},{"id":"cache-avalanche","title":"Cache avalanche","depth":4},{"id":"cache-penetration","title":"Cache penetration","depth":4}]},{"id":"local-and-distributed-caches","title":"Local and distributed caches","html":"<h3 id=\"local-and-distributed-caches\">Local and distributed caches</h3>\n<p>So far, \"the cache\" has meant a shared server such as Redis. A cache can also live inside each application process. Large systems often use both.</p>\n<figure class=\"diagram\" data-text=\"Client -> App server 1 L1 local cache; Client -> App server 2 L1 local cache; App server 1 L1 local cache -> L2: Redis cluster (L1 miss); App server 2 L1 local cache -> L2: Redis cluster (L1 miss); L2: Redis cluster -> Database (L2 miss)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 0 640 287\" width=\"640\" role=\"img\" aria-label=\"A two-tier cache\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"151\" y1=\"84.3\" x2=\"247\" y2=\"60.3\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"151\" y1=\"119.8\" x2=\"247\" y2=\"143.8\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"391\" y1=\"59.8\" x2=\"487\" y2=\"83.8\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"391\" y1=\"144.3\" x2=\"487\" y2=\"120.3\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"560\" y1=\"136\" x2=\"560\" y2=\"212\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-client\" x=\"12\" y=\"78\" width=\"136\" height=\"48\" rx=\"24\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"106.5\">Client</tspan></text><rect class=\"dg-node dg-box\" x=\"252\" y=\"16\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"38.5\">App server 1</tspan><tspan x=\"320\" y=\"54.5\">L1 local cache</tspan></text><rect class=\"dg-node dg-box\" x=\"252\" y=\"136\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"158.5\">App server 2</tspan><tspan x=\"320\" y=\"174.5\">L1 local cache</tspan></text><path class=\"dg-node dg-store\" d=\"M492,78 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M492,78 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"101.5\">L2: Redis</tspan><tspan x=\"560\" y=\"117.5\">cluster</tspan></text><path class=\"dg-node dg-store\" d=\"M492,224 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M492,224 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"253.5\">Database</tspan></text><rect class=\"dg-edge-bg\" x=\"412.3\" y=\"62.8\" width=\"53.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"439\" y=\"75.8\" text-anchor=\"middle\">L1 miss</text><rect class=\"dg-edge-bg\" x=\"412.3\" y=\"123.3\" width=\"53.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"439\" y=\"136.3\" text-anchor=\"middle\">L1 miss</text><rect class=\"dg-edge-bg\" x=\"533.3\" y=\"165\" width=\"53.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"560\" y=\"178\" text-anchor=\"middle\">L2 miss</text></svg></div><figcaption>A two-tier cache</figcaption></figure>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th></th><th>Local, in-process (Caffeine, Guava, a plain LRU map)</th><th>Distributed (Redis, Memcached)</th></tr></thead><tbody><tr><td>Where it lives</td><td>In the application's own memory</td><td>On separate servers, reached over the network</td></tr><tr><td>Read latency</td><td>Well under a microsecond: a hash lookup in memory</td><td>Typically a few hundred microseconds to about a millisecond, mostly network round trip</td></tr><tr><td>Throughput</td><td>Limited by the app's CPU, often millions of lookups a second</td><td>Roughly 100,000 or more simple operations a second per Redis node</td></tr><tr><td>Consistency</td><td>Each server has its own copy, and copies can disagree</td><td>One copy per key, shared by every app server</td></tr><tr><td>Size</td><td>Limited by the process's memory, usually a few GB</td><td>Can grow to terabytes by adding nodes</td></tr><tr><td>Survives a deploy</td><td>No, it's empty after a restart</td><td>Yes</td></tr></tbody></table></div>\n<p>Using both is called a <strong>two-tier cache</strong>. Each app server keeps a small local cache (L1) with a TTL of a few seconds, which limits how stale it can get. On an L1 miss, it reads the shared cache (L2). When an entity changes, a message on Redis pub/sub or Kafka tells every app server to drop its L1 copy, and the short TTL covers any lost message. The L1 takes the hottest keys off the network, which also stops one hot key from overloading a single Redis node. <a href=\"/learn/consistent-hashing#hot-keys\">Consistent hashing</a> covers other ways to handle hot keys.</p>","headings":[{"id":"local-and-distributed-caches","title":"Local and distributed caches","depth":3}]},{"id":"redis-internals-and-memcached","title":"Redis internals and Memcached","html":"<h3 id=\"redis-internals-and-memcached\">Redis internals and Memcached</h3>\n<p>A single Redis process commonly handles more than 100,000 simple operations a second. Keeping data in RAM is part of the reason. Two design choices explain the rest.</p>\n<h4 id=\"a-single-thread-for-commands\">A single thread for commands</h4>\n<p>Redis runs every command on <strong>one thread</strong>.</p>\n<ul>\n<li>There are no locks and no contention between threads. Each command runs to completion before the next starts, so every single command is atomic. Two clients running <code>INCR</code> at once can't lose an update.</li>\n<li>One thread serves thousands of connections using I/O multiplexing (<strong>epoll</strong> on Linux, <strong>kqueue</strong> on macOS and BSD): the kernel reports which sockets have data, and Redis handles each in turn.</li>\n<li>Since Redis 6.0, socket reads and writes can use extra I/O threads, but commands still run on the main thread.</li>\n</ul>\n<p>The cost is that one slow command blocks everyone. <code>KEYS *</code> on a large database, or deleting a set with millions of members, stalls every client. Use <code>SCAN</code> instead of <code>KEYS</code>, and <code>UNLINK</code>, which frees memory in the background, instead of <code>DEL</code>.</p>\n<h4 id=\"data-structures-built-for-memory\">Data structures built for memory</h4>\n<p>Redis stores typed values, and it picks a compact encoding when a value is small:</p>\n<ul>\n<li><strong>Strings</strong> are binary-safe and store their length, so <code>STRLEN</code> is O(1).</li>\n<li><strong>Hashes, lists and small sorted sets</strong> are stored as a compact <strong>listpack</strong> (one contiguous block of memory) while small, and switch to a real hash table, list or skip list past a configurable size (128 entries by default for hashes).</li>\n<li><strong>Sets of integers</strong> are stored as a sorted integer array (an intset) up to 512 members by default.</li>\n<li><strong>Sorted sets</strong> use a <strong>skip list</strong> plus a hash table, giving O(log N) inserts, rank lookups and range queries. That makes Redis popular for <a href=\"/learn/design-gaming-leaderboard\">leaderboards</a>.</li>\n<li><strong>HyperLogLog</strong> estimates how many distinct items a set has in at most 12 KB, with a standard error of 0.81%.</li>\n<li><strong>Bitmaps</strong> store one bit per user ID, for things like daily active user flags.</li>\n</ul>\n<h4 id=\"redis-vs-memcached\">Redis vs Memcached</h4>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<div class=\"table-wrap\"><table><thead><tr><th></th><th>Redis</th><th>Memcached</th></tr></thead><tbody><tr><td>Values</td><td>Strings, hashes, lists, sets, sorted sets, streams and more, with server-side operations on them</td><td>Opaque byte strings</td></tr><tr><td>Threads</td><td>One thread runs commands (I/O can use more)</td><td>Multi-threaded, so one process uses all cores</td></tr><tr><td>Persistence</td><td>Optional snapshots (RDB) and append-only log (AOF)</td><td>None, memory only</td></tr><tr><td>Replication and failover</td><td>Built in, plus Sentinel and Cluster</td><td>None. Clients spread keys across servers with <a href=\"/learn/consistent-hashing\">consistent hashing</a></td></tr><tr><td>Eviction</td><td>Configurable policy (LRU, LFU, random, TTL-based, or none)</td><td>LRU within each slab class</td></tr><tr><td>Typical use</td><td>Cache plus counters, queues, rate limits, leaderboards, locks</td><td>A large, simple, fast key-value cache</td></tr></tbody></table></div>\n<p>If all you need is a big cache of serialized objects, Memcached is simple and scales well on machines with many cores. Most teams choose Redis, because its data structures, replication and persistence let one system do several jobs.</p>","headings":[{"id":"redis-internals-and-memcached","title":"Redis internals and Memcached","depth":3},{"id":"a-single-thread-for-commands","title":"A single thread for commands","depth":4},{"id":"data-structures-built-for-memory","title":"Data structures built for memory","depth":4},{"id":"redis-vs-memcached","title":"Redis vs Memcached","depth":4}]},{"id":"redis-sentinel-vs-redis-cluster","title":"Redis Sentinel vs Redis Cluster","html":"<h3 id=\"redis-sentinel-vs-redis-cluster\">Redis Sentinel vs Redis Cluster</h3>\n<p>Redis offers two ways to run more than one node, and they solve different problems. Sentinel keeps one dataset available when a server fails. Cluster splits the dataset across several servers.</p>\n<figure class=\"diagram\" data-text=\"Primary -> Replica (replicates); Sentinels (3 processes) -> Primary (monitor); Sentinels (3 processes) -> Replica (promote); Node A slots 0–5460; Node B slots 5461–10922; Node C slots 10923–16383\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 0 784 237\" width=\"784\" role=\"img\" aria-label=\"Sentinel (failover) and Cluster (sharding)\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"80\" y1=\"74\" x2=\"80\" y2=\"152\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"249\" y1=\"92.7\" x2=\"153\" y2=\"63.9\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"249\" y1=\"135.3\" x2=\"153\" y2=\"164.1\" marker-end=\"url(#dg-arrow)\"></line><path class=\"dg-node dg-store\" d=\"M12,20 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M12,20 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"49.5\">Primary</tspan></text><path class=\"dg-node dg-store\" d=\"M12,164 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M12,164 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"193.5\">Replica</tspan></text><rect class=\"dg-node dg-box\" x=\"252\" y=\"88\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"110.5\">Sentinels</tspan><tspan x=\"320\" y=\"126.5\">(3 processes)</tspan></text><path class=\"dg-node dg-store\" d=\"M476,18 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M476,18 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"544\" y=\"41.5\">Node A</tspan><tspan x=\"544\" y=\"57.5\">slots 0–5460</tspan></text><path class=\"dg-node dg-store\" d=\"M636,82 a68,7 0 0,1 136,0 v64 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M636,82 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"704\" y=\"105.5\">Node B</tspan><tspan x=\"704\" y=\"121.5\">slots</tspan><tspan x=\"704\" y=\"137.5\">5461–10922</tspan></text><path class=\"dg-node dg-store\" d=\"M476,154 a68,7 0 0,1 136,0 v64 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M476,154 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"544\" y=\"177.5\">Node C</tspan><tspan x=\"544\" y=\"193.5\">slots</tspan><tspan x=\"544\" y=\"209.5\">10923–16383</tspan></text><rect class=\"dg-edge-bg\" x=\"44\" y=\"104\" width=\"72\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"80\" y=\"117\" text-anchor=\"middle\">replicates</text><rect class=\"dg-edge-bg\" x=\"174.3\" y=\"69.3\" width=\"53.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"201\" y=\"82.3\" text-anchor=\"middle\">monitor</text><rect class=\"dg-edge-bg\" x=\"174.3\" y=\"140.7\" width=\"53.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"201\" y=\"153.7\" text-anchor=\"middle\">promote</text></svg></div><figcaption>Sentinel (failover) and Cluster (sharding)</figcaption></figure>\n<h4 id=\"redis-sentinel\">Redis Sentinel</h4>\n<ul>\n<li>You run one primary and one or more replicas, each holding the full dataset.</li>\n<li>At least three Sentinel processes, on separate machines, check them with heartbeats.</li>\n<li>When enough Sentinels to reach the configured quorum agree that the primary is down, they elect a leader, which promotes a replica and tells clients where the new primary is.</li>\n<li>Sentinel gives you <strong>high availability</strong>. It doesn't add write capacity: every write still goes to one primary, and the dataset must fit on one machine.</li>\n<li>Replication is asynchronous, so writes the old primary acknowledged just before it failed can be lost in a failover.</li>\n</ul>\n<h4 id=\"redis-cluster\">Redis Cluster</h4>\n<p>Redis Cluster splits the key space across several primaries, each with its own replicas.</p>\n<ul>\n<li>\n<p>The key space is divided into <strong>16,384 hash slots</strong>. Each key belongs to one slot:</p>\n<pre class=\"shiki shiki-themes github-light github-dark\" style=\"background-color:#fff;--shiki-dark-bg:#24292e;color:#24292e;--shiki-dark:#e1e4e8\" tabindex=\"0\"><code><span class=\"line\"><span>slot = CRC16(key) mod 16384</span></span></code></pre>\n</li>\n<li>\n<p>Each primary owns a set of slots. With three primaries, a default setup gives node A slots 0–5,460, node B slots 5,461–10,922 and node C slots 10,923–16,383.</p>\n</li>\n<li>\n<p>To move data between nodes, you move slots. Adding a node means handing it some slots from the others, and keys never need to be rehashed.</p>\n</li>\n<li>\n<p><strong>Hash tags</strong> keep related keys together. If a key contains <code>{...}</code>, only the part inside the braces is hashed. <code>{user:100}:profile</code> and <code>{user:100}:orders</code> both hash <code>user:100</code>, so they land in the same slot. This matters because multi-key commands (<code>MGET</code>, <code>MULTI</code>/<code>EXEC</code> transactions, Lua scripts) only work when all their keys are in one slot. Otherwise Redis returns a <code>CROSSSLOT</code> error.</p>\n</li>\n<li>\n<p><strong>Clients route requests themselves.</strong> A cluster-aware client keeps a copy of the slot-to-node map and sends each command straight to the right node. If a slot has moved, the node replies with a redirect such as <code>-MOVED 3999 10.0.0.2:6379</code>, and the client updates its map and retries. During a slot migration, a node can answer <code>-ASK</code>, which redirects just that one request.</p>\n</li>\n</ul>\n<p>Why 16,384 slots? The Redis author's explanation is about message size. Nodes gossip a bitmap of the slots they own, and 16,384 bits is 2 KB. That keeps heartbeats small while allowing far more slots than the recommended maximum of about 1,000 nodes.</p>","headings":[{"id":"redis-sentinel-vs-redis-cluster","title":"Redis Sentinel vs Redis Cluster","depth":3},{"id":"redis-sentinel","title":"Redis Sentinel","depth":4},{"id":"redis-cluster","title":"Redis Cluster","depth":4}]},{"id":"keeping-the-cache-consistent-with-the-database","title":"Keeping the cache consistent with the database","html":"<h3 id=\"keeping-the-cache-consistent-with-the-database\">Keeping the cache consistent with the database</h3>\n<p>When data changes, two questions come up. Should you update the cached copy or delete it? And should you change the database first or the cache first? The wrong answers leave a stale value in the cache that nothing corrects until its TTL runs out.</p>\n<h4 id=\"why-update-the-database-then-update-the-cache-goes-wrong\">Why \"update the database, then update the cache\" goes wrong</h4>\n<p>Two writers change the same user at nearly the same time:</p>\n<ol>\n<li>Writer A sets the name to \"Ann\" in the database.</li>\n<li>Writer B sets the name to \"Bea\" in the database. The database now says \"Bea\".</li>\n<li>Writer B writes \"Bea\" into the cache.</li>\n<li>Writer A, delayed by a slow network or a garbage collection pause, writes \"Ann\" into the cache.</li>\n</ol>\n<p>The database says \"Bea\" and the cache says \"Ann\". Every reader sees \"Ann\" until the TTL expires. The two writes reached the two systems in different orders, and nothing ties them together. A homemade write-through layer that runs several writers in parallel has the same problem, because it makes the same two separate writes.</p>\n<h4 id=\"cache-aside-with-delete-after-commit\">Cache-aside with delete after commit</h4>\n<p>The usual fix is the write rule from <a href=\"/learn/cache#cache-aside\">cache-aside</a>: update the database, and after the transaction <strong>commits</strong>, delete the key instead of updating it. Deletes have no ordering problem between writers. Two deletes in either order leave the same result, no entry, and the next read loads whatever the database holds at that moment.</p>\n<figure class=\"diagram\" data-text=\"App server -> Database (1. write, commit); App server -> Cache (2. DEL key)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 0 512 203\" width=\"512\" role=\"img\" aria-label=\"Cache-aside: reads fill the cache, writes delete from it\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"151\" y1=\"89.9\" x2=\"359\" y2=\"54.4\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"151\" y1=\"114.1\" x2=\"359\" y2=\"149.6\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-box\" x=\"12\" y=\"78\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"106.5\">App server</tspan></text><path class=\"dg-node dg-store\" d=\"M364,20 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M364,20 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"432\" y=\"49.5\">Database</tspan></text><path class=\"dg-node dg-store\" d=\"M364,140 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M364,140 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"432\" y=\"169.5\">Cache</tspan></text><rect class=\"dg-edge-bg\" x=\"200.4\" y=\"63.2\" width=\"109.2\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"255\" y=\"76.2\" text-anchor=\"middle\">1. write, commit</text><rect class=\"dg-edge-bg\" x=\"219\" y=\"122.8\" width=\"72\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"255\" y=\"135.8\" text-anchor=\"middle\">2. DEL key</text></svg></div><figcaption>Cache-aside: reads fill the cache, writes delete from it</figcaption></figure>\n<h4 id=\"why-not-delete-the-cache-first\">Why not delete the cache first?</h4>\n<p>Deleting before the database write leaves a wide window for a race:</p>\n<ol>\n<li>Writer A deletes <code>user:1</code> from the cache, then starts a database transaction to change \"Alice\" to \"Bob\".</li>\n<li>Before A commits, reader B gets a miss on <code>user:1</code>.</li>\n<li>B reads the database and gets the old value, \"Alice\", because A hasn't committed.</li>\n<li>B writes \"Alice\" into the cache.</li>\n<li>A commits \"Bob\".</li>\n</ol>\n<p>The database says \"Bob\", the cache says \"Alice\", and nothing fixes it before the TTL. The window lasts as long as A's whole transaction, so this happens regularly under load.</p>\n<p>Delete-after-commit still has a race, but a much narrower one. A reader misses, reads the old value from the database, and stalls. Meanwhile a writer commits and deletes the key. Then the reader wakes up and writes the old value into the cache. The reader had to be slower than the entire write, so this is rare, but it does happen at scale. Common defenses:</p>\n<ul>\n<li><strong>Always set a TTL</strong>, so a stale value can only last so long.</li>\n<li><strong>Delayed double delete:</strong> delete the key again a second or so after the write, to catch a late stale fill.</li>\n<li><strong>Leases or versioned writes:</strong> let the cache reject a fill that started before the most recent delete. Facebook's memcache leases do this.</li>\n</ul>\n<h4 id=\"invalidation-with-change-data-capture\">Invalidation with change data capture</h4>\n<p>Delete-after-commit has one more gap. If the app server crashes or loses its connection to Redis between the commit and the <code>DEL</code>, the key is never deleted, and a crash loses any retry too.</p>\n<p><strong>Change data capture (CDC)</strong> moves invalidation out of the application. The database already records every committed change in its log (the write-ahead log, or WAL, in PostgreSQL; the binlog in MySQL). A CDC tool reads that log and turns each change into an event.</p>\n<figure class=\"diagram\" data-text=\"App server -> Database WAL / binlog (SQL writes only); Database WAL / binlog -> Debezium (CDC) (reads the log); Debezium (CDC) -> Kafka topic (change event); Kafka topic -> Invalidation worker; Invalidation worker -> Redis (DEL user:42)\"><div class=\"diagram-scroll\"><svg class=\"dg\" viewBox=\"0 -15 640 242\" width=\"640\" role=\"img\" aria-label=\"Cache invalidation driven by the database log\"><defs><marker id=\"dg-arrow\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\"><path class=\"dg-arrowhead\" d=\"M0,1 L9,5 L0,9 z\"></path></marker></defs><line class=\"dg-edge\" x1=\"151\" y1=\"42\" x2=\"247\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"391\" y1=\"42\" x2=\"487\" y2=\"42\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"560\" y1=\"71\" x2=\"560\" y2=\"157\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge dg-dashed\" x1=\"489\" y1=\"186\" x2=\"393\" y2=\"186\" marker-end=\"url(#dg-arrow)\"></line><line class=\"dg-edge\" x1=\"249\" y1=\"186\" x2=\"153\" y2=\"186\" marker-end=\"url(#dg-arrow)\"></line><rect class=\"dg-node dg-box\" x=\"12\" y=\"18\" width=\"136\" height=\"48\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"46.5\">App server</tspan></text><path class=\"dg-node dg-store\" d=\"M252,18 a68,7 0 0,1 136,0 v48 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M252,18 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"41.5\">Database</tspan><tspan x=\"320\" y=\"57.5\">WAL / binlog</tspan></text><rect class=\"dg-node dg-box\" x=\"492\" y=\"16\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"560\" y=\"38.5\">Debezium</tspan><tspan x=\"560\" y=\"54.5\">(CDC)</tspan></text><rect class=\"dg-node dg-queue\" x=\"492\" y=\"162\" width=\"136\" height=\"48\" rx=\"6\"></rect><line class=\"dg-line\" x1=\"612\" y1=\"170\" x2=\"612\" y2=\"202\"></line><line class=\"dg-line\" x1=\"602\" y1=\"170\" x2=\"602\" y2=\"202\"></line><line class=\"dg-line\" x1=\"592\" y1=\"170\" x2=\"592\" y2=\"202\"></line><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"546\" y=\"190.5\">Kafka topic</tspan></text><rect class=\"dg-node dg-box\" x=\"252\" y=\"160\" width=\"136\" height=\"52\" rx=\"8\"></rect><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"320\" y=\"182.5\">Invalidation</tspan><tspan x=\"320\" y=\"198.5\">worker</tspan></text><path class=\"dg-node dg-store\" d=\"M12,164 a68,7 0 0,1 136,0 v44 a68,7 0 0,1 -136,0 z\"></path><path class=\"dg-line\" d=\"M12,164 a68,7 0 0,0 136,0\"></path><text class=\"dg-label\" text-anchor=\"middle\"><tspan x=\"80\" y=\"193.5\">Redis</tspan></text><rect class=\"dg-edge-bg\" x=\"147.5\" y=\"-11\" width=\"103\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"199\" y=\"2\" text-anchor=\"middle\">SQL writes only</text><rect class=\"dg-edge-bg\" x=\"393.7\" y=\"-11\" width=\"90.6\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"439\" y=\"2\" text-anchor=\"middle\">reads the log</text><rect class=\"dg-edge-bg\" x=\"517.8\" y=\"105\" width=\"84.4\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"560\" y=\"118\" text-anchor=\"middle\">change event</text><rect class=\"dg-edge-bg\" x=\"161.9\" y=\"177\" width=\"78.2\" height=\"18\" rx=\"4\"></rect><text class=\"dg-edge-label\" x=\"201\" y=\"190\" text-anchor=\"middle\">DEL user:42</text></svg></div><figcaption>Cache invalidation driven by the database log</figcaption></figure>\n<p>How it works:</p>\n<ol>\n<li>The application runs ordinary SQL transactions, with no cache invalidation code.</li>\n<li><strong>Debezium</strong>, usually running as a Kafka Connect connector, reads the log through the database's replication interface (logical decoding in PostgreSQL, the binlog in MySQL). It emits events only for committed transactions, so a rolled-back write never causes an invalidation.</li>\n<li>Each change becomes an event on a <strong>Kafka</strong> topic, one topic per table, named like <code>&#x3C;prefix>.&#x3C;schema>.&#x3C;table></code>. The event carries the row's key and its values before and after the change.</li>\n<li>An invalidation worker reads the topic and runs <code>DEL user:42</code> in Redis.</li>\n</ol>\n<p>What you get:</p>\n<ul>\n<li><strong>No coupling in the write path.</strong> A Redis outage can't fail or slow down a database transaction.</li>\n<li><strong>No lost invalidations.</strong> The worker commits its Kafka offset only after the <code>DEL</code> succeeds. After a crash it replays from the last committed offset. Deleting a key twice is harmless, so the duplicates don't matter.</li>\n<li><strong>Every writer is covered,</strong> including batch jobs, admin scripts and other services that know nothing about the cache.</li>\n</ul>\n<p>What it costs: Kafka, Kafka Connect and Debezium all need running and monitoring. Invalidation also lags the commit by the pipeline's delay, usually milliseconds to a second or two, and readers can see the old value in that time. So CDC gives eventual consistency. It doesn't close the stale-fill race above either, so keep TTLs. <a href=\"/learn/event-streaming\">Event streaming</a> covers Kafka in more depth.</p>","headings":[{"id":"keeping-the-cache-consistent-with-the-database","title":"Keeping the cache consistent with the database","depth":3},{"id":"why-update-the-database-then-update-the-cache-goes-wrong","title":"Why \"update the database, then update the cache\" goes wrong","depth":4},{"id":"cache-aside-with-delete-after-commit","title":"Cache-aside with delete after commit","depth":4},{"id":"why-not-delete-the-cache-first","title":"Why not delete the cache first?","depth":4},{"id":"invalidation-with-change-data-capture","title":"Invalidation with change data capture","depth":4}]},{"id":"check-yourself","title":"Check yourself","html":"<h3 id=\"check-yourself\">Check yourself</h3>\n<details>\n<summary>Crumb caches the results of \"top Thai recipes\" by hashing the SQL. Then one recipe's rating changes. What goes wrong, and how does caching objects avoid it?</summary>\n<p>The changed rating might affect \"top Thai recipes\", \"top-rated this week\" and many other cached queries. Crumb has no cheap way to find them all, so it either deletes far too much or leaves stale results behind. With object caching, the recipe lives under one key, <code>recipe:812</code>, and Crumb deletes that one key when the rating changes.</p>\n</details>\n<details>\n<summary>A cache node fails and is replaced by an empty one. How does a cache-aside system recover, and how does a write-through system recover?</summary>\n<p>With cache-aside, requests to the new node miss, load from the database and fill the cache, so the node warms up by itself while latency is higher. With pure write-through, entries only enter the cache when written, so a recipe nobody edits never reaches the new node. The usual fix is to combine them: writes go through the cache, and reads fill in anything missing.</p>\n</details>\n<details>\n<summary>Crumb wants to count recipe views. Why might write-behind suit this, and what does Crumb risk by using it?</summary>\n<p>Popular recipes get thousands of views a second. Write-behind lets each increment return as soon as the cache has it, and it combines many updates to the same counter into one database write. The risk is data loss: if the cache fails before its queued writes reach the database, those increments are gone. For a view counter, losing a few seconds of counts is usually acceptable. For a payment, it isn't.</p>\n</details>\n<details>\n<summary>How does refresh-ahead lower latency compared with read-through, and when can it make things worse?</summary>\n<p>With read-through, the first read after an entry expires waits for the database. Refresh-ahead reloads a recently read entry in the background before it expires, so readers of popular entries never wait. It only helps if the cache predicts correctly which entries will be read again. If it reloads entries nobody reads, it sends extra queries to the database, and performance can end up worse than with no refresh-ahead.</p>\n</details>\n<details>\n<summary>How does XFetch avoid a stampede without any locks, and why is the early-refresh window scaled by how long the value takes to compute?</summary>\n<p>Each reader draws a random head start that averages <code>delta × beta</code>, where <code>delta</code> is how long the last rebuild took. If the head start reaches past the expiry time, that reader refreshes early. Far from expiry the chance is close to zero, and in the last few multiples of <code>delta</code> it rises quickly. A hot key has many readers drawing every second, so one of them very likely refreshes before expiry and the rest never miss.</p>\n<p>Scaling by <code>delta</code> matters because the refresh has to finish before the key expires. A value that takes 2 seconds to build must start refreshing seconds ahead. One that takes 20 ms needs only a few tens of milliseconds.</p>\n</details>\n<details>\n<summary>A batch job warms 5 million keys at 02:00 with a TTL of exactly one hour. What happens at 03:00, and what one-line change fixes it?</summary>\n<p>At 03:00 all 5 million keys expire within about a second, and the database gets the whole read load the cache was absorbing: a cache avalanche. The fix is jitter. Set each TTL to the base value plus a random offset, such as 3,600 s ± 15%, so the expirations spread over about 18 minutes.</p>\n</details>\n<details>\n<summary>Why is \"delete the cache, then update the database\" worse than \"update the database, then delete the cache\"?</summary>\n<p>If you delete first, any reader that arrives while the write transaction is open misses, reads the old committed value, and puts it back in the cache. After the commit, the cache holds the old value until the TTL. The window is the whole transaction, so this happens often.</p>\n<p>If you delete after the commit, the only race needs a reader that read the old value before the commit and wrote it to the cache after the delete. That reader was slower than the entire write, which is much rarer. TTLs, a delayed second delete, or leases handle the rest.</p>\n</details>\n<details>\n<summary>How does Redis Cluster decide which node holds a key, and how do you make sure `user:42`'s profile and orders can be read in one `MGET`?</summary>\n<p>Redis Cluster computes <code>CRC16(key) mod 16384</code> to get one of 16,384 hash slots, and each primary owns a set of slots. <code>user:42:profile</code> and <code>user:42:orders</code> probably hash to slots on different nodes, so an <code>MGET</code> across them fails with a <code>CROSSSLOT</code> error. Name them <code>{user:42}:profile</code> and <code>{user:42}:orders</code> instead. Only the text inside the braces is hashed, so both land in the same slot.</p>\n</details>\n<details>\n<summary>Your service caches users, but a nightly batch job and a second service also write to the users table. Why is CDC a better fit here than delete-after-commit in your code?</summary>\n<p>Delete-after-commit only covers writes that go through your code. The batch job and the other service never trigger a delete, so the cache serves stale users until their TTLs expire. CDC reads the database's own log, so every committed change produces an invalidation, whoever made it. It also survives crashes, because the worker resumes from its last Kafka offset and repeating a <code>DEL</code> is harmless. The cost is running the pipeline and a short lag before each invalidation.</p>\n</details>\n<h3 id=\"further-reading\">Further reading</h3>\n<ul>\n<li><a href=\"http://horicky.blogspot.com/2010/10/scalable-system-design-patterns.html\" target=\"_blank\" rel=\"noopener noreferrer\">Scalable System Design Patterns</a> (Ricky Ho, 2010): a short catalog of eight patterns, including the result cache, each with a diagram.</li>\n<li><a href=\"https://lethain.com/introduction-to-architecting-systems-for-scale/\" target=\"_blank\" rel=\"noopener noreferrer\">Introduction to Architecting Systems for Scale</a> (Will Larson, 2011): a readable overview of application caching, database caching, in-memory caches, CDNs and cache invalidation.</li>\n<li><a href=\"https://web.archive.org/web/20230126233752/https://www.lecloud.net/post/9246290032/scalability-for-dummies-part-3-cache\" target=\"_blank\" rel=\"noopener noreferrer\">Scalability for Dummies, part 3: Cache</a> (Sebastian Kreutzberger, 2011, archived): the short case for caching objects instead of query results.</li>\n<li><a href=\"https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/Strategies.html\" target=\"_blank\" rel=\"noopener noreferrer\">Caching strategies for Memcached</a> (AWS ElastiCache guide): lazy loading, write-through and TTLs, with the advantages and disadvantages of each and pseudocode.</li>\n<li><a href=\"https://docs.oracle.com/en/middleware/standalone/coherence/14.1.1.2206/develop-applications/caching-data-sources.html\" target=\"_blank\" rel=\"noopener noreferrer\">Caching Data Sources</a> (Oracle Coherence documentation): precise definitions of read-through, write-through, write-behind and refresh-ahead, including write coalescing and the refresh-ahead factor.</li>\n<li><a href=\"https://www.slideshare.net/tmatyashovsky/from-cache-to-in-memory-data-grid-introduction-to-hazelcast\" target=\"_blank\" rel=\"noopener noreferrer\">From Cache to In-Memory Data Grid</a> (Taras Matyashovsky): slides covering eviction policies, the cache access patterns, and local, replicated, distributed and near caches, with Hazelcast as the example.</li>\n<li><a href=\"https://www.slideshare.net/jboner/scalability-availability-stability-patterns/\" target=\"_blank\" rel=\"noopener noreferrer\">Scalability, Availability &#x26; Stability Patterns</a> (Jonas Bonér, 2010): a long slide deck whose caching section covers HTTP caching, write-through and write-behind, and eviction policies. Best used as a checklist.</li>\n<li><a href=\"https://redis.io/docs/latest/develop/reference/eviction/\" target=\"_blank\" rel=\"noopener noreferrer\">Key eviction</a> (Redis documentation): every <code>maxmemory-policy</code>, how to pick one, and how Redis approximates LRU and LFU.</li>\n<li><a href=\"https://en.wikipedia.org/wiki/Cache_(computing)\" target=\"_blank\" rel=\"noopener noreferrer\">Cache (computing)</a> (Wikipedia): background on caches in hardware and software, and on write policies.</li>\n<li><a href=\"https://www.vldb.org/pvldb/vol8/p886-vattani.pdf\" target=\"_blank\" rel=\"noopener noreferrer\">Optimal Probabilistic Cache Stampede Prevention</a> (Vattani, Chierichetti and Lowenstein, VLDB 2015): the XFetch algorithm and why the exponential rule is the right one.</li>\n<li><a href=\"https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/nishtala\" target=\"_blank\" rel=\"noopener noreferrer\">Scaling Memcache at Facebook</a> (Nishtala et al., NSDI 2013): leases against stampedes and stale sets, and invalidation across regions.</li>\n<li><a href=\"https://redis.io/docs/latest/operate/oss_and_stack/reference/cluster-spec/\" target=\"_blank\" rel=\"noopener noreferrer\">Redis cluster specification</a>: hash slots, hash tags, gossip, and <code>MOVED</code> and <code>ASK</code> redirects.</li>\n<li><a href=\"https://debezium.io/documentation/reference/stable/architecture.html\" target=\"_blank\" rel=\"noopener noreferrer\">Debezium architecture</a>: how Debezium reads database logs and publishes change events through Kafka Connect.</li>\n<li><a href=\"https://dl.acm.org/doi/10.1145/362686.362692\" target=\"_blank\" rel=\"noopener noreferrer\">Space/Time Trade-offs in Hash Coding with Allowable Errors</a> (Burton H. Bloom, 1970): the paper that introduced Bloom filters.</li>\n</ul>","headings":[{"id":"check-yourself","title":"Check yourself","depth":3},{"id":"further-reading","title":"Further reading","depth":3}]}]}