Lorixen.
EN

Cache stampedes: when every request rebuilds the same page

Short answer: A cache stampede happens when a popular cached item expires and many requests notice at the same moment, so they all hit the database or rebuild the page at once, slowing everything down or knocking the site over. Prevent it by letting only one request rebuild an item while others wait or get the old copy, serving slightly stale content during refresh with stale-while-revalidate or your proxy's equivalent, adding random jitter to expiry times, and refreshing the most popular items before they expire.

What a stampede looks like

Caching is how many business sites stay fast: the home page, product listings, price tables or an API response are computed once and reused. Everything is fine until the cached copy expires. Then:

  1. The cache entry for the busy page expires.
  2. The next request finds nothing, so it starts rebuilding the page from the database.
  3. Before it finishes, dozens or hundreds more requests arrive, find nothing, and start rebuilding too.
  4. The database is suddenly handling far more work than usual, everything slows, rebuilds take longer, and more requests pile in behind them.

It's also called the thundering herd or dog-piling. The symptoms are distinctive: regular spikes in response time or database load at predictable intervals, often matching your cache lifetime, or a brief outage right after a deploy that clears the cache.

Fix 1: let one request rebuild

The most direct fix is to make sure only one request regenerates a given item.

In the application cache. When a request misses the cache, it tries to take a short-lived lock for that key, for example with an atomic "set if not exists" in Redis with an expiry. The request that gets the lock rebuilds and stores the item. The others either wait briefly and read the result, or receive the previous value if you kept it.

Give the lock its own short expiry, so a crashed worker doesn't block the key forever.

In a reverse proxy. Nginx's proxy module includes proxy_cache_lock; when enabled, only one request at a time is allowed to populate a new cache element, and others wait for it, up to proxy_cache_lock_timeout. Many CDNs have a similar feature, often called request collapsing or coalescing. Check whether yours is on.

Fix 2: serve stale while refreshing

Often the best user experience is to keep serving the slightly old copy while one request fetches a new one in the background.

RFC 5861 defines two Cache-Control extensions for this. stale-while-revalidate lets a cache serve a stale response for a set period while it revalidates in the background. stale-if-error lets it return a stale response when the origin returns an error, such as a 500, which protects you if the rebuild itself fails.

Cache-Control: max-age=300, stale-while-revalidate=60, stale-if-error=86400

Support varies between CDNs and caches, so check yours. In Nginx, the proxy_cache_use_stale directive with the updating parameter permits using a stale cached response while it's being updated, which the documentation notes minimises the number of accesses to proxied servers when updating cached data.

In your own application cache, you can do the same by storing the value with a "soft" expiry: after it passes, the first request triggers a background refresh but still returns the old value.

Fix 3: add jitter to expiry times

If many keys are cached at the same moment with the same lifetime, they all expire together. That's common after a deploy or cache flush. Add a small random amount to each expiry, for example 300 seconds plus or minus up to 30, so expirations spread out.

Fix 4: refresh hot items before they expire

For a handful of very popular items, such as the home page, a category listing or exchange rates, don't wait for expiry at all. A scheduled job refreshes them shortly before they would expire, so users always hit a warm cache. Some caching libraries support probabilistic early refresh, where a request close to expiry occasionally triggers a refresh ahead of time.

Watch for negative-result stampedes

Stampedes aren't only about popular pages. If a missing item, such as a product ID that doesn't exist, isn't cached at all, every request for it goes to the database. Bots probing random URLs can create the same pile-up. Caching "not found" results briefly, for a minute or so, closes that gap without hiding real new content for long.

Be careful after deploys

A deploy that clears the whole cache creates the worst possible stampede: every page is cold at once, at the moment traffic arrives. Options:

  • Avoid clearing caches on deploy unless the cached data's format changed.
  • Version cache keys, so new code uses new keys while old entries expire naturally.
  • Warm the most important pages straight after deploy, before switching traffic.

Measure first

Before adding locks everywhere, confirm it's a stampede:

  • Plot cache hit rate alongside database load and response times.
  • Look for spikes that line up with cache lifetimes or deploys.
  • Check how long a rebuild of the busiest items takes; long rebuilds make stampedes worse.

Sometimes the cheapest fix is simply making the expensive query faster, so a rebuild takes milliseconds instead of seconds.

A worked example

Picture a hypothetical Belfast ticketing site whose event listing page was cached for five minutes. Every five minutes during busy on-sale periods, response times spiked and the database CPU hit its ceiling. Logs showed hundreds of identical listing queries in the second after each expiry. The developer enabled proxy_cache_lock and proxy_cache_use_stale updating in Nginx, added stale-while-revalidate to the response, put jitter on application cache keys and added a job that refreshed the main listing every four minutes. The spikes disappeared.

FAQ

Isn't a longer cache lifetime enough?

It makes stampedes less frequent, not smaller. When the item does expire, the same rush happens.

Will serving stale content confuse customers?

For a short window, usually not. Avoid it for data that must be exact, such as stock levels at checkout or account balances.

Does this apply to small sites?

Mostly to pages with steady traffic and expensive rebuilds. A quiet site rarely sees enough simultaneous requests.

If your site slows down at regular intervals or after every deploy, ask us to look at your caching.

Sources

WhatsAppBook a call