Caching

Every cache exploits one asymmetry: memory access is orders of magnitude cheaper than network or disk access, and access patterns are skewed — a small working set absorbs most reads. On an OTT platform the skew is extreme: one live match manifest or trending series can be >10% of total read traffic.

The decision is a one-line inequality:

First, feel the difference between a hit and a miss. Step a request through the production topology:

Now the four write/read strategies. The tabs matter because strategy selection is per-dataset, not per-system — a real platform runs all four simultaneously.

LoadingCache<String, ContentMeta> cache = Caffeine.newBuilder()
    .maximumSize(50_000)                       // W-TinyLFU eviction — beats LRU and LFU
    .expireAfterWrite(Duration.ofMinutes(10))
    .refreshAfterWrite(Duration.ofMinutes(5))  // async refresh; also coalesces loads per key
    .build(repository::loadById);

Fast trackRead in full. The publish-before-commit race below is the highest-value five minutes in this module — it's subtle, common in event-driven estates like yours, and invisible in code review unless you know to look.

Ordered by sophistication. Rule zero: always set a TTL even when using explicit invalidation — it bounds the blast radius of every invalidation bug you will ever ship.

• TTL only. TTL = maximum acceptable staleness. Sufficient for reference data.
• Evict-on-write (@CacheEvict). Only reliable when writer and cacher are the same service. In a 200-microservice estate, they usually aren't.
• Event-driven invalidation. Owner publishes ContentUpdated to RabbitMQ; every caching consumer evicts/refreshes. The correct pattern for your estate — with one deadly ordering rule, below.
• Versioned keys. Write content:123:v42, swap a version pointer. Eviction becomes atomic; old versions age out via TTL. Eliminates delete-vs-concurrent-read races entirely.
• Pub/Sub L1 broadcast. Redis PUBLISH invalidation content:123; every pod evicts its Caffeine entry. The only way to invalidate in-JVM caches across hundreds of pods.

The publish-before-commit race

The single most common event-driven invalidation bug. Watch both orderings:

Fix: @TransactionalEventListener(phase = AFTER_COMMIT) — or, when you need a delivery guarantee that survives a crash between commit and publish, the transactional outbox (event row written in the same DB transaction, relayed to MQ by a poller/CDC). Outbox gets its own module in Week 3.

Eviction policy (when memory is full)

Distinct from invalidation. For a catalog with stable popularity, Redis maxmemory-policy allkeys-lfu usually beats LRU — one viral title shouldn't flush your steady-state working set. Caffeine's W-TinyLFU already approximates the best of both, which is a real reason to prefer it over hand-rolled maps.

Cache stampede (thundering herd)

A hot key expires; every concurrent request misses simultaneously and lands on the DB. Run it:

But the stampede alone isn't what kills you. Watch what it triggers:

The cache miss is the spark; the retry feedback loop is the fire. Any complete answer mitigates both the stampede and the amplification.

Five mitigation families, cheapest first:

• Jittered TTL — ttl = base + rand(0, 20%). Prevents synchronized expiry waves after deploys/warm-ups (the avalanche case). Costs one line. Limit: does nothing for a single extremely hot key — that one key still expires at some instant.
• Single-flight — one loader per key, others wait. Per-JVM: Caffeine does it automatically. Cross-pod: Redis lock SET k v NX PX 3000 with a double-check of the cache after acquiring the lock (code in §06). Trade-off: losers block for the load duration, so you need lock TTLs against crashed loaders and bounded waits.
• Stale-while-revalidate (soft/hard TTL) — store a logical expiry inside the value, keep the physical Redis TTL longer. Past soft expiry: serve the stale value instantly, let exactly one request refresh in the background. Losers never block — often the best production answer for catalog-class data. Same idea at the CDN via Cache-Control: stale-while-revalidate, stale-if-error plus request collapsing — for OTT manifests behind a CDN, this outermost layer matters most of all.
• Proactive / probabilistic early refresh — refresh known hot keys before expiry. XFetch: refresh when now − delta·β·ln(rand()) ≥ expiry (delta = last recompute time, β≈1) — refresh probability rises smoothly as expiry nears, so the hot key never actually expires. Trade-off: wasted work on keys nobody reads anymore; reserve for known-hot data.
• Protect the database itself — the cache is not the last line of defense. Bounded connection pools, short DB timeouts, exponential backoff with retry budgets, circuit breakers, and load shedding stop the feedback loop when everything above fails. Rejecting some requests beats losing all of them. (Circuit breakers, retries, and rate limiting each get a full Week 3 module.)

The composite read path

These compose into one algorithm — this is the design-review answer:

The rest of the incident catalog

A · Two-tier config — Caffeine L1 + Redis L2

@Configuration @EnableCaching
public class CacheConfig {

  @Bean // L1 — per-pod, absorbs hot keys
  CaffeineCacheManager l1() {
    var m = new CaffeineCacheManager("contentMeta");
    m.setCaffeine(Caffeine.newBuilder()
        .maximumSize(20_000).expireAfterWrite(Duration.ofMinutes(5)));
    return m;
  }

  @Bean // L2 — cluster-wide, JSON values, 30m TTL
  RedisCacheManager l2(RedisConnectionFactory cf) {
    return RedisCacheManager.builder(cf)
      .withCacheConfiguration("contentMeta",
        RedisCacheConfiguration.defaultCacheConfig()
          .entryTtl(Duration.ofMinutes(30))
          .serializeValuesWith(SerializationPair.fromSerializer(
              new GenericJackson2JsonRedisSerializer())))
      .build();
  }
}

B · Event-driven invalidation — note AFTER_COMMIT

// writer: content-service
@Transactional
public void updateContent(String id, ContentUpdate req) {
  repository.save(apply(req));
  events.publishEvent(new ContentUpdated(id)); // in-process, tx-bound
}

@TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT) // ← the fix from §04
public void relay(ContentUpdated e) {
  rabbit.convertAndSend("content.events", "content.updated", e);
}

// consumer: any caching service
@RabbitListener(queues = "playback.content-invalidation")
public void on(ContentUpdated e) {
  redis.delete("content:meta:" + e.id());          // L2
  l1.invalidate(e.id());                             // this pod's L1
  redis.convertAndSend("l1-invalidation", e.id());  // every other pod's L1
}

C · Cross-pod single-flight — stampede protection

public ContentMeta get(String id) {
  String key = "content:meta:" + id;
  String hit = redis.opsForValue().get(key);
  if (hit != null) return deser(hit);

  Boolean lock = redis.opsForValue()               // SET NX PX — one loader wins
      .setIfAbsent(key + ":lock", "1", Duration.ofSeconds(3));
  if (Boolean.TRUE.equals(lock)) {
    try {
      String recheck = redis.opsForValue().get(key);  // DOUBLE-CHECK: another pod may have
      if (recheck != null) return deser(recheck);    // loaded between our miss and this lock
      ContentMeta m = repo.findById(id).orElseThrow();
      redis.opsForValue().set(key, ser(m), jittered(30)); // base + rand(0,20%)
      return m;
    } finally { redis.delete(key + ":lock"); }      // PX expiry guards abandoned locks on crash
  }
  return retryCacheRead(key, 3, Duration.ofMillis(100))  // losers: bounded wait,
      .orElseGet(() -> repo.findById(id).orElseThrow());   // then DB as last resort
}

Fast trackCondensed exercise (~45 min): do steps 5–6 only — the stampede counter and the break-it-on-purpose race — on any existing cached service or a throwaway app. These two produce the evidence and the scar tissue; steps 1–4 you've effectively built before.

One Spring Boot service, ~2 hours. The point is producing your own latency table and DB-hit counter — measured numbers, not remembered ones.

Acceptance: your measured latency table (none / L2 / L1+L2), the stampede counter evidence, and a one-paragraph mapping of your platform's real datasets onto the §02 staleness table.

Stretch: maxmemory 50mb + allkeys-lfu, load 100k keys under a Zipfian access pattern, and observe which keys survive.

Self-check