System Design Masterclass · Module 33 · Extension of Weeks 1 & 4

Mechanical Sympathy: Concurrency, Queueing Laws & Integrity

The single-node truths under the distributed course: the atomic primitive (CAS) beneath every lock-free structure and half this course's patterns, the thread-model decision your Java 21 platform is facing right now (pools vs Loom vs event loops), the two laws — Little's and the Universal Scalability Law — that were silently underneath Modules 13 and 25, and the data-integrity discipline (checksums, scrubbing, end-to-end) that catches the corruption replication happily copies.

~3h study~1.5h exercise1 interactive simFast track ~75 min

Fast track — sized a thread pool with math before?

Compresses to ~75 min. The hierarchy collapses to its rule; CAS, the Loom decision, the laws calculator, and integrity stay mandatory.

  1. §01 takeaway · 5 min
  2. §02 CAS/ABA, in full · 15 min · MANDATORY
  3. §03 thread models, in full · 20 min · MANDATORY
  4. §04 calculator, both laws · 20 min · MANDATORY
  5. §05 integrity, in full · 15 min · MANDATORY
STEP 01

The coordination hierarchy

SKIM
Fast-track takeawayShared mutable state is where concurrency bugs live, and every mechanism for taming it has a price. Fastest to slowest: immutable (nothing to coordinate) → thread-local/partitioned (nothing shared) → lock-free/CAS (~25 ns, progress guaranteed) → lock-based (25 ns uncontended, unbounded when contended) → distributed coordination (network RTT + M19's whole module). The rule: take the highest level that's correct — and notice it's the same ladder this course climbed at system scale: immutable events (M15/M18), partition-per-owner (M05), optimistic version guards (M16), locks only when demotion fails (M19's demote-the-lock tree). One design instinct, two scales — that's the "mechanical sympathy" the module title means.
STEP 02

CAS: the primitive under everything

MANDATORY
STEP 03

Thread models — and the Loom decision

MANDATORY
ModelMechanicsFits
Thread-per-connectionone OS thread per client; ~1 MB stack each → 10k conns ≈ 10 GB of stacks<~1k concurrent; simple internal tools
Pool + queue (classic Tomcat/Spring MVC)fixed OS threads, work queued; a blocked thread is a lost slotbounded concurrency; the model your M13 bulkheads assume
Event loop (Netty/WebFlux/Node)few threads, non-blocking IO, callbacks/reactive; nothing may block the loopC10K+: gateways, proxies, push/SSE fan-out (M07)
Virtual threads (Loom, Java 21)thread-per-request syntax, event-loop economics: JVM parks virtual threads on block, ~KB footprint, millions cheapIO-bound services — i.e., most of your 200
STEP 04

Little's Law & the Universal Scalability Law

MANDATORY

Little's Law: L = λ × W — in-flight work = arrival rate × time-in-system. No assumptions, no distribution fine print; it's conservation. It was silently under M13 (pool size must cover λ×W or the queue grows), M12 (a timeout stretch multiplies W, thus L — why slow is worse than down), M02 (throughput ceilings), and M25 (pods = λ×W ÷ per-pod concurrency). USL adds what happens when you scale out: C(N) = N / (1 + α(N−1) + βN(N−1)) — α is contention (the serialized fraction: locks, single writers, one hot partition), β is coherence (crosstalk: cache invalidation, consensus rounds, gossip — everyone syncing with everyone). α caps your speedup (Amdahl); β makes throughput decrease past a peak N — retrograde scaling, where adding nodes makes it slower.

The laws, live
λ arrival (rps)
1000
W latency (ms)
200
α contention
5%
β coherence
0.5%
148121620242832
STEP 05

Data integrity: corruption is a workload

MANDATORY
STEP 06

Exercise

MANDATORY

Fast trackStep 2 (~30 min): the Loom pinning audit is the one with immediate production value for your estate.

1
Feel the hierarchy. JMH benchmark a hot counter four ways: synchronized, AtomicLong, LongAdder, thread-local-with-sum — at 1, 8, and 32 threads. Plot ops/sec vs threads; you'll watch §01's ladder and a live USL curve (the synchronized version goes retrograde) in one chart.
2
Loom pilot + pinning audit. Take one IO-bound service: enable virtual threads (spring.threads.virtual.enabled=true on Boot 3.2+), load-test before/after at high concurrency, and run with -Djdk.tracePinnedThreads=full to catch synchronized-around-IO. Write the two-paragraph migration recommendation for your architecture group: which service classes, what to audit, what stays reactive.
3
Apply the laws to real numbers. Pull one hot service's peak λ and p99 W from M24 dashboards; compute L and compare to its actual pool/semaphore limits — headroom or hidden collapse? Then fit USL: throughput at 2, 4, 8 pods from a load test → estimate α and β → predicted useful ceiling. One page, real numbers.
4
Integrity walk. Trace one content segment source→CDN→player and one event producer→Kafka→consumer→store: where do checksums exist, where are they *verified*, and what's the first end-to-end check in each path? Any long-retention store without a scrub answer gets a ticket.

Self-check

Connect ABA, the M19 stale-holder, and the fix they share.
Both are validate-then-act races where the validated state was recycled in between: ABA — the memory word returned to value A via B, so CAS's equality check passes against a different world; stale-holder — the lock was released and re-acquired by another process, so "I hold the lock" is true-looking but stale. Shared fix: make state non-recyclable by pairing it with a monotonic stamp (AtomicStampedReference / version column / fencing token) so equality of value can't masquerade as continuity of state. It's one bug and one fix, at nanosecond and network scale.
Why doesn't Loom make bulkheads obsolete — argue from Little's Law.
Pre-Loom, the thread pool was an accidental admission controller: L couldn't exceed pool size, so excess arrivals queued visibly. Loom makes threads ~free, so L = λ×W is now unbounded by default — during a downstream brownout W stretches and in-flight work balloons (memory, connections, downstream pressure) with no natural ceiling. Bulkheads/semaphores (M13) must become *explicit*: you're no longer limited by thread cost, so you must limit by policy. Loom removed the accident, not the need.
A service scales linearly to 8 pods, plateaus at 12, and gets *slower* at 20. Diagnose with USL and name two likely culprits per parameter.
Plateau = α (contention: a serialized fraction — single hot partition M05, one shared DB connection pool, a global lock/single-writer table); decline past a peak = β > 0 (coherence: work that grows with N² — cache-invalidation chatter between pods, full-mesh health gossip, consensus or leader coordination on the request path M30). Fit the two runs to C(N) to quantify, then attack α by partitioning the serialized thing and β by removing pairwise crosstalk (hierarchies, batching, taking consensus off the hot path). Adding pods past the peak is paying to be slower.
TCP has checksums and you use TLS everywhere — make the case that a transcoded video segment still needs an application-level hash.
End-to-end argument: TCP's checksum covers the wire hop (and is weak — 16-bit), TLS covers its session — but corruption in the transcoder's memory, a buggy serializer, a bad disk write at the origin, or a caching layer's truncation happens *between* protected hops, and every lower layer will then faithfully protect the corrupt bytes onward. Only a hash computed at packaging and verified at (or near) the player spans the actual journey. Lower layers optimize by catching most errors early; they cannot provide the guarantee.