System Design Masterclass · Module 17 / Week 3

Sagas & Distributed Transactions

ACID stops at the service boundary. Why 2PC isn't the answer, sagas as sequences of local transactions with compensations, choreography vs orchestration for real team topologies, and the isolation anomalies sagas re-introduce.

~3h study~2h exercise1 interactive simFast track ~80 min

Fast track — shipped a saga before?

Compresses to ~80 min. 2PC history collapses; saga mechanics, the topology decision, and the isolation anomalies stay mandatory — the anomalies are what production sagas actually get wrong.

  1. §01 takeaway · 5 min
  2. §02 mechanics + sim, both runs · 20 min · MANDATORY
  3. §03 topology decision, in full · 15 min · MANDATORY
  4. §04 anomalies, in full · 20 min · MANDATORY
  5. §05 design rules · 10 min
STEP 01

Why not 2PC

SKIM
Fast-track takeawayTwo-phase commit gives cross-system atomicity by having a coordinator collect prepare-votes then commit — and pays with blocking: between prepare and commit, every participant holds locks waiting; a crashed coordinator leaves them locked indefinitely (in-doubt transactions needing manual resolution). Add PACELC: 2PC is maximal EC — multiple round trips of coordination per transaction, availability hostage to the slowest/deadest participant. Across microservices (heterogeneous stores, Redis and Rabbit and vendors that don't speak XA, teams that deploy independently) it's operationally dead. The alternative isn't "no transactions" — it's atomicity replaced by guaranteed eventual completion: the saga.
STEP 02

Saga mechanics

MANDATORY

A saga = a sequence of local transactions, each atomic in its own service, each publishing its completion. On failure at step N, run compensations for steps N-1…1 in reverse — semantic undo, not rollback. Watch a subscription purchase:

Purchase saga · orchestrated
T1
Order service · create order (PENDING)
T2
Payment service · authorize & capture
T3
Entitlement service · grant subscription
T4
Notification · send confirmation
Each step is a local ACID transaction. Between steps, the system is in a visible intermediate state — hold that thought for §04.
STEP 03

Choreography vs orchestration

MANDATORY
Choreography (events, no conductor)Orchestration (explicit coordinator)
Flow lives ineach service's subscriptions — emergentone state machine, readable end-to-end
Couplingloose; adding consumers is freeorchestrator knows every participant
Debuggability"where is order 4711?" = archaeology across N services' logssaga state table answers it in one query
Failure handlingcompensation logic scattered; cyclic-listening riskcentralized: timeouts, retries, compensation sequencing in one place
Risk profileinvisible complexity — the flow exists nowherethe orchestrator becoming a god-service with business logic pulled in
Staff expectationThe working rule: 2–3 steps, single team, no pivot ambiguity → choreography; anything with money, ≥4 steps, or cross-team ownership → orchestration — because at 3 AM the question is always "what state is this purchase in and what happens next?", and only an orchestrator's state table answers it cheaply. Guard the orchestrator's scope: it sequences and books state; it must not accrete the participants' business rules (the fat-gateway disease, M09, saga edition). Tooling spectrum: hand-rolled state table → Spring Statemachine → Temporal/Camunda-class engines; adopt an engine when saga count, not saga complexity, justifies the platform.
STEP 04

The isolation you gave up

MANDATORY

Fast trackThis is the section production sagas fail on. ACID's I is gone: intermediate states are visible and interleavable. The anomalies and their countermeasures:

STEP 05

Design rules, compressed

SKIM
Fast-track takeawayThe checklist: (1) every step + compensation idempotent with sagaId:step keys · (2) step order = compensatable → pivot → retriable · (3) every transition through the outbox · (4) saga state persisted and queryable (the 3 AM table) · (5) timeouts per step with explicit timeout-actions (a saga that can wait forever is a leak) · (6) semantic locks leased, never eternal · (7) end-to-end saga metrics: started/completed/compensated/stuck, age of oldest in-flight · (8) before building any saga, try to shrink it — often re-drawing service boundaries so the invariant lives in ONE service (single ACID transaction) beats coordinating two. The best saga is the one you redesigned away.
STEP 06

Exercise

MANDATORY

Fast trackSteps 1–2 (~60 min): building the orchestrator and then breaking it mid-saga is the module.

1
Build the purchase saga. Three Spring services (order, payment-stub, entitlement) + an orchestrator with a saga_state table, steps wired through RabbitMQ, every command carrying sagaId:step idempotency keys (M16's pattern). Happy path green.
2
Break it everywhere. (a) Payment fails → verify compensation order and final CANCELLED state. (b) Kill the orchestrator between T2 and T3; restart; verify resume from state table. (c) Redeliver T3's command twice; verify one grant. (d) Make the refund fail 3× then succeed; watch the retry ladder + stuck-saga alert fire.
3
Reproduce an anomaly. Run a concurrent "cancel subscription" flow against an in-flight purchase saga; capture the lost-update. Fix with a semantic lock (status flag + lease) and show the second flow now waits/aborts cleanly.
4
Paper. Take one real multi-service flow from your platform. First, argue whether a boundary re-draw could make it a single local transaction. If not: step table with compensations, pivot marked, isolation anomalies listed with countermeasures, topology chosen with the §03 rule.

Self-check

What exactly does a saga guarantee that 2PC guaranteed differently?
2PC: atomic visibility — all-or-nothing at one instant, bought with blocking and coordinated availability. Saga: guaranteed eventual completion in one of two terminal states (all steps done, or all executed steps compensated) — no atomicity, no isolation, full availability of participants in between. You trade "never see partial" for "never stuck holding locks"; §04 is the bill for that trade.
Why must post-pivot steps be retriable-forever rather than failable?
The pivot committed the business outcome (money captured). A post-pivot "failure" would require compensating the pivot — refunding a purchase the user legitimately made — because a notification email bounced. Post-pivot steps therefore retry until success (with DLQ + human escalation as the asymptote); anything that can legitimately fail belongs before the pivot.
Order 4711 is "stuck" — walk the diagnosis under orchestration vs choreography.
Orchestration: one query on saga_state → current step, attempts, last error, age; act on that step. Choreography: no single truth — grep order service (emitted?), broker (delivered? DLQ?), payment service (consumed? crashed?), entitlement (waiting on an event that never came?); the flow must be reconstructed from N partial views. That asymmetry, at incident frequency × saga count, is the strongest orchestration argument.
Why does a semantic lock need a lease, and what breaks without one?
The lock is cleared by the saga's completion — and sagas die (orchestrator crash before recovery, poison state). Without an expiry, the record stays PENDING_PURCHASE forever: every future saga on that subscription waits/aborts eternally — a wedged customer. Lease + expiry-with-reconciliation (check the actual saga state, then release or resume) is M19's lock discipline applied at the business layer.