The edge's job description, how services find each other and how that knowledge goes stale, the fat-gateway failure mode, and the honest service-mesh trade-off for a 200-service estate.
~2.5h study~1.5h exercise1 interactive simFast track ~70 min
Fast track — living this daily?
Compresses to ~70 min. Responsibility lists collapse; discovery staleness math, gateway anti-patterns, and the mesh decision stay mandatory.
§01 takeaway · 5 min
§02–§03 discovery + staleness sim, in full · 25 min · MANDATORY
§04 anti-patterns, in full · 15 min · MANDATORY
§05 mesh trade-off, in full · 15 min · MANDATORY
Exercise step 3 · 15 min
STEP 01
What the gateway owns
SKIM
Fast-track takeawayThe gateway owns cross-cutting edge concerns exactly once: TLS termination, authN (token validation — not authZ business rules), coarse rate limiting, routing/canaries, request normalization, edge observability. The test for whether logic belongs there: "is this identical for every route?" Yes → gateway. No → it's business logic wearing a gateway costume (§04). BFFs sit beside it: per-client-type aggregation layers (mobile BFF, TV BFF) that own screen-shaping so neither gateway nor domain services do.
Terminate & authenticate: TLS, token signature/expiry validation, attaching verified identity headers for downstream trust (paired with mTLS or signed headers so services can trust them — Module 21).
Protect: coarse per-client rate limits, request size caps, basic WAF — the cheap 80% of abuse handling at the cheapest layer.
Route: path/host/header routing, weighted canaries, header-based dark launches. The gateway is where traffic-shaping policy lives because it's the one hop all external traffic shares.
Observe: the authoritative edge request log — the "what did the user actually experience" source of truth that per-service logs can't give.
STEP 02
Discovery: who knows where everyone is
MANDATORY
Server-side discovery — client calls a stable name (VIP/Service); an LB consults the registry. Clients stay dumb; you pay an extra hop and inherit the LB's algorithm limits (Module 08). This is vanilla Kubernetes: DNS name → kube-proxy → endpoints.
Client-side discovery — client fetches the instance list and balances itself (Eureka/Consul + Spring Cloud LoadBalancer, or gRPC resolver on headless-Service DNS). Per-call decisions over long-lived connections — the Module 07/08 gRPC answer — at the cost of discovery logic in every client runtime.
Registration: self-registration (service heartbeats the registry — simple, but a wedged-but-heartbeating process stays registered) vs third-party registration (the platform registers from its own pod lifecycle knowledge — k8s endpoints; strictly better signal when available).
Registry health semantics decide partition behavior: a CP registry (etcd/Consul-strict) can refuse reads during partitions — discovery down = everything down; an AP registry (Eureka's design) serves stale instance lists on the theory that stale endpoints + client retries beat no endpoints. For discovery, AP-with-staleness is almost always the right PACELC call (Module 03) — which makes staleness handling a first-class design problem, next section.
STEP 03
The staleness window
MANDATORY
Every discovery system is a cache of cluster topology, and Module 01's rules apply: it has a TTL, and it lies during the window. When an instance dies, requests keep routing to the corpse until heartbeat timeout + client refresh interval elapse:
Requests round-robin A→B→C. Kill B and watch the client keep sending it traffic through the staleness window.
The three defenses, in the order they save you:
Retry-next-instance on connect failure — the staleness window becomes a latency blip instead of an error. This is why discovery-based estates treat connection-level retry as mandatory, distinct from request retry semantics (idempotency — Module 16).
Passive outlier ejection in the client LB (Module 08) — the client stops trusting the ghost long before the registry admits it's dead.
Tighten the window with eyes open: faster heartbeats/refresh = more registry load and more flapping on transient blips. The window is a knob on the same availability↔freshness axis as every cache TTL you've tuned.
Staff expectationBeing able to state your estate's actual worst-case staleness window as a number — heartbeat interval × miss threshold + client refresh period + DNS/connection-pool TTLs stacked on top — and what percentage of a deploy's traffic that window can misroute. Most teams discover their stack has three stacked caches of topology (registry, client list, connection pool) only during the incident.
STEP 04
Gateway anti-patterns
MANDATORY
The fat gateway. Aggregation, response mapping, "small" business rules accrete at the edge because it's the easy place to deploy. End state: a monolith in front of your microservices — every team's changes queue behind one deploy pipeline, one blast radius, one shared fate. Guard rail: the gateway config is policy (routes/limits), never code with domain knowledge; aggregation lives in BFFs owned by client teams.
Shared-fate underprovisioning. The gateway is 100% of external traffic; it must be capacity-planned for peak-of-peaks plus retry amplification (Module 02's feedback loop arrives here first) and deployed more conservatively than anything behind it.
AuthZ at the edge. The gateway can verify who; only services know may they. Edge-enforced business authorization drifts from the domain rules and fails open on new routes. Verify identity once at the edge, decide permissions in the domain.
Gateway-level retries without budgets. Edge retries multiply every downstream brownout by the retry factor at the worst possible layer. Retries at the gateway need budgets and only on idempotent routes (Modules 12/16).
Discovery bypass hardcoding. "Temporary" hardcoded IPs/hosts in configs outlive the instances they point to; the registry exists so topology changes are non-events. Lint for it.
STEP 05
Service mesh: the honest trade
MANDATORY
A mesh (Istio/Linkerd-class: Envoy at every pod, sidecar or ambient) moves Module 07/08's client concerns — per-call LB, retries, timeouts, outlier detection, mTLS, telemetry — out of application code into infrastructure, uniformly across every language and legacy service. That uniformity is the entire value proposition, and it's real: one policy change rolls consistent retry budgets or mTLS across 200 services without touching a single pom.xml.
The honest costs: per-pod resource overhead × every pod; +1–2 network hops of latency per call each way; a control plane that is itself a critical distributed system to operate and upgrade; and debugging that now includes "is it the app or the sidecar" on every incident.
The decision ruleCount how many of these you're already solving per-service in libraries: mTLS everywhere, uniform retry/timeout policy, per-call gRPC balancing, consistent traffic-split canaries, uniform golden-signal telemetry. Solving ≥3 of them badly in N languages/framework-versions ≥ the mesh's operating cost → mesh wins. Solving them fine in one shared Spring Boot starter your platform team controls → the starter is your mesh, cheaper. The variable is organizational: heterogeneity and platform-team capacity, not technology.
STEP 06
Exercise
MANDATORY
Fast trackStep 3 (~20 min): measuring your own stacked staleness window converts §03 from concept to number.
1
Build the edge. Spring Cloud Gateway in front of two services: route predicates, a token-validation filter attaching X-User-Id, a Redis-backed rate limiter, and a 90/10 weighted canary route. Verify each with curl.
2
Wire discovery. Eureka (or Consul) + Spring Cloud LoadBalancer, two instances of a service. Watch the gateway's instance list; scale to three and observe propagation delay.
3
Measure the window.kill -9 one instance under steady load. Time from kill → last failed request, with default heartbeat/refresh settings. Then add connect-failure retry-next-instance and repeat: errors → zero, staleness window → latency blip. Record both numbers; compare with your production stack's settings.
4
Paper. Audit your real gateway config: list everything it does, mark each item policy vs domain-logic-in-disguise, and write the eviction plan for the latter. Then apply the §05 decision rule to your estate honestly — count the ≥3 list.
Self-check
Why is an AP registry usually right, given it guarantees serving lies?
A CP registry that refuses reads during partitions makes discovery a single point of total failure — nobody can find anybody, including healthy pairs. Stale lists + client-side defenses (retry-next, outlier ejection) degrade to a few misrouted-then-recovered requests. Discovery data is self-healing and loss-tolerant; the PACELC call (Module 03) is clear.
Where does authorization belong and why not the gateway?
Authentication (who is this) at the gateway — uniform, route-independent. Authorization (may they do this) in domain services — it *is* domain logic: entitlement windows, plan tiers, regional rights. Edge-enforced authZ duplicates domain rules, drifts, and silently fails open on routes added after the rule.
A wedged-but-heartbeating instance: which registration model catches it, and what catches it regardless?
Self-registration can't — the heartbeat thread happily outlives the useful process. Third-party registration keyed on real health (k8s readiness) can. Regardless: passive outlier ejection in callers ejects it on observed errors — reality-based signal beats any registration model, which is why you run both.
Your gateway team wants to add "combine catalog + entitlement into one response for mobile". Ruling?
Not in the gateway — that's aggregation with domain knowledge (what mobile screens need, how entitlement shapes catalog). It belongs in a mobile BFF owned by the mobile-facing team, deployed on their cadence. The gateway routes to the BFF. Same feature, right blast radius.