Certificates as the identity layer of the internet, what mTLS actually changes inside your perimeter, cert lifecycle as an operational discipline (expiry is a self-inflicted outage), and zero trust as an architecture rather than a slogan.
~2.5h study~1h exercise2 interactive simsFast track ~65 min
Fast track — rotated a cert under fire before?
Compresses to ~65 min. TLS mechanics collapse (M06 covered the handshake cost); the trust chain, mTLS trade-offs, and zero-trust architecture stay mandatory.
§01 takeaway · 10 min
§02 chain sim, both runs · 10 min · MANDATORY
§03 mTLS, in full · 20 min · MANDATORY
§04 zero trust + lateral-movement sim · 20 min · MANDATORY
STEP 01
TLS in operational terms
SKIM
Fast-track takeawayTLS 1.3 gives confidentiality + integrity + server authenticity in 1 RTT (M06 priced it). The operational surface: where to terminate — edge termination (CDN/LB) with plaintext behind is the legacy default; re-encryption or mesh-mTLS behind the edge is the modern one (§04 explains why) · cert lifecycle is the real work — issuance (ACME/Let's Encrypt for public, internal CA for private), renewal automation, and expiry monitoring as an SLO (expired certs are a top-5 self-inflicted outage class; alert at 30/14/7 days, and know your longest-lived forgotten cert: the RabbitMQ listener, the webhook receiver, the partner SFTP) · TLS 1.3 only + strong defaults, and pin nothing in apps you can't update instantly (cert pinning in smart-TV firmware with a 2-year update lag is how you brick playback during rotation — pin CAs, not leaves, if at all).
STEP 02
The chain of trust
MANDATORY
A certificate is a signed claim: "this public key belongs to this name," countersigned up a chain to a root the verifier already trusts. Validation walks: leaf → intermediate(s) → trusted root, checking signatures, validity windows, name match (SAN), and revocation:
Validate api.stc.tv's chain
leaf
CN=api.stc.tv · signed by Intermediate R11
—
int
Intermediate R11 · signed by Root X2
—
root
Root X2 · in client trust store
—
The server must send leaf + intermediates; clients hold only roots. The classic failure is a server sending only its leaf.
The missing-intermediate bug is diagnosable in one line — openssl s_client -connect host:443 -showcerts — and explains "works in Chrome (AIA-fetches the intermediate), fails from Java" incidents.
Trust stores are config: the JVM's cacerts, the OS store, and your container base image can disagree — pin your base images' trust store state, and know that old smart-TV devices ship frozen root stores (root rotations like Let's Encrypt's have broken exactly this client class before — your device-matrix testing must include TLS).
Revocation is weak in practice (CRL/OCSP soft-fail); the industry's real answer is short-lived certs — which makes automation non-optional, and which is exactly the design mTLS meshes use internally (hours-long certs, §03).
STEP 03
mTLS: identity for machines
MANDATORY
Fast trackFull read. mTLS is the mechanism that turns "which pod is calling me" from an IP guess into a verified identity — the foundation everything in §04 stands on.
What changes: both sides present certificates. The server now knows which workload is calling — a cryptographic identity (SPIFFE-style: spiffe://cluster/ns/prod/sa/entitlement-service), not a network location. Authorization can then be "playback-api may call entitlement-service" instead of "10.2.0.0/16 may reach port 8443."
The cert-management problem it creates is the reason meshes won (M09): per-workload certs, minted at pod start, rotated every few hours, private keys never touching disk — this is not a spreadsheet-and-Jenkins problem; it needs machinery (Istio/Linkerd's identity plane, SPIRE, or cloud workload identity). Hand-run mTLS at 200 services is how cert expiry becomes your top incident class.
mTLS authenticates; it does not authorize. Knowing the caller is entitlement-service still needs a policy layer saying what entitlement-service may do — mesh authorization policies or app-level checks. And it doesn't replace user-level authN (M22): the request carries two identities — the workload (mTLS) and the end user (token) — and confusing them is a real bug class (a service trusting any in-mesh caller to assert any userId).
Edges of the mesh: external partners (mTLS with exchanged certs and painful manual rotation — negotiate overlap windows into the contract), managed databases/brokers (their own TLS + auth), and legacy boxes (the mesh's egress/ingress gateways as the boundary adapters).
STEP 04
Zero trust as an architecture
MANDATORY
The claim: network position is not identity. Being inside the VPC proves you breached the VPC. Compare what an attacker who compromises one service can reach:
thumbnail-service is compromised · what can the attacker reach?
Compromised
thumbnail-service 💀
Rest of the estate
entitlement-service
billing-service
user-db
image-store (its real dependency)
In the perimeter model, "inside" is one flat trust zone. Zero trust makes every call re-earn itself.
The implementation stack: workload identity everywhere (mTLS, §03) → default-deny service-to-service policy with explicit allow lists derived from real call graphs (your M09 dependency map becomes security policy) → end-user identity propagated and re-verified per hop (M22 tokens, not trusted headers) → per-request policy decisions, logged.
Getting there incrementally: permissive-mode mTLS first (observe, don't enforce) → enforce mTLS → observe the call graph → default-deny with the observed graph as the allow list → tighten. Each step is independently valuable; the big-bang version fails.
What it costs: the mesh's overhead (M09), policy-as-code maintenance (allow lists must live in repos with the services, or they rot), and debugging that includes authorization denials as a failure mode (make denials loudly observable — a silent 403 between services is a miserable incident).
Staff expectationFraming security investments by blast radius reduction: "today, any compromised pod can reach billing; after default-deny, a compromised thumbnail-service can reach exactly the image store" is an argument executives and auditors both understand — and it converts zero trust from a compliance checkbox into the same discipline as bulkheads (M13): containment as a first-class design property, for attackers instead of failures.
STEP 05
Exercise
MANDATORY
Fast trackStep 1 (~30 min): running your own CA and breaking the chain teaches cert debugging permanently.
1
Be a CA.openssl: root → intermediate → two leaf certs (client + server). Wire mutual TLS between two Spring Boot services (keystore/truststore). Break it four ways — expired leaf, missing intermediate in the served chain, wrong SAN, client cert from a different root — and match each to its exact Java exception. That mapping is on-call gold.
2
Audit expiry. Script (or use your monitoring) to inventory every TLS endpoint your platform serves or consumes — external, internal, partner, broker — with days-to-expiry. The inventory itself usually finds a cert nobody owns; file it.
3
Paper. Draw the two §04 diagrams for your real estate: today's reachability from one compromised mid-tier service, and the target default-deny graph derived from actual call patterns. The delta is the zero-trust roadmap, sequenced by the incremental path.
Self-check
curl works, Chrome works, your Java service fails with "unable to find valid certification path". Diagnose.
Browsers fetch missing intermediates (AIA) and carry huge trust stores; the JVM does neither by default. Either the server omits the intermediate (fix the served chain — the right fix) or the JVM trust store lacks the root (check cacerts in the *container image*, not your laptop). openssl s_client -showcerts tells you which in ten seconds.
Why do short-lived certs beat revocation, and what do they demand in exchange?
Revocation checking soft-fails (clients proceed when OCSP is unreachable), so a stolen long-lived cert stays usable. A 24-hour cert makes theft nearly worthless with no revocation infrastructure at all. The price: issuance/rotation must be fully automated and highly available — the CA/identity plane becomes serving-path infrastructure with M13-grade resilience requirements.
With mesh mTLS live, a service reads X-User-Id set by the gateway. Attack and fix?
Any in-mesh workload can send that header — mTLS proved *which service* is calling, not that the user claim is true; a compromised service can impersonate any user to any peer. Fix: propagate the actual end-user token (JWT) and verify its signature per hop (M22), or bind headers to the gateway's verified identity only (accept X-User-Id solely from the gateway's SPIFFE ID — weaker, but honest about its trust).
Sequence a zero-trust rollout for 200 services so nothing breaks. What's the invariant per step?
Permissive mTLS (measure adoption) → enforce mTLS (all traffic now identified) → log-only policy from the observed call graph → default-deny with that allow list → per-route tightening. The invariant: every step is observe-then-enforce — you never enforce a policy you haven't watched running in log-mode against real traffic, because the call graph you *think* you have always misses the monthly batch job that only runs on the 1st.