What the transport layer actually costs you: handshakes in RTTs, head-of-line blocking through HTTP/1.1 → 2 → 3, when UDP is the right answer, and why OTT video streams over plain HTTP anyway.
~2.5h study~1.5h exercise2 interactive simsFast track ~70 min
Fast track — comfortable at L4?
Compresses to ~70 min. Protocol mechanics collapse; HOL blocking across the HTTP generations and the handshake-latency math stay mandatory — they decide real architecture choices (HTTP/2 for gRPC, HTTP/3 at the edge).
§01–§02 takeaways · 10 min
§03 HOL blocking + sim, in full · 20 min · MANDATORY
§04 handshake sim + connection-reuse math · 15 min · MANDATORY
§05 OTT delivery reality, in full · 15 min · MANDATORY
Exercise step 2 · 20 min
STEP 01
TCP: what you're paying for
SKIM
Fast-track takeawayTCP sells four guarantees — delivery, ordering, integrity, congestion control — and bills you in RTTs (3-way handshake), buffers, and stalls. The knobs that matter in Java estates: TCP_NODELAY on (Nagle + delayed-ACK interaction adds up to ~40 ms to small RPCs — the classic "mystery 40 ms"), keepalives shorter than your NAT/LB idle timeout (or pooled connections die silently and the next request eats a reconnect), and congestion control meaning throughput ≈ window/RTT — long fat pipes need window scaling, and loss halves your rate (why cross-region bulk transfer underuses the pipe).
Connection = state: 3-way handshake (SYN, SYN-ACK, ACK — 1 RTT before any data), send/receive buffers, sequence numbers on both ends. This state is what connection pools amortize and what LBs must track (Module 08).
Reliability & ordering: lost segments retransmit; later segments wait in the kernel until the gap fills. That wait is head-of-line blocking (§03) — the property everything after TCP is trying to escape.
Flow vs congestion control: flow control protects the receiver (advertised window); congestion control protects the network (cwnd: slow start, AIMD, CUBIC/BBR). Effective throughput ≤ window / RTT — a 64 KB window at 100 ms RTT caps at ~5 Mbps regardless of link speed.
The small-write trap: Nagle's algorithm batches small writes; combined with delayed ACKs it can add ~40 ms to request-response protocols. Every serious RPC stack sets TCP_NODELAY — verify yours does.
Idle connections lie: NATs and LBs silently drop idle flows (often 350 s on cloud LBs). Without keepalive below that threshold, a pooled connection's next use costs a timeout + reconnect. This is the root of "first request after quiet period is slow" tickets.
STEP 02
UDP: the escape hatch
SKIM
Fast-track takeawayUDP = datagrams, no handshake, no ordering, no retransmit, no congestion control — a blank sheet. Right when: latest-value-wins telemetry (a retransmitted stale heartbeat is worse than a lost one), DNS-style tiny request/reply, real-time media where a late packet is a useless packet (WebRTC/live), and as the substrate for QUIC, which rebuilds TCP's guarantees per-stream in userspace. Wrong when you'd end up reimplementing TCP badly. Ops reality: UDP is deprioritized/blocked by some middleboxes — every UDP protocol ships a TCP fallback.
The selection logic, compressed: use UDP when retransmission is worse than loss. A live-position heartbeat from a player that arrives 2 s late via retransmit is misinformation; dropping it and sending the next one is correct. When some reliability is needed but TCP's global ordering isn't, modern practice is not raw UDP but QUIC — reliability and congestion control per-stream, in userspace, over UDP, with TLS built in.
STEP 03
Head-of-line blocking: HTTP/1.1 → 2 → 3
MANDATORY
Fast trackRead fully. Each HTTP generation moved HOL blocking down one layer instead of eliminating it — knowing which layer it lives in per protocol is what makes the HTTP/2-vs-3 decision rational instead of fashionable.
HTTP/1.1: one request at a time per connection (pipelining is dead in practice). HOL at the application layer: one slow response blocks the connection. Workaround was 6+ parallel connections per host — expensive and unfair.
HTTP/2: many streams multiplexed on one TCP connection. Application-layer HOL solved — but all streams share one TCP byte sequence, so one lost packet stalls every stream: HOL moved down to the transport. On clean networks this is fine; on lossy networks HTTP/2 can be slower than HTTP/1.1 with its parallel connections.
HTTP/3 (QUIC): streams are independent at the transport. A lost packet stalls only the stream whose data it carried. HOL is finally confined to within a single stream — which is inherent, since bytes of one response must arrive in order.
HOL blocking · 3 responses on one connection, packet 4 lost
manifest
thumbnail
subtitles
One packet of the thumbnail response will be lost. Watch who else pays for it under each protocol.
Staff expectationMapping this to decisions: gRPC runs on HTTP/2 — inside a low-loss datacenter, transport HOL is negligible and HTTP/2's multiplexing is pure win. At the edge — mobile networks, wifi, the last mile where your viewers live — loss is routine, which is exactly where HTTP/3 pays. Hence the standard shape: HTTP/3 client↔CDN/edge, HTTP/2 (gRPC) service↔service. Same reasoning, opposite conclusions, because the loss rate differs by two orders of magnitude.
STEP 04
The handshake tax
MANDATORY
Before the first byte of response: TCP handshake (1 RTT) + TLS 1.3 (1 RTT) + request/response (1 RTT) = 3 RTTs cold. QUIC merges transport and TLS into 1 RTT; both support 0/1-RTT resumption for repeat visitors. At mobile-edge RTTs this is the difference between snappy and sluggish app start:
Cold connection to first response byte
TCP + TLS 1.3 + HTTP
SYN → · ← SYN-ACK · ACK → (1 RTT)
ClientHello → · ← ServerHello+Fin → (1 RTT)
GET → · ← first byte (1 RTT)
first byte at 180 ms
QUIC (HTTP/3)
Initial+crypto → · ← handshake done (1 RTT)
GET → · ← first byte (1 RTT)
resumption: 0-RTT — GET rides the first flight
first byte at 120 ms · resumed: 60 ms
Two consequences that matter more than the protocol choice itself: connection reuse dominates everything — a pooled/kept-alive connection pays 1 RTT per request instead of 3, so pool tuning and keepalive hygiene (§01) usually beat protocol upgrades; and QUIC connection migration — connections survive IP changes (wifi→cellular) via connection IDs, which for a mobile-heavy OTT audience means mid-stream network switches don't reset delivery.
STEP 05
Why OTT video streams over HTTP anyway
MANDATORY
The apparent paradox: video is latency-sensitive real-time media, yet HLS/DASH deliver it over plain HTTP on TCP — the "wrong" stack by §02 logic. The resolution is that VOD and live-with-buffer are not real-time: the client holds a multi-second buffer, so throughput and cacheability beat per-packet latency. HTTP delivery buys:
The entire CDN ecosystem for free: segments are just cacheable HTTP objects — every edge cache, origin shield, and request-collapsing mechanism (Module 10) applies. A custom UDP protocol would forfeit all of it.
Adaptive bitrate as an application-layer decision: the client measures segment download throughput and switches renditions — no transport cooperation needed.
Middlebox and firewall traversal: port 443 works everywhere; bespoke UDP doesn't.
The genuinely real-time cases — sub-second live sports betting feeds, video calls, cloud gaming — do go WebRTC/QUIC, accepting the operational cost because buffered delivery can't meet the latency. And modern low-latency HLS/DASH (LL-HLS chunked transfer, ~2–5 s glass-to-glass) plus HTTP/3 at the edge is the current mainstream compromise: HTTP semantics, shrinking buffers, QUIC transport underneath.
The pattern worth generalizing"Which transport?" is really "where does my latency tolerance come from?" A buffer anywhere in the pipeline converts a latency problem into a throughput problem — and throughput problems have cheaper, more cacheable solutions. This trade reappears in messaging (Module 15): queues are buffers doing the same conversion.
STEP 06
Exercise: see the tax on the wire
MANDATORY
Fast trackStep 2 only (~20 min): measuring the 40 ms Nagle interaction on your own machine permanently changes how you read RPC latency graphs.
1
Watch a handshake.curl -w "%{time_connect} %{time_appconnect} %{time_starttransfer}\n" -o /dev/null -s https://example.com against a nearby and a far origin. Decompose each into RTTs; repeat with --http3 if your curl supports it.
2
Reproduce the mystery 40 ms. Java echo client/server exchanging small messages in a request-response loop. Measure p50 with setTcpNoDelay(false) vs true. Capture with tcpdump and find the delayed-ACK gap in the trace.
3
Break a pool. Set an aggressive idle timeout on a local HAProxy/nginx in front of your echo server; make your Java client's pool keepalive longer than it. Observe the first-request-after-idle failure/latency, then fix with keepalive < LB idle timeout. Write the two numbers into your team's HTTP-client defaults doc.
4
Stretch — loss and HOL.tc qdisc add dev lo root netem loss 3%, then compare a 20-request burst over HTTP/1.1 (6 connections), HTTP/2 (1 connection), and HTTP/3 via curl timings. Explain the ranking you observe using §03.
Self-check
Your gRPC p50 between two services is 42 ms but network RTT is 1 ms. First suspicion?
The Nagle × delayed-ACK interaction (~40 ms) — small writes being held for batching while the peer withholds the ACK. Check TCP_NODELAY on the client channel; most gRPC stacks set it, but custom Netty bootstraps sometimes lose it.
Why can HTTP/2 be slower than HTTP/1.1 on a lossy mobile link?
HTTP/2 concentrates all streams on one TCP connection, so every loss stalls everything (transport HOL). HTTP/1.1's 6 parallel connections are 6 independent loss domains — accidental fault isolation. HTTP/3 gets the same isolation per-stream, deliberately and without the connection overhead.
Player heartbeats every 2 s carrying playback position: TCP or UDP, and why is the naive answer wrong either way?
Semantically it's latest-value-wins → UDP-shaped. But in practice you'd send it over the existing HTTPS/QUIC session: a new UDP path means new firewall/middlebox risk, separate auth, separate observability — for a payload whose loss tolerance you can get by just not retrying stale beats at the app layer. Transport purity loses to operational reality; know why, not just that.
What does QUIC connection migration give a mobile OTT audience that TCP can't?
TCP connections are bound to the 4-tuple (IPs+ports) — switch from wifi to cellular and every connection resets: player re-handshakes, possibly re-buffers. QUIC identifies connections by connection ID, so the same session continues across the IP change; delivery hiccups instead of resetting.