Notesfoundations🌱 seedling

Timeout

Also: Deadline · Deadline Propagation · Request Timeout

A timeout bounds how long a caller waits for a remote call before treating it as failed. A deadline is the same budget set once at the edge and propagated, so every hop gives up when the original caller does.

Why it exists

A remote call can hang for many reasons (paused process, lost packet, held lock). Without a bound, the caller holds a thread, a connection and memory indefinitely; enough stuck calls exhaust the caller too, and a slow dependency becomes a Cascading Failure. Slow is more dangerous than dead: dead fails fast.

How it works

  • Pick the value from data: choose an acceptable false-timeout rate (say 0.1%) and set the timeout at the matching percentile of the dependency (p99.9), plus padding for network variance.
  • Propagate a deadline: the edge sets an absolute budget; each hop spends some and forwards only the remainder. Sending remaining time rather than a wall-clock time avoids clock skew.
  • The server must stop too: a cancelled request should abort its own downstream work, or the deadline only saved the caller.
  • Many clients ship with no default timeout (gRPC sets no deadline unless you do), so the bug is invisible until a dependency slows down.

Trade-offs

Too long Too short
stuck calls tie up threads and connections abandons calls that would have succeeded
failures surface slowly false failures trigger retries (Retry with Backoff), adding load

In practice

  • gRPC: deadlines are first-class; the client gets DEADLINE_EXCEEDED, the server sees the call cancelled. Java and Go propagate deadlines to outgoing calls automatically.
  • Load shedding: a server that knows a request's deadline can drop it before doing any work if it has already waited too long in a queue (Load Shedding).

Pitfalls

  • Independent per-hop timeouts longer than the caller's: servers keep working for clients that already left.
  • One global timeout for every call, regardless of each dependency's latency profile.

Recall

These cards are also in your review deck, scheduled with spaced repetition.