Notesfoundations🌱 seedling

Load Balancer

Also: LB · Reverse Proxy · Load Balancing

A load balancer puts one stable address in front of a pool of interchangeable backends and decides, per request or per connection, which healthy backend gets it. It spreads load, routes around failures and hides changes to the pool.

Why it exists

One server has a throughput ceiling and is a Single Point of Failure. Adding servers (Horizontal Scaling) means something has to answer "which one?" for every request, without clients knowing the pool.

How it works

  • Clients address a virtual IP or DNS name, never a backend.
  • The balancer keeps a pool, usually fed by Service Discovery.
  • A Health Check signal filters the pool to backends that should get traffic now.
  • A load balancing algorithm picks one, e.g. the Power of Two Choices.
  • Replies flow back through it, or, in some L4 designs, straight from the backend (direct server return).
  • It can sit in a dedicated tier, in a sidecar proxy, or inside the client library (see L4 vs L7 Load Balancing).

Trade-offs

Placement Gains Costs
dedicated tier one place to configure, observe, terminate TLS extra hop; it must be made redundant itself
sidecar proxy per-instance, language-agnostic, rich L7 features a proxy per instance to run and upgrade
client-side library no extra hop, no shared bottleneck discovery and logic in every client, in every language

In practice

  • Draining: it lets you remove a backend gracefully (stop new work, let in-flight requests finish), which is what makes a Rolling Upgrade zero-downtime.
  • Warm-up: new backends get a ramp ("slow start") so cold caches and JITs aren't flooded.

Pitfalls

  • A redundant fleet behind one balancer just moves the single point of failure.
  • Killing a backend without a drain period cuts in-flight requests.

Recall

These cards are also in your review deck, scheduled with spaced repetition.