Notesfoundations🌱 seedling

Rolling Upgrade

Also: Rolling Update · Rolling Deployment · Staged Rollout · Version Skew

A rolling upgrade replaces a fleet's nodes one (or a few) at a time, so the service never goes down. For its duration old and new versions run side by side, so they must be compatible in both directions.

Why it exists

If the system tolerates losing any machine (Fault Tolerance), it can lose one on purpose: drain it, upgrade it, return it, repeat. No maintenance window needed.

How it works

  • Bound the churn: Kubernetes Deployments default to at most 25% of pods unavailable and 25% surge, and flag a rollout as stuck after 600 s without progress.
  • Health and readiness checks gate each step; a failing step stops the rollout.
  • Mixed versions share data and messages, so you need Backward and Forward Compatibility: new code reads old data; old code reads new data and preserves fields it doesn't understand.
  • Write the rule down: Kubernetes' own skew policy keeps API servers within one minor version and lets kubelets lag up to three, never lead.

Trade-offs

Gains Costs
zero planned downtime a window where two versions coexist
rollback is the same mechanism in reverse every change must be compatible both ways
capacity drops during the roll unless you surge

In practice

  • Pair with a Canary Release: the canary is the first step of the roll, with an evaluation before continuing.

Pitfalls

  • Old code that reads a record, drops an unknown field and writes it back: silent data loss mid-rollout.
  • Assuming rollback is safe when new code has already written data the old code can't read.

Recall

These cards are also in your review deck, scheduled with spaced repetition.