Notesfoundations🌱 seedling

Scalability

Also: Scalable System · Scaling

Scalability is a system's ability to cope with increased load. It isn't a yes/no property: it only means something once you say which load grows, by how much, and what performance you need to keep.

Why it exists

The most common reason a working system degrades is growth, not bugs: 10× the users, 1,000× the data. Scalability is the discipline of asking, before it happens, "if load grows this way, what are our options?"

How it works

Three steps:

  • Describe the load with a few Load Parameters: requests per second, read/write ratio, data size, fan-out, and the shape of skewed distributions.
  • Describe performance as a distribution: percentiles for online systems, throughput for batch.
  • Add capacity: Vertical Scaling (a bigger machine), Horizontal Scaling (more machines), or a design change such as Fan-Out on Write that moves work off the hot path.
  • Queueing explains why performance falls off a cliff near saturation; Little's Law links throughput, latency and concurrency.

Trade-offs

Gains Costs
headroom for growth without a rewrite architecture tuned for one load shape is wrong for another
options ready before you need them building for scale you don't have is expensive and slows you down

In practice

  • ChatGPT's PostgreSQL (2026): one primary plus ~50 read replicas served 800M users; shardable, write-heavy workloads are being moved to sharded systems.
  • Twitter timelines (2012): the hard load parameter wasn't tweets per second but followers per account.

Pitfalls

  • "It scales" with no load parameter attached is a sentiment, not a property.
  • An architecture for a million users is usually wrong for a hundred, and the other way round.
  • Elasticity (automatic scaling) helps unpredictable load but adds operational surprises; manual scaling is simpler.

Recall

These cards are also in your review deck, scheduled with spaced repetition.