Notespartitioning🌱 seedling

Rebalancing

Also: Resharding · Shard Rebalancing · Partition Rebalancing · Range Splitting · Shard Splitting · Logical Shards

Moving data between nodes as data, load or the node count changes, so that load ends up fair, the database keeps serving during the move, and no more data moves than necessary.

How it works

Three ways to decide what moves:

  • Fixed number of partitions: many more partitions than nodes from day one; whole partitions move, keys never change partition. Redis Cluster's 16,384 slots, Elasticsearch shards per index, Notion's 480 logical Postgres shards.
  • Dynamic splitting: split partitions that grow too big or busy, merge small ones. CockroachDB (512 MiB ranges), HBase, DynamoDB.
  • Proportional to nodes: each node owns a fixed number of ring tokens (Consistent Hashing); a new node takes a share from everyone.

How a move happens safely: copy while the old owner serves, catch up from its change log, verify, then flip ownership in the Request Routing map, keeping reverse replication for rollback.

Trade-offs

Strategy Gains Costs
fixed partitions simple moves, no rehashing count is a ceiling chosen up front
dynamic splits partitions track data can't split a single hot key; empty tables start on one node
per-node tokens grows with the cluster random token choice can be uneven

In practice

  • Notion went from 32 to 96 Postgres hosts in 2023 (15 → 5 logical shards each) with logical replication; users saw at most about a second of a "saving" spinner.
  • Vitess Reshard: copy, VReplication catch-up, VDiff, SwitchTraffic (a brief write cutover), with reverse replication until completed.

Pitfalls

  • Fully automatic rebalancing plus automatic failure detection: a slow node is declared dead, its data moves, the copying slows others, and the cluster spirals.
  • Picking a partition count with few divisors (Notion chose 480 because it divides evenly by many host counts: 32, 40, 48, 96…).

Recall

These cards are also in your review deck, scheduled with spaced repetition.