Notespartitioning🌱 seedling

Hash Partitioning

Also: Hash Sharding · Hashed Sharding · Partitioning by Hash of Key · Compound Key · Partition Key and Sort Key

Place each record by a hash of its key rather than the key itself, so consecutive keys scatter evenly across shards. Writes spread out; range scans over the original key now hit every shard.

Why it exists

Range Partitioning turns skewed or sequential keys into a Hot Spot. A good hash makes similar keys unrelated, so load spreads regardless of key order.

How it works

  • shard = f(hash(key)). The hash needn't be cryptographic (Cassandra: Murmur3; Redis Cluster: CRC16) but must be stable across processes, languages and versions.
  • The hash space is then split into ranges, slots or a ring (Consistent Hashing). Plain hash mod N moves most keys when N changes.
  • Compound keys recover locality: hash the first part (partition key) to pick the shard, sort by the rest (sort key / clustering columns) inside it. (sensor_id, time) spreads sensors yet keeps each sensor's history in order.

Trade-offs

Gains Costs
even spread of keys and writes range queries on the original key fan out to every shard
no need to know the key distribution one very popular key is still one shard
compound keys give per-entity order queries must supply the partition-key part

In practice

  • DynamoDB hashes the partition key and keeps each item collection sorted by the sort key.
  • Figma hashed its shard keys because auto-increment and timestamp-prefixed IDs would have made hot shards.
  • Discord partitions messages by (channel_id, bucket), a ~10-day bucket, to bound partition size.

Pitfalls

  • Language built-in string hashes are often seeded per process; two services would disagree on the shard.
  • Believing hashing evens out load: it evens out keys.

Recall

These cards are also in your review deck, scheduled with spaced repetition.