Notespartitioning🌱 seedling
Hash Partitioning
Also: Hash Sharding · Hashed Sharding · Partitioning by Hash of Key · Compound Key · Partition Key and Sort Key
Place each record by a hash of its key rather than the key itself, so consecutive keys scatter evenly across shards. Writes spread out; range scans over the original key now hit every shard.
Why it exists
Range Partitioning turns skewed or sequential keys into a Hot Spot. A good hash makes similar keys unrelated, so load spreads regardless of key order.
How it works
shard = f(hash(key)). The hash needn't be cryptographic (Cassandra: Murmur3; Redis Cluster: CRC16) but must be stable across processes, languages and versions.- The hash space is then split into ranges, slots or a ring (Consistent Hashing). Plain
hash mod Nmoves most keys when N changes. - Compound keys recover locality: hash the first part (partition key) to pick the shard, sort by the rest (sort key / clustering columns) inside it.
(sensor_id, time)spreads sensors yet keeps each sensor's history in order.
Trade-offs
| Gains | Costs |
|---|---|
| even spread of keys and writes | range queries on the original key fan out to every shard |
| no need to know the key distribution | one very popular key is still one shard |
| compound keys give per-entity order | queries must supply the partition-key part |
In practice
- DynamoDB hashes the partition key and keeps each item collection sorted by the sort key.
- Figma hashed its shard keys because auto-increment and timestamp-prefixed IDs would have made hot shards.
- Discord partitions messages by
(channel_id, bucket), a ~10-day bucket, to bound partition size.
Pitfalls
- Language built-in string hashes are often seeded per process; two services would disagree on the shard.
- Believing hashing evens out load: it evens out keys.
Connections
- Part of
- Partitioning
- Contains
- Consistent Hashing
- Learn first
- Horizontal Scaling
- Contrasts with
- Range Partitioning
- In real systems
- DynamoDBCassandraVitessApache KafkaMongoDB
Recall
These cards are also in your review deck, scheduled with spaced repetition.