NotesSystem

Vitess

Also: Vitess MySQL · VTGate

A sharding layer for [[MySQL]], originally built at YouTube and now a CNCF project. Applications talk to it like one MySQL database; it routes each query to the shards that own its keys.

Also in: Distributed Transactions › In the real world

Design choices

  • Vindexes: a primary Vindex maps a column (often through a hash) to a keyspace ID. Hash Partitioning in effect, then Range Partitioning over keyspace IDs.
  • Range-named shards: a shard's name is the keyspace-ID range it owns: -80, 80-c0, c0-. Shards needn't be equal.
  • Request Routing: VTGate is a stateless routing tier that computes the keyspace IDs for each query; the topology lives in etcd, ZooKeeper or Consul.
  • Rebalancing: Reshard and MoveTables workflows copy data with VReplication, verify with VDiff, then SwitchTraffic (reads first, then writes) with only a brief write cutover, keeping reverse replication for rollback.
  • Cross-shard transactions: the default MULTI mode is best-effort and can partially commit; TWOPC mode gives atomic commits with Two-Phase Commit, at a latency cost, and still allows fractured reads.

Notable trade-offs

  • Queries that don't include the sharding key scatter to every shard.
  • Each shard is still a MySQL primary with replicas, so per-shard failover and replication lag remain your concern.