Notesstream🌱 seedling

Integrity vs Timeliness

Also: Integrity and Timeliness · Timeliness · Data Integrity · Perpetual Inconsistency

"Consistency" bundles two properties: timeliness (readers see up-to-date state; violations heal by themselves) and integrity (nothing lost, duplicated or contradictory; violations are permanent). Log-based systems can guarantee integrity while relaxing timeliness, without distributed transactions.

Also in: Streaming Systems as Architecture › Enforcing constraints

Why it exists

Distributed transactions buy both properties at once, at the cost of coordination and availability. Most applications care far more about never losing or double-applying a payment than about seeing it within milliseconds.

How it works

  • Write each multi-part operation to a log once, with a request ID. That single append is the atomic commit point.
  • A deterministic processor derives the parts (debit A, credit B), each tagged with the request ID, and routes them to the partitions that own each entity.
  • Every consumer deduplicates by request ID, so crashes and replays change nothing.
  • A reader may briefly see the debit without the credit: a timeliness violation that heals.
  • Constraints that need coordination (uniqueness of a chosen value, "never below zero") are enforced by routing them to one log partition and deciding them in order. Others can be checked late and fixed by compensation (an apology), as businesses already do with overbooking.

Trade-offs

Distributed transaction Log + idempotent processing
integrity yes yes
timeliness immediate eventual (seconds)
coordination every write, across systems one partition append

In practice

  • Invariant confluence identifies which invariants survive coordination-free execution: 10 of TPC-C's 12 do. Uniqueness of a client-chosen value and lower bounds under concurrent decrements don't.

Pitfalls

  • Relaxing integrity when you meant to relax timeliness, for example by dropping request IDs to "simplify".
  • Treating a lagging view as corrupt and "fixing" it, which turns a timeliness issue into an integrity one.

Recall

These cards are also in your review deck, scheduled with spaced repetition.