Notesstream🌱 seedling
Event Time vs Processing Time
Also: Event Time · Processing Time · Ingestion Time · Event-Time Skew · Out-of-Order Events
Event time is when something happened; processing time is when the stream processor sees it. The gap between them (skew) is never zero and never constant, so results about the world must be computed in event time to be correct and reproducible.
Why it exists
Events are delayed by offline devices, producer batching, broker lag and consumer restarts. During a replay, processing time is "now" for every event, so anything bucketed by processing time changes on every run.
How it works
- Each event carries an event timestamp, set at the source. The processor assigns records to windows by that timestamp.
- Skew varies per event. A phone offline for three minutes delivers its events three minutes late, all at once.
- Because the input is never complete, the processor needs a Watermark to decide when a window's data has probably all arrived.
- Ingestion time (stamped at the broker) is a middle ground: monotonic and simple, but still not when the event happened.
Trade-offs
| Event time | Processing time | |
|---|---|---|
| correctness under delay | right | wrong (events land in the wrong bucket) |
| replay / backfill | same answer | different answer |
| latency | waits for watermarks | immediate |
| complexity | late data, watermarks | none |
In practice
- Flink, Beam, Kafka Streams window by record timestamps (event time) by default.
- Use processing time for pipeline-health metrics (throughput, lag) or where approximate-now beats correct-later.
Pitfalls
- Device clocks drift and can be wrong. Record the receive time too, and bound implausible timestamps (Clock Skew).
- Processing-time windows look fine live, then disagree with the backfill.
Connections
- Part of
- Stream Processing
- Related
- Clock SkewWatermarkWindowing
- In real systems
- Apache Flink
Recall
These cards are also in your review deck, scheduled with spaced repetition.