Notesstream🌱 seedling

Dead Letter Queue

Also: DLQ · Dead-Letter Exchange · DLX · Poison Message · Poison Pill · Redrive

A side queue where messages go after a bounded number of failed attempts, so one message that can never succeed (a poison message) stops blocking or burning the stream. It only helps if someone is alerted and redrives the messages after a fix.

Why it exists

Some messages always fail: malformed payloads, unknown schemas, references to deleted rows. Retried forever, they waste capacity in a queue. In a log, they block their whole partition (head-of-line blocking), because records are read in order.

How it works

  • Count attempts per message. After N attempts, publish it to the DLQ with error metadata (exception, attempt count, source offset), then ack or commit past it.
  • Separate permanent failures (bad data: dead-letter it) from transient ones (dependency down: Retry with Backoff and slow the consumer down).
  • Alert on DLQ depth. Once the bug is fixed, redrive the messages to the source queue or topic.

Trade-offs

Gains Costs
the stream keeps flowing past bad records ordering broken for dead-lettered keys
bad data preserved for diagnosis another queue to monitor and drain
bounded retry cost dead-lettering transient failures turns an outage into a backlog

In practice

  • SQS: a redrive policy moves a message after maxReceiveCount receives, and DLQ redrive moves it back.
  • RabbitMQ: dead-letter exchanges receive messages rejected with requeue=false, expired by TTL, over a length limit, or past a quorum queue's delivery limit (20 by default since 4.0).
  • Kafka: consumers dead-letter to their own topic; Kafka Connect sinks have a built-in DLQ; share groups added one in Kafka 4.4.

Pitfalls

  • A DLQ with no alert is silent data loss.
  • Redriving without fixing the cause just fills the DLQ again.
  • Dead-lettering one event of an entity, then processing later ones, can leave that entity inconsistent. Consider parking the whole key.

Recall

These cards are also in your review deck, scheduled with spaced repetition.