Notesencoding🌱 seedling

Message Passing

Also: Asynchronous Messaging · Async Messaging · Message-Driven Communication

In asynchronous message passing a sender hands a message to an intermediary, usually a message broker, and carries on without waiting. The broker buffers, redelivers and fans out messages, decoupling sender and recipient in time and address.

Why it exists

Direct calls need both sides up at the same moment and couple the caller to the callee's address and speed. A broker absorbs outages and bursts, and lets new consumers subscribe without the sender changing.

How it works

  • The sender publishes to a queue or topic on a Message Broker and moves on; communication is one-way unless you add a reply queue.
  • The broker buffers while consumers are down, redelivers unacknowledged messages, and fans out to many consumers.
  • Delivery is usually at least once, so consumers see duplicates and must be idempotent: record processed message IDs atomically with their effects, or make effects naturally idempotent.
  • Messages are encoded bytes: producers and consumers evolve independently, often under a Schema Registry.
  • Actor frameworks (Erlang/OTP, Akka, Orleans) apply the same model inside a program, assuming messages can be lost.

Trade-offs

Gains Costs
sender and receiver needn't be up together no immediate reply; results arrive later, if at all
absorbs bursts and outages another system to run; backlogs can hide problems
easy fan-out and new consumers duplicates and reordering to handle

In practice

  • Apache Kafka: a log-based broker; its idempotent producer (default since Kafka 3.0) removes duplicates from producer retries, not consumer reprocessing.
  • RabbitMQ and SQS: queue-based brokers that delete messages once acknowledged.

Pitfalls

  • Assuming exactly-once processing end to end. Delivery is at least once; consumers deduplicate.
  • A consumer that republishes messages and drops fields it doesn't understand.

Recall

These cards are also in your review deck, scheduled with spaced repetition.