Skip to content

Pub/Sub vs Message Queue: Diagrams, Delivery Semantics and When to Use Each

A queue gives each message to one consumer; pub/sub gives every subscriber a copy. Delivery guarantees, ordering, replay, and how Kafka, RabbitMQ, SQS and Pub/Sub map.

· Founder & developer

Published

Two lanes. Top: Upload API sends m1 to m3 to the resize-jobs queue, and three resize workers each receive one message. Bottom: Order service publishes m1 to m3 to the order-events topic, which copies them into three subscriptions that feed three services.

Your email service sends order receipts. You scale it to three replicas, and customers start getting three receipts per order. Each replica subscribed with its own consumer group, so each received its own copy of every message. The opposite mistake is as common: resize jobs go onto a broadcast topic, a second worker joins, and every image is resized twice.

Both bugs mix up two messaging models: a message queue, where each message goes to one consumer, and publish-subscribe, where every subscriber gets its own copy. This post compares them with a diagram, then covers delivery guarantees, ordering, replay, fan-out and dead-letter queues, and how Kafka, RabbitMQ, SQS/SNS and Pub/Sub map onto each model.

Queue vs topic: who gets each message

In the point-to-point model, a producer sends a message to a queue and one consumer receives it. Consumers on a queue compete: the broker hands each message to one of them and deletes it once that consumer acknowledges it. They are replicas of one service, so adding replicas drains the queue faster. This competing consumers pattern fits work that must happen once, such as resizing an image.

In publish-subscribe, a publisher sends a message to a topic, and the topic delivers a copy to every subscription. Subscribers are different services that do not know about each other, and adding one does not change the publisher.

Most systems combine the two: a topic fans out to subscriptions, and each subscription is a queue that one service's replicas compete on. The design question is where fan-out and competition happen.

Message queue (point-to-point) Publish-subscribe
Consumers per message One One per subscription
Consumers are Replicas of one service Different services
Scale by Adding consumers Adding consumers to a subscription
Retention Until acknowledged or expired Per subscription, or a time window in a log
Replay No Yes, if the broker keeps a log
Typical use Jobs and commands Events and notifications

How Kafka, RabbitMQ, SQS/SNS and Pub/Sub map onto the models

Each broker implements both models:

System Fan-out (pub/sub) Competing consumers (queue) Ordering Replay
Apache Kafka Each consumer group reads every record Consumers in a group split partitions; share groups split records Per partition Reset group offsets
RabbitMQ Exchanges copy messages to bound queues Consumers on one queue Per queue, one consumer Streams only
Amazon SNS + SQS SNS copies to each subscribed queue Consumers on one SQS queue FIFO, per message group SNS FIFO topics only
Google Cloud Pub/Sub Each subscription gets every message Clients on one subscription Ordering keys Seek

Kafka. Each partition is consumed by exactly one consumer within each subscribing consumer group (design docs), so a group never has more active consumers than partitions. Kafka 4.2 (February 2026) made share groups (KIP-932, Queues for Kafka) production-ready: consumers share records from the same partitions and acknowledge each one, without the partition limit or ordering.

SNS to SQS. SNS copies each message into one SQS queue per subscribing service, in a JSON envelope unless you enable raw message delivery (AWS fan-out guide). Each service's workers compete on their own queue.

RabbitMQ. Producers publish to an exchange, and bindings decide which queues get a copy: fanout copies to every bound queue, topic matches patterns such as order.#, direct matches the key exactly, and headers matches message headers (exchange types).

Google Pub/Sub. One subscription per service; its subscriber clients share the messages.

The three-receipts bug is a subscription per replica instead of per service. Name subscriptions, queues and group IDs after the service (billing-sub, group.id=billing), never after a host or pod.

Delivery guarantees

Brokers redeliver unacknowledged messages, so the question is when your consumer acknowledges:

  • At most once: acknowledge before processing. A crash loses the message, and nothing is redelivered.
  • At least once: acknowledge after processing. A crash causes a redelivery, so a message can be processed twice. SQS standard queues and Pub/Sub both deliver at least once by default.
  • Exactly once: each message processed once and only once, which brokers offer only within limits.

SQS FIFO queues drop a resend with the same deduplication ID within 5 minutes. Google Pub/Sub's exactly-once delivery stops redelivery after a successful acknowledgment, for pull subscriptions in one region. Kafka transactions cover reads and writes inside Kafka. None of them stops a consumer from charging a card twice if it crashes between the charge and the acknowledgment.

The practical answer is effectively-once processing: at-least-once delivery plus an idempotent consumer that records each message ID in the side effect's transaction:

BEGIN;
INSERT INTO processed_messages (consumer, message_id)
VALUES ('billing', 'm2')
ON CONFLICT DO NOTHING;
-- 0 rows: duplicate, skip the side effect
-- 1 row: apply the side effect in this transaction
COMMIT;
-- acknowledge the message only after the commit

On the producing side, the transactional outbox pattern keeps the database write and the publish consistent.

Ordering

Competing consumers break first-in, first-out: m1 goes to worker 1, m2 to worker 2, and m2 can finish first. SQS standard queues promise only best-effort ordering anyway. Strict order needs a scope, with one consumer at a time per scope:

  • SQS FIFO: order holds within a message group (MessageGroupId), and groups run in parallel. Throughput is 300 API calls per second per action without batching, 3,000 messages per second with batches of 10, and more in high-throughput mode.
  • Kafka: records with the same key land in the same partition, which one consumer per group reads.
  • Google Pub/Sub: messages with the same ordering key, published in one region, arrive in order on a subscription with ordering enabled. Each key is capped at 1 MBps of publishing, and a redelivery also redelivers the key's later messages.

Order per account or device, never globally: a global order means one consumer and a throughput ceiling.

Retention and replay: log versus queue

A queue is a to-do list: acknowledged messages are deleted, and SQS expires unconsumed ones after 4 days by default (60 seconds to 14 days). A log is a history: Kafka keeps records for its retention period, read or not (log.retention.hours defaults to 168, seven days), and each consumer group stores only an offset, so a new group can start from the oldest record and an existing one can rewind. RabbitMQ streams are logs too, read from a start point set by x-stream-offset. Google Pub/Sub sits between: unacknowledged messages stay 7 days by default (10 minutes to 31 days), acknowledged ones optionally, and subscriptions can seek back.

Replaying a Kafka group after a bad deploy, with its consumers stopped:

kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --group billing --topic order-events \
  --reset-offsets --to-datetime 2026-11-28T00:00:00.000 --execute

Replay rebuilds CQRS read models and backfills new services, so every consumer must tolerate reprocessing. The Kafka architecture diagram covers partitions and offsets in depth.

Back-pressure, visibility timeouts and dead-letter queues

Queues absorb bursts because consumers take work at their own pace, and per-channel queues in a notification system keep a slow SMS provider from holding up email and push. RabbitMQ pushes messages to consumers, so cap unacknowledged messages per consumer with a basic.qos prefetch limit.

While a consumer works, the broker hides the message from the others for a limited time, much like the leases in a job scheduler. If processing runs longer, the message reappears and a second consumer gets it too. Set the timeout above your slowest normal processing time, or extend it:

Broker Lease while processing Extend with Stop retrying after
Amazon SQS Visibility timeout: 30 s default, max 12 h ChangeMessageVisibility maxReceiveCount, then a dead-letter queue
Google Pub/Sub Ack deadline: 10 s default, max 600 s ModifyAckDeadline 5 attempts by default (up to 100), then a dead-letter topic
Kafka share groups Acquisition lock: 30 s default RENEW acknowledgment 5 attempts by default, then the record is archived; Kafka 4.4 can also write it to a dead-letter topic

RabbitMQ redelivers when a consumer's channel closes before it acknowledges, and dead-letters through dead-letter exchanges.

Poison messages, ones that always fail, belong in a dead-letter queue: alert on its depth and keep a redrive procedure. In pub/sub, failures stay isolated: a broken analytics consumer grows only its own backlog, so alert on the oldest unacknowledged message per subscription.

Choosing by use case

Use case Model Good fits
Background jobs: thumbnails, PDFs, email Queue SQS standard, a RabbitMQ quorum queue, a Kafka share group
Commands in order per account Queue with an ordering scope SQS FIFO grouped by account, Kafka keyed by account
An event several services react to Pub/sub, one queue per subscriber SNS to SQS, a RabbitMQ fanout exchange, Pub/Sub subscriptions, Kafka consumer groups
Rebuilding state or backfilling a consumer Log Kafka, RabbitMQ streams
A multi-step process across services Commands on queues, outcomes on pub/sub The saga pattern

The queue and pub/sub diagram

Two lanes. Top: Upload API sends m1 to m3 to the resize-jobs queue, and three resize workers each receive one message. Bottom: Order service publishes m1 to m3 to the order-events topic, which copies them into three subscriptions that feed three services.

18 components · 11 connections

Open in Archboard
Diagram contents
  • Upload API
  • resize-jobs
  • Resize worker 1
  • Resize worker 2
  • Resize worker 3
  • Order service
  • order-events
  • email-sub
  • billing-sub
  • analytics-sub
  • Email service
  • Billing service
  • Analytics service
  • send m1, m2, m3
  • m1
  • m2
  • m3
  • publish m1, m2, m3
  • copy
  • m1, m2, m3
A queue and a pub/sub topic side by side. The queue hands m1, m2 and m3 to three competing workers, one message each; the topic copies all three into three subscriptions, so the email, billing and analytics services each process every message.

How to read it:

  • Two lanes, same message IDs. Top: a point-to-point queue. Bottom: publish-subscribe. Each producer sends m1 to m3.
  • Top lane: the resize-jobs queue hands each message to a different worker, all replicas of one service.
  • Bottom lane: the order-events topic copies every message into three subscriptions, one per service, so email, billing and analytics each receive m1, m2 and m3.
  • Solid arrows are send, publish and copy steps; dashed arrows are asynchronous deliveries that consumers acknowledge.
  • The Markdown block lists timeout and retention defaults.

Drawing messaging diagrams clearly

  • Draw subscriptions, not only topics. Arrows from a topic straight to services hide where messages wait and who owns each backlog.
  • Label edges with message names or IDs, not "events": m1, m2 and m3 show splitting versus copying.
  • Show replicas when competition matters, and mark the ordering key and the dead-letter path next to their queue or topic.

The event-driven architecture diagram guide extends this notation to a full Kafka system.

Build it in Archboard

  1. Sign in with Google or GitHub and create a diagram.
  2. Add a Core Text heading for each lane.
  3. Top lane: add a Container from the C4 Model pack named Upload API, a Queue from the Data pack named resize-jobs and three Container nodes, Resize worker 1 to 3. Draw a solid edge "send m1, m2, m3" to the queue and dashed edges m1, m2 and m3 to the workers.
  4. Bottom lane: add a Container named Order service, a Topic from the Data pack named order-events, three Queue nodes (email-sub, billing-sub, analytics-sub) and three Container nodes for the email, billing and analytics services. Label the topic's edges "copy" and the dashed deliveries "m1, m2, m3".
  5. Color each lane with one accent, blue or teal, and add a Core Text note under the queue and the topic.
  6. Add a Markdown block from the Core pack with the defaults.

For a larger system, start from the event-driven architecture template; for flows already in Mermaid, the Mermaid to editable diagram tool makes a first draft.

The example uses 14 diagram nodes, within the Free plan.

FAQ

What is the difference between pub/sub and a message queue? A queue gives each message to one consumer and deletes it after the acknowledgment. Pub/sub gives every subscription its own copy, so several services can react to one event.

Is Kafka a message queue or a pub/sub system? Both. Every consumer group reads every record, which is pub/sub, and consumers in a group split the partitions like a queue. Kafka 4.2 share groups add per-record acknowledgment.

Kafka vs RabbitMQ vs SQS: which should I use? SQS for a managed queue on AWS with nothing to operate, RabbitMQ for flexible routing, and Kafka for high throughput with replayable history.

Do I need SNS if I already use SQS? Only if more than one service needs each message. SNS then copies each message to one SQS queue per service, each with its own retries and dead-letter queue.

Does any message queue guarantee exactly-once delivery? Not end to end. SQS FIFO, Google Pub/Sub and Kafka transactions prevent duplicates within limits, but emails or payments still need an idempotent consumer.

Start free — one seat, no credit card.

Related posts

← Back to blog

Open a canvas in a minute

Sign in with Google or GitHub. Free for a single seat. Upgrade when the team joins.