Chat App Architecture Diagram: WebSockets, Fan-out and Message Storage

A chat app is the system design question that hides the most state. Every user holds a live connection to one specific server, every message has to arrive exactly once and in order, and the same message may need to reach one person or ten thousand. A diagram that shows "client, server, database" answers none of that.

This post walks through a chat backend one decision at a time: how clients connect, how a message gets its place in a conversation, how it reaches everyone who should see it, and how it is stored. It ends with an architecture diagram you can use as a starting point.

Requirements

• One-to-one and group conversations, with text and media.
• Messages arrive in the same order for every member of a conversation.
• No duplicates when a client retries a send.
• Online users get messages within a second; offline users get a push notification and catch up when they reconnect.
• Delivery and read receipts, and an online indicator.

Connections: why WebSockets, and why gateways

Polling for new messages wastes requests and adds delay. A WebSocket connection stays open, so the server can push a message the moment it exists. Each device keeps one connection to a chat gateway.

Gateways are the stateful part of the system. A gateway knows which users are connected to it and nothing else. Keep them thin: authenticate the connection once, receive messages and hand them to a stateless service, and push outgoing messages down the right socket. When you deploy a new gateway version, connections drop and clients reconnect, so clients must reconnect with backoff and jitter and resume from the last message they saw.

Put a layer 4 load balancer in front, or one that understands WebSocket upgrades, and expect long-lived connections to make load uneven. New connections should prefer the least loaded gateway.

Sending a message: idempotency and ordering

The client generates a client_msg_id (a UUID) before it sends. If the connection drops before the acknowledgment arrives, the client sends again with the same ID, and the message service drops the duplicate. Without it, flaky mobile networks produce double messages.

Ordering is per conversation, not global. The message service assigns each message the next sequence number in its conversation and stores it. Clients sort by that number, never by their own clock. Two practical ways to assign it:

• A counter per conversation, incremented in the same write that stores the message.
• A time-sortable ID, such as a Snowflake-style ID generated by the server, when you can accept that "order" means "order the server saw".

The sequence number also makes catching up cheap. A reconnecting client says "I have conversation 42 up to 1871" and gets everything after it.

Fan-out: online, offline and big groups

After the message is stored, the service publishes an event. Fan-out workers take it from there:

1. Look up the conversation's members.
2. Look up which members are online and which gateway holds their connection, in a presence and session registry such as Redis.
3. Send the message to those gateways, which push it down the sockets.
4. Send push notifications through APNs and FCM to members who are offline.

Writing the message once and fanning out reads keeps storage linear in the number of messages, not members. Large groups need care: a channel with 100,000 members should not trigger 100,000 push notifications per message. Notify only members who opted in, or batch and summarize.

Presence

Presence is a heartbeat with a TTL. The gateway refreshes "user 7 is connected to gateway 3" every 30 seconds or so; when the key expires, the user is offline. Do not broadcast every presence change to every contact in real time. It is one of the most expensive features in chat for how little users notice delays in it. Fetch presence when a conversation opens and refresh it periodically. Expiring keys work the same way as cache entries; the caching strategies post covers TTLs and what happens when they all expire at once.

Receipts and end-to-end encryption

Delivery and read receipts are messages too. When a device receives message 1871 it sends a small "delivered up to 1871" event; when the user opens the conversation it sends "read up to 1871". Store the highest number per member and conversation instead of one receipt per message, and the cost stays flat in busy groups.

End-to-end encryption changes what the server can see. With a protocol such as Signal's, the server stores and forwards ciphertext, so search, moderation and link previews move to the clients. Decide early, because it changes both the storage and the fan-out.

Storage

Messages are written once, read mostly recently, and almost always read by conversation. That points to a store partitioned by conversation and sorted by sequence number. Discord, for example, partitions messages by channel plus a fixed time bucket so one busy channel does not become one enormous partition; its post on how it stores trillions of messages describes moving that model from Cassandra to ScyllaDB.

Keep media out of the message store. Clients upload images and video straight to object storage with a pre-signed URL, and the message carries a reference. The message service never handles the bytes.

The architecture diagram

How to read it:

• Left: users connect over WebSocket (wss) through the load balancer to a chat gateway. Media uploads go straight to the bucket.
• Middle: the gateway hands each send to the message service, which drops duplicates by client_msg_id, assigns the next sequence number and appends the message to the store.
• Right: the service publishes an event. Fan-out workers check presence and sessions, deliver through the right gateway, and send offline members a push notification.
• The heartbeat from gateway to presence is dotted because it runs on a timer, not per message.
• The Markdown block shows the message envelope, so everyone agrees on the fields.

What to label

• Which components hold state. Gateways hold connections; the message service holds nothing. That decides how you deploy and scale each one.
• The ordering rule. "Sequence per conversation, assigned by the server" belongs on the diagram.
• The dedupe key. Name client_msg_id on the send path.
• Online and offline paths. Draw both; most bugs live in the second.

The event-driven architecture post covers the event and consumer side in more depth, and the sequence diagram example is a good way to document the send and acknowledgment exchange step by step.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. Add a Person from the C4 Model pack for the users and a Cloud LB from the Cloud pack.
3. Add three Container nodes from the C4 Model pack: chat gateway, message service and fan-out workers. Note in each whether it is stateful.
4. Add a Database from the C4 Model pack for the message store, Redis and Kafka from the Data pack for presence and events, an Object Storage node from the Cloud pack for media, and an External System for APNs and FCM.
5. Connect them, draw the heartbeat and the media upload as dashed or dotted edges, and add a Markdown block from the Core pack with the message envelope.

The example uses 11 diagram nodes, within the Free plan.

FAQ

Do I need WebSockets for a chat app? For real-time delivery, yes, or a similar push channel such as server-sent events plus HTTP for sends. Long polling works for small apps but costs more requests and adds delay.

How do I keep messages in order? Let the server assign a sequence number per conversation when it stores the message, and have clients sort by it. Client clocks disagree and cannot be trusted for ordering.

How do I avoid duplicate messages? The client generates a message ID before sending and reuses it on retries. The server stores the ID and ignores a second message with the same one.

What database should store chat messages? One that reads fast by conversation and sorts by sequence, such as a wide-column store partitioned by conversation and time bucket. Postgres works well for smaller apps with an index on conversation and sequence.

How are messages delivered to users who are offline? They are stored like any other message. The fan-out step sends a push notification, and when the app reconnects it asks for everything after the last sequence number it has.

Start free — one seat, no credit card.