The short answer
Quick answer: A messaging app like WhatsApp keeps a persistent connection open between each phone and a chat server. When you send a message, your phone encrypts it and sends it over that connection. The server looks up which server the recipient is connected to and forwards the message. If the recipient is offline, the message waits in a queue and a push notification wakes their device. Acknowledgements flow back at each stage to produce the sent, delivered and read ticks. With end-to-end encryption, only the participants' devices hold the keys, so the servers relay messages they cannot read.
This article describes the general architecture of large chat systems, drawing on what WhatsApp and Signal have said publicly. The exact internals of any one company's system are not public and change over time.
The requirements
- Deliver one-to-one and group messages in well under a second.
- Never lose a message, and never deliver one twice.
- Keep messages in order within a conversation.
- Work when the recipient is offline and across flaky mobile networks.
- Show online status and delivery receipts.
- Keep message content private from the service itself.
- Do all of this for billions of users and tens of billions of messages a day.
Persistent connections
HTTP's request-response model means the server cannot speak first. A chat app needs the server to push a message the instant it arrives.
So each client opens one long-lived, encrypted TCP connection to a gateway (or chat) server and keeps it open. WhatsApp uses its own compact protocol, originally derived from XMPP; web-based chat apps typically use WebSockets. The principle is identical: both sides can send at any time, and each message is a small frame instead of a full HTTP request.
Holding hundreds of millions of mostly idle connections is the first scaling challenge. WhatsApp is well known for building its backend on Erlang, a language designed for telecom switches, where each connection is handled by an extremely lightweight process. The company has publicly described handling millions of connections per server. Phones send periodic heartbeats so that dead connections are noticed and cleaned up.
Sending a one-to-one message
When Alice sends "hello" to Bob:
- Encrypt. Alice's phone encrypts the message for Bob's device(s).
- Send. It goes over Alice's connection to her gateway server, tagged with a unique message ID.
- Acknowledge. The server stores the message and replies "received". Alice sees one tick.
- Route. A session registry, an in-memory store mapping each user to the server holding their connection, tells the system where Bob is.
- Forward. The message is passed to Bob's gateway server and pushed down his connection.
- Delivery receipt. Bob's phone acknowledges. The server tells Alice, who sees two ticks.
- Read receipt. When Bob opens the chat, his phone sends a read receipt, and Alice's ticks turn blue.
Each hop is acknowledged, and unacknowledged messages are retried. Retries can cause duplicates, so the receiver deduplicates by message ID. Retry plus deduplication is how systems achieve "exactly once" behaviour in practice; see why distributed systems are hard.
When the recipient is offline
If Bob has no active connection:
- The server keeps the encrypted message in Bob's offline queue.
- It asks Apple's or Google's push service to send a push notification, which wakes the app. See how notification systems work.
- When Bob's phone reconnects, the server delivers everything in the queue, in order.
- Once Bob's device acknowledges, the server deletes its copy.
WhatsApp has said that messages are removed from its servers once delivered, and that undelivered messages are kept only for a limited time. The conversation history lives on the devices. That choice keeps server storage small and supports the privacy model. Other apps, such as Telegram's default chats or Slack, store history on the server instead, which makes multi-device sync easier.
Ordering
Networks can reorder messages. Chat systems usually assign each message a sequence number or server timestamp per conversation, and clients display messages in that order. Clocks on different phones cannot be trusted for this; see why clocks can't be trusted in distributed systems.
Group messages
A group message must reach every member. Conceptually, the server fans out the message, placing a copy in each member's delivery queue and handling online and offline members as above.
Encryption makes groups more interesting. Encrypting separately for every member would be expensive for large groups. The Signal protocol's approach, which WhatsApp has adopted, uses sender keys: each member generates a key for sending to the group and shares it with the other members over their one-to-one encrypted channels. A group message is then encrypted once and relayed to everyone.
Group size limits exist partly because fan-out cost grows with membership.
End-to-end encryption
With end-to-end encryption, the server never has the keys. WhatsApp uses the Signal protocol, whose specifications are published in the Signal documentation. The main ideas:
- Identity keys. Each device generates a long-term key pair. The private key never leaves the device. See how public key cryptography works.
- Prekeys. Each device uploads a batch of public "prekeys" to the server. This lets Alice set up an encrypted session with Bob even when Bob is offline.
- Double ratchet. Keys change with every message. Compromising one key does not expose earlier messages (forward secrecy), and the session recovers its security after a compromise.
- Verification. Users can compare security codes or scan a QR code to confirm they are not talking to an impostor.
The server still sees metadata: who is messaging whom, when, and roughly how much. Encryption protects content, not the fact of communication.
Media
Photos and videos do not travel through the chat connection.
- The sender encrypts the file with a random key.
- It uploads the encrypted blob to media storage over HTTP.
- The chat message carries only a pointer to the blob, the key and a hash.
- The recipient downloads and decrypts it.
Large files therefore never clog the messaging path, and delivery of media can use CDNs.
Presence and typing indicators
"Online", "last seen" and "typing..." are ephemeral signals. They are sent only to people who currently have the relevant chat or contact visible, they are not stored, and it does not matter if one is lost. Treating them differently from messages keeps a very chatty feature cheap.
Scaling the backend
- Gateway servers hold connections and are scaled out by the thousand.
- The session registry is a fast distributed store, updated on every connect and disconnect.
- Message queues between components smooth out bursts and survive server failures. See how message queues work.
- Data is partitioned by user, so each user's queue lives on a known set of machines.
- Multiple data centres and reconnection logic let clients fail over when a server or region has trouble.
Frequently asked questions
What do the ticks mean in WhatsApp?
One grey tick: the server received your message. Two grey ticks: it reached the recipient's device. Two blue ticks: the recipient opened it (if read receipts are enabled).
Can WhatsApp read my messages?
With end-to-end encryption, message content is encrypted on your device with keys the server does not have. The service can still see metadata such as who you contact and when.
What happens to messages sent while my phone is off?
They wait, encrypted, on the server and are delivered when your phone reconnects. They are deleted from the server after delivery or after a retention period.
Why do chat apps use persistent connections instead of polling?
A persistent connection lets the server push messages instantly and avoids the overhead of repeated requests, which matters for battery life and latency.
Conclusion
A chat system is a relay built from a few robust ideas: persistent connections, a registry of who is connected where, queues for the offline, acknowledgements with retries, and encryption that keeps the relay blind. The scale is extraordinary, but the building blocks are the same ones you would use for a chat feature of any size.
Related articles
- How WebSockets Enable Real-Time Apps Like Chat and Live Scores
- How Message Queues Like Kafka Decouple Systems
- How Notification Systems Send Millions of Push Alerts
- How Public Key Cryptography Works
Sources and further reading
