News Feed System Design: Fan-out on Write vs Fan-out on Read

The home feed is the most read page in any social product and one of the most expensive to build. Each user follows hundreds of accounts, each account posts at its own pace, and the feed has to come back in under a few hundred milliseconds, ranked, every time someone opens the app.

The core design question has two classic answers, fan-out on write and fan-out on read, and every large feed ends up with a mix of both. This post compares them with numbers, shows the hybrid, and ends with a diagram of the full read and write paths.

Requirements

• A user sees recent posts from the accounts they follow, ranked or in time order.
• Opening the feed takes well under a second at the 99th percentile.
• A new post shows up in followers' feeds within seconds.
• Infinite scroll without duplicates or gaps.
• It keeps working when one account has tens of millions of followers.

Reads dominate. Twitter reported in 2013 that it served about 300,000 timeline reads per second against roughly 6,000 tweet writes per second, according to this write-up of its architecture. Design for reads first.

Back-of-envelope numbers

The averages look harmless. The outliers do not: one author with 20 million followers creates 20 million fan-out writes with a single post, a tenth of a whole day's fan-out work for everyone else. That outlier is what the design below has to handle.

Fan-out on write (push)

When someone posts, a background job looks up their followers and inserts the post ID at the top of each follower's cached feed. Reading the feed is then one lookup: take the newest IDs from your own list and fetch the posts.

• Strength: reads are fast and predictable.
• Cost: one post creates as many writes as the author has followers. An account with 30 million followers turns one post into 30 million cache writes, and those followers see it late. The same write-up describes posts from the largest accounts taking minutes to reach all followers.
• Waste: you write to feeds of users who will not open the app this week.

Keep each cached feed short. Twitter capped home timelines at 800 entries in Redis at the time; older posts come from the database when someone scrolls that far.

Fan-out on read (pull)

Do nothing when someone posts. When a user opens the feed, fetch recent posts from every account they follow, merge them and rank them.

• Strength: writes are cheap, and big accounts cost nothing extra.
• Cost: every feed read touches hundreds of authors. At 300,000 reads per second, that is a lot of queries for work that is mostly repeated.

The hybrid

Most large feeds combine the two:

• Push posts from ordinary accounts into followers' cached feeds.
• Skip fan-out for accounts above a follower threshold, and pull their recent posts at read time.
• At read time, merge the cached feed with the pulled posts, then rank.

The threshold is a tuning knob, not a law. Set it where the cost of fan-out writes starts to exceed the cost of the extra reads, and revisit it as your follower distribution changes. Users who follow no big accounts never pay for the pull path.

Ranking

A chronological feed sorts by time. A ranked feed works in three steps: gather candidate posts (the merged list above, plus recommendations if you have them), score each candidate with signals such as recency, how often the user interacts with the author, and engagement on the post, then filter (blocked accounts, already seen posts, near duplicates).

Keep ranking out of the fan-out path. Fan-out stores candidates; ranking happens at read time with the newest signals. That way a change to the ranking model needs no backfill of cached feeds.

Deletes, blocks and private accounts

Feeds hold IDs, so deleting a post is one write to the posts store; the feed service drops IDs that no longer resolve. Blocks and accounts that switch to private need the same filter at read time, because removing an ID from millions of cached feeds is slow and easy to get wrong.

Pagination

Never page a feed with offsets. New posts arrive at the top while the user scrolls, so "page 2" shifts and shows duplicates. Return a cursor that encodes the position, such as the ID or score of the last item served, and ask for "items after this cursor". With ranked feeds, store the ranked list for the session for a short time so later pages stay consistent with the first.

The architecture diagram

How to read it:

• Write path (top): the post service stores the post and publishes an event. Fan-out workers read the social graph and check the author's follower count.
• The decision: for most authors, workers push the post ID into each follower's feed cache. For accounts above the threshold, nothing is pushed; their posts are pulled at read time.
• Read path (bottom): the feed service reads the user's cached feed and recent posts from big accounts in the posts store, merges them and sends them to ranking.
• Media goes through a CDN and never through the feed service.
• The Markdown block keeps the rules of thumb next to the design.

What to put on the diagram

• Both paths, clearly separated. Writes and reads have different scaling problems.
• The fan-out threshold. Write the number on the decision, even if it changes later.
• What is stored where. Feed caches hold post IDs, not posts; the posts store holds content.
• Where ranking happens. Readers should see that it runs at read time.

The cache sitting in the middle of this design deserves its own diagram; see the caching strategies post. The event-driven architecture post covers the event and consumer side, and the system design diagram example shows the same conventions on a different system.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. Add a Person from the C4 Model pack, then Container nodes for the post service, fan-out workers, feed service and ranking.
3. Add Database nodes from the C4 Model pack for the posts store and the social graph, Kafka and Redis from the Data pack for post events and feed caches, and a CDN from the Cloud pack.
4. Add a Decision and a Terminator from the Flowchart pack for the follower threshold.
5. Draw the write path on top and the read path below, and add a Markdown block from the Core pack with your rules of thumb.

The example uses 13 diagram nodes, within the Free plan.

FAQ

Is fan-out on write or fan-out on read better? Neither on its own. Fan-out on write gives fast reads but explodes for accounts with huge followings; fan-out on read is cheap to write but slow to read. Large feeds use a hybrid.

What should the feed cache store? Post IDs, newest first, capped at a few hundred per user. Fetch the post content separately, so edits and deletions do not require rewriting every feed.

How do I handle a celebrity posting? Skip fan-out for accounts above a follower threshold and pull their recent posts when followers open the feed.

How should feed pagination work? With a cursor based on the last item served, not with page numbers or offsets, so new posts do not cause duplicates while the user scrolls.

Where does ranking happen? At read time, on the merged list of candidates. Keeping it out of fan-out means you can change the ranking without rebuilding cached feeds.

Start free — one seat, no credit card.