Fan-out: on write vs on read

Path assignment is the most important tuning parameter in a hybrid fan-out system. Set the push boundary too low and feed reads pull too many author timelines. Set it too high and high-follower posts storm the fan-out queue. Treat the boundary as a cost model:

Approximate the decision as: push cost ≈ active_followers × post_rate, bounded by fan-out capacity and freshness SLO. Pull cost ≈ follower_feed_reads × merge_cost, bounded by the number of pull-path authors each reader follows. Push while the write work fits the queue budget and saves more read work than it creates; pull when write amplification would dominate capacity.

Static thresholds break during viral moments. A user may gain followers quickly, post more often, or attract more active readers. If the threshold check happens only at post time, the next post can suddenly create a much larger fan-out than the queue budget expected. The reverse is also a problem: an author oscillates around the boundary, and some posts are pushed while others are pulled, creating inconsistent delivery and operational noise.

Use hysteresis and cooldowns so authors do not bounce between paths. Promote to pull only after the model stays above the boundary for several windows, and demote only after sustained lower cost. Evaluate at post time for routing, but update model inputs continuously from follower activity, queue depth, post rate, and read latency.

The storm diagram shows what happens without a boundary: one high-follower post expands into a huge number of inbox writes, saturates the queue, and delays unrelated delivery. With a measured boundary, that post goes to the author's timeline and is pulled by followers when they read, while ordinary authors keep the cheap pre-materialised path.

The inbox is the core data structure of fan-out on write. Each user's inbox is a Redis sorted set (ZSET) where members are post ids and scores are timestamps (or snowflake ids for ordering). The ZSET provides three critical operations:

1. 1ZADD — insert a post id with a score. O(log N) where N is the set size. With bounded inboxes (N ≤ 800), this is effectively O(1).
2. 2ZREVRANGE — read the top K entries by descending score. O(K + log N). For a feed page of 50 items: sub-millisecond.
3. 3ZREMRANGEBYRANK — trim the set to keep only the top N entries. Called after ZADD to enforce the size bound.

The fan-out worker pipelines these operations for efficiency. A typical batch: ZADD inbox:{user_id} {timestamp} {post_id} for each of 50 follower inboxes in a single Redis pipeline, followed by ZREMRANGEBYRANK inbox:{user_id} 0 -(max_size+1) to trim. The pipeline completes in under 1ms for 50 operations.

Memory budget: each ZSET entry costs ~80 bytes (64-byte member + 8-byte score + 8-byte overhead). An inbox of 800 entries costs ~64KB. For 200M users: 200M × 64KB = 12.8TB. This is the primary cost driver and the reason inboxes must be bounded. Without the bound, memory grows linearly with post volume × follower count, which is unsustainable.

Deep scroll (page 2+) works by continuing the ZREVRANGE with an offset. Beyond the inbox's 800 entries, the feed service falls back to pulling from author timelines — switching from push to pull for historical content. This is acceptable because deep scroll is rare (<5% of feed reads go past page 1) and latency expectations are relaxed.

TTL is an additional lever: set a 7-day TTL on inbox keys so that users who haven't opened the app in a week don't consume memory. When they return, the feed service detects the empty/expired inbox and does a full pull-based reconstruction, then resumes normal push delivery.

In the hybrid model, feed reads merge two data sources: the pre-materialised inbox (from push) and K celebrity timelines (from pull). The merge is a K-way merge of sorted streams, where K = 1 (inbox) + number of celebrities the user follows.

The algorithm uses a min-heap of size K+1. Initialise the heap with the most recent entry from each stream. To produce the next feed item, extract the max from the heap, then insert the next entry from that item's source stream. Repeat until you have a full page (typically 50 items).

Time complexity: O(N log K) where N is the page size and K is the number of streams. For a typical user following 10 celebrities: O(50 × log 11) ≈ 170 comparisons — trivial.

The expensive part isn't the merge — it's fetching the celebrity timelines. Each timeline read is a ZREVRANGE against a Redis key or a database query. With 10 celebrity pulls, and each taking 2-5ms (cached) or 20-50ms (cache miss), the total pull latency is 2-50ms depending on cache hit rates. Celebrity timelines are hot (millions of readers access them) so cache hit rates exceed 99% in practice.

Caching the merged result is tempting but tricky. The merged feed is personalised (different users follow different celebrities), so you can't share it across users. You can cache it per-user with a short TTL (30-60 seconds) to handle rapid pull-to-refresh. The cache key includes the user id and a version number that increments on each inbox write, ensuring the user sees new posts within one refresh cycle.

For the ranking pipeline, the merge step produces a candidate set of ~100-200 items (50 from inbox + 10-15 × 10 from celebrity timelines). This candidate set is then scored and re-ranked by the ranking model, which may reorder items based on engagement prediction, recency decay, and interest-graph affinity. The merge-K step is therefore a pre-filter, not the final ordering.

The hybrid boundary routes the most expensive authors to pull. But what about mid-tier high-follower authors who are still on the push path? A single post from a large account can generate hundreds of thousands of inbox writes. If several such authors post simultaneously, the fan-out queue can still spike.

Spread fan-out addresses this by chunking the follower list and scheduling chunks over time. Instead of enqueuing a single "fan out to 500K followers" task, the fan-out scheduler splits the follower list into chunks of 50K and schedules them with staggered delays: chunk 1 at t=0, chunk 2 at t=10s, chunk 3 at t=20s, and so on. The fan-out workers drain at a steady rate, never spiking above their provisioned throughput.

The trade-off is fan-out latency. A large post now reaches followers over a controlled window instead of all at once. For many feeds this is acceptable; users rarely notice whether a post appears a few seconds later. The most engaged or recently active followers can be placed in the first chunks if freshness matters.

Implementation details: the fan-out scheduler stores chunk metadata in a delayed message queue (e.g., SQS with delay seconds, or a Redis sorted set used as a delay queue). Each chunk message contains the post id, the follower list offset, and the chunk size. Workers process chunks idempotently — if a chunk is retried, the ZADD operations are naturally idempotent (same member + same score = no-op).

Rate limiting fan-out workers is the complementary control. Each worker has a configurable writes-per-second limit (e.g., 50K ZADD/sec). If the queue depth exceeds a threshold, the scheduler automatically increases the inter-chunk delay for new posts, providing backpressure without dropping fan-out tasks. The system self-regulates: during quiet periods, fan-out is near-instant; during peak load, it spreads gracefully.

The follow graph is not static. Users follow and unfollow accounts constantly. Each event has implications for the fan-out system that are easy to overlook in a design interview.

On follow: When user A follows user B, A's inbox should contain B's recent posts. Two approaches: (1) Backfill — a background worker fetches B's last N posts and ZADDs them into A's inbox. This provides immediate gratification but adds write load proportional to the follow rate. (2) Lazy — don't backfill; A will start seeing B's posts on the next fan-out. This is simpler but means A's feed looks sparse right after following someone. Most production systems use backfill with a small N (e.g., 10 posts).

On unfollow: When user A unfollows user B, B's posts should disappear from A's feed. Two approaches: (1) Eager deletion — scan A's inbox and remove all post ids authored by B. This requires a secondary index (post_id → author_id) and is expensive for large inboxes. (2) Lazy filter — leave B's posts in A's inbox but filter them out at read time by checking the follow graph. This is cheaper but means stale entries consume inbox space.

The production answer is lazy filter + nightly sweep. At read time, the feed service checks each candidate post's author against the reader's current follow set (cached) and filters out unfollowed authors. A nightly batch job scans inboxes and removes entries from unfollowed authors, reclaiming space. The lazy filter ensures correctness immediately; the nightly sweep ensures space efficiency eventually.

Edge case: A follows B, B posts, A unfollows B, A re-follows B. With lazy filter, B's posts are filtered out during the unfollow window and reappear on re-follow (because the filter checks current follow state). With eager deletion, the posts are gone and would need backfill on re-follow. The lazy approach handles this naturally.

Another subtlety: if B is on the pull path, follow/unfollow doesn't affect A's inbox at all (B's posts were never pushed). The follow graph update only affects the pull list used at read time — the feed service checks which pull-path authors to read from based on A's current follow set.

A modern feed is not chronological — it is ranked. The ranking pipeline transforms a raw candidate set into a personalised, engaging feed. In the hybrid fan-out model, the pipeline sits between the merge step and the final response.

Stage 1: Candidate generation. The merge-K step produces 100-200 candidates from the inbox (push) and celebrity timelines (pull). This is the initial candidate pool.

Stage 2: Feature enrichment. Each candidate is annotated with features from the feature store: post age, author engagement rate, media type, number of likes/retweets in the first 5 minutes (early engagement signal), the reader's historical interaction rate with this author, topic embeddings, and recency decay. The feature store is typically a low-latency key-value store (Redis or a purpose-built feature service) pre-computed by offline pipelines.

Stage 3: Scoring. A lightweight ML model (typically a gradient-boosted tree or a small neural network) scores each candidate. The model predicts P(engagement) — the probability the reader will like, comment, or share the post. Training data comes from historical engagement logs. Inference must complete in <10ms for the full candidate set, which constrains model complexity.

Stage 4: Diversity filtering. A ranked list of the top 50 by score goes through a diversity filter that ensures no single author dominates the feed (e.g., max 3 posts per author in a single page), mixes content types (text, image, video), and injects exploration candidates (posts from authors the user hasn't interacted with recently) to avoid filter bubbles.

Stage 5: Materialisation. The final ranked page is cached per-user with a 60-second TTL. Subsequent feed reads within the TTL window return the cached result. On pull-to-refresh, the cache is invalidated and the pipeline runs again.

The ranking pipeline adds 20-50ms to feed read latency (on top of the ~10ms for inbox read + merge). The total feed read latency budget is typically 50-100ms. Keeping the pipeline within budget requires aggressive pre-computation (features in the feature store, not computed at serving time), model simplicity (inference must be <10ms), and parallelism (feature fetch and model inference can overlap for different candidates).

Raffi Krikorian's QCon talk "Timelines at Scale" is the classic public reference for Twitter-style timelines. The talk summary describes handling thousands of events per second: tweets, social-graph mutations, and direct messages.

The architectural lesson is the push/pull split. For most authors, Twitter could materialise home timelines ahead of reads by copying tweet ids into follower timeline storage. For high-follower accounts, write-time fan-out becomes the expensive path; their tweets are better stored once and merged into a reader's home timeline when that reader asks for it.

The talk is useful because it names the real constraint: power-law reach means the average author and the outlier author are different workloads. It does not give a universal threshold to copy. Derive your own boundary from active followers, post rate, feed-read rate, fan-out capacity, and freshness SLO.

Key technical details to carry forward: materialised inboxes must be bounded, writes must be idempotent, high-follower content needs a pull path, and feed reads need a merge/filter/rank step that can enforce current blocks, mutes, privacy, deletes, and moderation state.

Instagram is useful here as a public example of the feed moving from pure recency toward ranking. Its official ranking explainer says Feed and Stories start from recent posts by accounts you follow, then use signals about the post, the person who posted, your activity and your interaction history to predict which posts matter most.

That explainer is about ranking; Instagram has not published its fan-out thresholds or storage internals in it. It still teaches the key interaction: fan-out produces candidates, ranking decides order. A pre-materialised inbox is valuable because it gives the ranking system a fast candidate set; it is not the final feed order once personalisation is involved.

The interview move is to keep these concerns separate. First choose how candidates arrive cheaply (push, pull, or hybrid). Then describe how ranking filters and orders them using product-specific features.

Illustrative example: a professional network mixes structurally different content types — posts, job changes, endorsements, work anniversaries, article shares — each with different fan-out characteristics.

If the primary social graph is mutual and product-bounded, push can dominate for connection-sourced content because every write has a known ceiling. If the same product also supports unbounded follows for companies, creators, or topics, those sources may need the pull path because their reach is no longer bounded by the connection graph.

The ranking pipeline then handles heterogeneous candidates. A job-change notification, a long-form article, and a short post have different engagement signals and different time decay. Fan-out decides how candidates arrive; ranking decides whether they are relevant enough to show.

This is a useful interview pattern because it prevents over-generalising from open social graphs. Sometimes the product model itself gives you a safe push boundary. Sometimes a second relationship type reintroduces the high-follower problem.