Loading…

Design a Rate Limiter - SystemRound

Design a Rate Limiter

mediumSystem design45 min5 stages

Asked atGoogleAmazonStripeCloudflareMeta

Design a rate limiting system that controls the rate of requests a client can send to an API. The rate limiter should support different rate limiting algorithms, work in a distributed environment, and provide clear feedback to clients when they are throttled. Consider how this would work as both a standalone service and as middleware.

Best once you can already structure a clean answer and want sharper trade-off pressure.

What this problem tests

Rate Limit Check InterfaceConfiguration APIError ResponsesRate Limiting AlgorithmsArchitecture ComponentsDistributed Design

Round shape

5 stages

Time budget

45 min

Feedback loop

Grade anytime

Guided practice·Primary loop

Guided practice

Workspace-first, hints visible, stage retry available. The cheap, repeatable loop — build the answer shape before you take it under pressure.

Stage-by-stage workspace instead of a blank page.
Grade one stage or the whole answer whenever you want.
Compare your reasoning against reference criteria and model answers.

Solve once, compare against the checklist, then come back to the weak stage instead of starting over.

Mock interview·Pressure test

Mock interview

Strict timer, hints hidden, debrief deferred to the end. Use this once you can already structure a clean answer and want to pressure-test pacing and pushback.

Best once the answer shape is already in your head.
Pressure-test pacing, pushback handling, and communication.
Use diagnosis after the interview for exact misses and next study steps.

Best after one structured rep · timed · focused on pacing and communication.

Requirements

This is the framing pass. A strong answer quickly defines what the system must do, what quality bar it has to hit, and the numbers that will justify the rest of the design.

First 5 min of the round

What must exist

Functional Requirements

6 items

1Limit requests per client (identified by API key, user ID, or IP)

2Support different rate limits per API endpoint (e.g., /search: 10/sec, /upload: 5/min)

3Support tiered limits based on user plan (Free: 100/hour, Pro: 1000/hour)

4Return HTTP 429 Too Many Requests with Retry-After header and remaining quota

5Support multiple time windows simultaneously (100/minute AND 1000/hour)

6Both hard limiting (strict reject) and soft limiting (allow with degraded response)

What good looks like

Non-Functional Requirements

6 items

1Ultra-low latency: p99 decision latency under 1ms in-region (rate limiter is on the hot path)

2Highly available: if rate limiter crashes, fail OPEN (let traffic through) — better to over-serve than block legitimate users

3Distributed: multiple API servers share a single rate limit counter

4Accurate: minimal race conditions, but small inaccuracy is acceptable

5Memory efficient: handle millions of clients without excessive memory

6Scalable: handle 1M+ requests per second across the fleet

Numbers to anchor the design

Scale Estimation

4 items

11M+ requests/sec across the fleet at peak — every request needs a sub-millisecond allow/deny decision.

2Millions of distinct clients (keys) tracked at once; per-key state is just a few small counters, so memory scales with client cardinality, not request volume.

3A single Redis node tops out around 100K–1M ops/sec, so the counter store shards by key — on the order of 10+ nodes at 1M/sec.

4Counters live in an in-memory store (Redis) in-region because the decision must be p99 < 1ms; cross-region sync is async, not on the hot path.

What to cover

Placement

1 item

1I'd recommend implementing it as API Gateway middleware (like Kong, Envoy). This catches traffic before it hits application servers, reducing load. However, for fine-grained, application-specific limits, a second layer at the app level is needed. Client-side rate limiting is unreliable (clients can bypass it).

How the round unfolds

Each stage has a distinct job. Treat them like separate deliverables instead of one giant answer, and the round becomes much easier to navigate.

4 design stages · 40 pts after framing

🔌

Stage 2~5 min10 pts

API Design

Define the contract clearly: the endpoints, auth boundary, error semantics, and the one or two decisions that matter most.

What you should produce

How would the rate limiter integrate with existing APIs?

Strong answers cover

Rate Limit Check InterfaceConfiguration APIError Responses

🏗️

Stage 3~10 min10 pts

High-Level Architecture

Lay out the main components and trace the write path, read path, and any async path cleanly.

What you should walk through

Walk me through the high-level architecture.

Strong answers cover

Rate Limiting AlgorithmsArchitecture ComponentsDistributed Design

💾

Stage 4~10 min10 pts

Data Model & Storage

Pick the store, show the schema or key model, and explain why that storage choice fits the access pattern.

What you should lock in

Let's discuss how you'd store rate limit state.

Strong answers cover

Redis Data StructuresAtomic OperationsMemory Estimation

📈

Stage 5~15 min10 pts

Scaling & Deep Dive

Name the first bottleneck, failure modes, and the trade-offs that keep the system fast and reliable under pressure.

What you should pressure-test

Let's scale this system. How would you handle millions of requests per second? What about edge cases like distributed clock skew, hot keys, and gra...

Strong answers cover

Scaling StrategiesEdge CasesMonitoring & Observability

What a strong first rep looks like

Scope clearly

Translate the prompt into concrete requirements, scale, and trade-offs before drawing architecture.

Stay stage-specific

Give APIs in the API stage, data models in the storage stage, and failure modes in scaling. Don't blur them together.

Iterate fast

Grade early, compare to the reference reasoning criteria, fix the biggest misses, and re-submit the weak stage instead of starting over.