Loading…

Understanding the problem

A request reaches a gateway with an API key for ordinary calls and belongs to one plan plus possibly an endpoint policy. The limiter must answer before backend work starts: allow it, or return a useful response that tells the client when to slow down. The hard parts are shared state across gateways, a counting rule that matches the plan, the last slot, dependency failure, one very large key and one allowance across regions.

The practice brief gives the requirements; confirm them quickly, then spend the interview on where the decision is made and how it fails. The limiter covers API calls, exports and the public sign-in endpoint. It does not replace lower-level DDoS protection or bot detection.

Functional requirements

CoreDecide for every API request whether it may go through, from the customer’s plan and the endpoint called.
CoreEnforce each API key’s plan: a per-minute allowance and a burst size (Free: 100 a minute, bursts of up to 20; Pro: 1,000 and 100; Enterprise per contract).
CoreEnforce endpoint allowances on every plan (creating an export: 10 a minute), and an allowance on the public sign-in endpoint, where callers have no API key yet.
CoreTell a turned-away client when it can try again, and let every client see how much of its allowance is left.
CoreSupport staff can change a customer’s allowance without a deploy; the change applies within about 10 seconds.
Below the lineA usage dashboard for customers.
Below the lineTemporary boosts for a customer’s launch day.They are policy rows in the same control plane once the core limiter is sound.
Below the lineExempting the platform’s own internal traffic.
Below the lineCustomer usage dashboards.They read from logs or aggregates after the request path, not from the limiter’s hot counters.
Below the lineBilling for overages.Billing needs durable usage records; the limiter’s temporary counters are not the ledger.

Non-functional requirements

ConsistencyA key that has been quiet can send a burst of up to its plan’s burst size at once (an app sends 20 requests when it starts), and over any 60 seconds a key never gets more than its allowance plus one burst, whatever the timing.
ConsistencyAn allowance covers the customer’s traffic through every gateway and every region combined.
LatencyThe check adds at most about 2 ms at p99 to a request.
AvailabilityThe API promises 99.99% availability.
ConsistencyBeyond that, a brief overshoot of a few percent from counting across many machines is acceptable; turning away a customer that is under its allowance is worse.
Security

Scale

From the briefNumbers here are given in the practice brief.

ScaleAbout 1 million requests a second at peak across three regions; the busiest region takes about half.
ScaleAbout 500,000 API keys make requests in a busy minute.
ScaleThe largest customer sends up to about 100,000 requests a second at its peak, all under one API key.
ScaleDozens of gateway instances per region; a customer’s requests can land on any of them.

Out of scope

Out of scopeNetwork-level DDoS protection below HTTP.
Out of scopeBot detection and WAF rules.
Out of scopeBilling and invoicing for usage.
Out of scopeWhat the API endpoints themselves do.
Out of scopeClient SDKs.

Check yourself

Why not count every public call by IP address?
Show answerHide answer
Plan allowances belong to the customer’s API key. IP addresses can be shared by many customers or spread across many servers, so IP-only counting both hurts the wrong callers and misses real overuse. Sign-in is the exception because it has no API key yet.

The numbers that shape the design

From the briefNumbers here are given in the practice brief.

Use the brief’s numbers to find the constraint: state fits in memory, while operations per second and one very large key shape the architecture.

Back of the envelope from the brief

Quantity	Value	Design effect
Peak checks	~1M requests/s across three regions	The limiter is on every request, so it must be horizontally sharded.
Busiest region	About half of peak	Plan the regional store for roughly 500k checks/s before replicas and headroom.
Active keys	~500k in a busy minute	Temporary state is modest; idle entries should expire.
Largest customer	~100k requests/s under one key	One key can saturate a store node unless batched or split.
Check budget	~2 ms p99	No cross-region call, no rules database lookup, no retries on the request path.
Policy changes	Apply within ~10 s	A control-plane feed or polling version is enough; deploys are not.

What the numbers tell you

Convert traffic into checks per second, then store operations per check.
Separate memory from operations: temporary entries are small, but every request writes.
Look for skew: the largest key is large enough to need a special path.
Keep the latency budget for the slow path, not just the healthy path.

Architecture diagramWhere the scale lands

Read:Learn: Numbers and capacity estimation Learn: Latency budgeting

Try it: The numbers that shape the designGuided practice · Rate limiter, API platform · Senior

Check yourself

Which number rules out a single shared store node?
Show answerHide answer
Operations per second, especially in the busiest region, plus the largest customer under one key. Memory for active counters is much smaller than the write rate problem.

Core entities and stores

From the briefNumbers here are given in the practice brief.

Model two kinds of data. Policy is durable and audited; counting state is temporary and updated on every request. Keeping them apart lets gateways answer quickly while support changes a plan without a deploy.

The durable policy model

Entity	Key fields	Why it exists
Customer	`id`, `plan_id`	The account that owns the API key and its base allowance.
API key	`key_id`, `customer_id`, status	The subject for ordinary API calls.
Plan	`id`, plan allowance, burst	Free, Pro and contract-specific limits.
Endpoint policy	`endpoint_group`, allowance, burst, cost	Exports and other expensive endpoints can be stricter than the plan.
Override	`customer_id`, `endpoint_group`, allowance, burst	Enterprise contracts and support changes, with audit.

The temporary counting model

Entry	Holds	Lifecycle
`rl:{key}:{policy}`	Bucket level and last update time, or rolling counts	Created on first use and expires after the key goes quiet.
`signin:ip:{ip}`	Small sign-in allowance by client IP	Exact or strict because the allowance is small.
`signin:account:{account}`	Small sign-in allowance by target account	Stops shared or rotating IPs from avoiding the sign-in policy.
`exports:{key}`	Export allowance or queued slots	Refused or bounded during a store outage.

Architecture diagramRules are durable, counts are temporary

Read:Learn: Data model design Learn: Storage choice justification Learn: Redis

Try it: Plans, keys and countersGuided practice · Rate limiter, API platform · Senior

Check yourself

Why not store counts in the same Postgres rows as the customer plan?
Show answerHide answer
Counts change on every request and need a tiny check budget. Plans change rarely and need audit. A durable row update per request makes the policy store the bottleneck.

The check and the client contract

From the briefNumbers here are given in the practice brief.

The gateway asks an internal limiter for every applicable policy and forwards the request only if all allow it. A customer over its allowance receives HTTP 429 Too Many Requests, which RFC 6585 defines for rate limiting and says may include Retry-After (RFC 6585 (opens in a new tab)). Retry-After itself is defined in RFC 9110 as either an HTTP date or seconds to delay (RFC 9110 (opens in a new tab)). If you expose IETF RateLimit fields, call out that RateLimit-Policy and RateLimit are from an Internet-Draft dated 23 May 2026, not an RFC (IETF draft (opens in a new tab)). Stripe’s public docs are a real example of a public API distinguishing global, endpoint, concurrency and resource-specific limits, and returning 429 for rate-limited responses (Stripe docs (opens in a new tab)).

Internal and external contract

Call	Request	Response	Notes
`POST /internal/limits/check`	`subject`, endpoint group, cost, request id	`allowed`, `limit`, `remaining`, `retry_after_s`, reason	The gateway asks about all applicable policies; the strictest policy decides.
Any public API call	API key and endpoint	Normal response plus quota and remaining allowance	Use the draft RateLimit fields or the X-RateLimit convention; document one.
Over allowance	Same request	`429` with `Retry-After` and a JSON body	This means the customer exceeded its allowance.
Platform dependency unavailable	Sensitive policy cannot be checked	`503` with `Retry-After`	This is not a customer overage.
`POST /v1/auth/sign-in`	IP, account and credentials	Session, `401`, `429` or `503`	No API key yet: count by generous IP policy and by target account.
`PUT /admin/customers/{id}/limits`	Policy override	Audited rule version	Support changes apply through the policy distributor.

Architecture diagramThe client contract

Read:Learn: API contract design Learn: Rate limiting and abuse prevention

Try it: The check, and what the client seesGuided practice · Rate limiter, API platform · Senior

Check yourself

Why distinguish 429 from 503 here?
Show answerHide answer
A 429 means the customer used its allowance and should slow down. A 503 means the platform cannot safely decide or is overloaded, so the client should retry later for a different reason.
Which limits apply to a Pro export request?
Show answerHide answer
At least the Pro plan allowance and the export endpoint allowance. The request goes through only if both allow it, and the stricter remaining allowance is what the client should see.

High-level design

Build the design in the order the request experiences it: identify the caller, load policy locally, spend from one shared count, and return either a normal response or a documented limit response.

Check at the gateway

The gateway authenticates the API key, maps the route to an endpoint group, and then asks the limiter before the backend service does work. Sign-in is checked at the same point, but by IP and target account because there is no API key yet. Service-owned limits can still exist for facts the gateway cannot know, such as rows exported by a job.

A request reaches the gateway with an API key, except sign-in.
The gateway resolves the customer, plan and endpoint group from its local policy snapshot.
It asks the limiter for every applicable policy.
Only an allowed request reaches the backend service.

Architecture diagramThe check sits at the gateway

The gateway knows the API key and endpoint group before it spends any service work.

Diagram as text

Components: Client (Client); Authenticate key (Auth); Gateway check (API gateway); Limiter (Rate limiter); API service (Service); Sign-in request (Client). Connections: Client → Authenticate key (API key); Authenticate key → Gateway check (customer + plan); Gateway check → Limiter (policies to check); Gateway check → API service (only if allowed); Sign-in request → Gateway check (IP + account).

Read:Learn: API Gateway Learn: High-level architecture

Try it: One limit, many gatewaysGuided practice · Rate limiter, API platform · Senior

Check yourself

Why not let each backend service add its own limiter?
Show answerHide answer
Then requests do work before they are refused, every service reimplements the rule, and a cross-service customer allowance is hard to enforce consistently.

Keep one count per customer in the region

A customer’s requests can land on many gateways. Each gateway must spend from the same regional state for that customer and policy, or forward to the owner of that key. Accepted shapes include a Redis-style shared counter store, a dedicated limiter service sharded by key, one owner per key, or leased local shares whose totals cannot exceed the allowance. What is not accepted is a full allowance on every gateway.

The load balancer sends the same key to different gateways.
Each gateway addresses the same limiter shard or shared store entry for that key.
The limiter spends from the plan and endpoint policies and returns one decision.
The gateway includes remaining allowance in the response or sends the documented denial.

Architecture diagramOne allowance across many gateways in a region

Distribute policy off the request path

From the briefNumbers here are given in the practice brief.

Rules live in a durable database, but a request does not read that database. Gateways keep a local snapshot and refresh it from a durable feed, a config-service watch, or a polled version every few seconds. This meets the brief’s change window without adding a rules lookup to every request.

Support writes an override through the Admin API, with audit.
The rules database records a new version.
Gateways refresh the snapshot from a replayable feed or polling version.
The next checks use the new policy from memory.

Architecture diagramPolicy changes off the hot path

Design a rate limiter

The answer in two minutes

Also asked as

What interviewers look for

The decisions that matter

Understanding the problem

Functional requirements

Non-functional requirements

Scale

Out of scope

The numbers that shape the design

Core entities and stores

The check and the client contract

High-level design

Check at the gateway

Keep one count per customer in the region

Distribute policy off the request path

Sign in with Google to read the rest

Practise this

The answer in two minutes

Also asked as

What interviewers look for

The decisions that matter

01Understanding the problem

Functional requirements

Non-functional requirements

Scale

Out of scope

02The numbers that shape the design

03Core entities and stores

04The check and the client contract

05High-level design

Check at the gateway

Keep one count per customer in the region

Distribute policy off the request path

Sign in with Google to read the rest

12Practise this

Understanding the problem

The numbers that shape the design

Core entities and stores

The check and the client contract

High-level design

Practise this