Loading…

Understanding the problem

A feature flag lets a team change what an application does without deploying it: show the new checkout to a tenth of users, or to employees only, and switch it off when it misbehaves. A hosted flag service does this for many companies at once, through a dashboard their engineers use and an SDK their applications embed.

The twist is where the load is. Applications check flags on nearly every request, millions of times a second, so the design is less about answering checks than about getting rules to millions of places quickly, and about what happens when that fails.

The practice brief states the requirements up front; in an interview, you would confirm them in the first few minutes. Two carry the design: a check adds under a millisecond, and the service’s outage must not reach its customers.

Functional requirements

CoreTeams create flags (on/off, or several variations) per project and environment, and target them by user attributes, named segments and a percentage of users.
CoreCustomer applications get each flag’s value for the current user on nearly every request.
CoreA change made in the dashboard reaches running applications within about 5 seconds.
CoreDuring an incident, an engineer can turn a flag off everywhere within seconds.
CoreEvery change records who made it and what changed, and can be rolled back; changes to production flags that a team marks as protected need a second person’s approval.
CoreWhen a flag runs as an experiment, record which users saw which variation, for the customer’s analytics.
CoreTeams can see which flags have stopped changing or are no longer checked, so they can remove them.
Below the lineExperiment statistics.The service records who saw what; working out the result is the customer’s analytics.
Below the lineBilling.
Below the lineThe dashboard’s UI.
Below the lineSDKs for every language.Design one SDK contract; the languages follow it.

Non-functional requirements

LatencyA flag check adds under about 1 ms at p99 to the customer’s request.
LatencyA change reaches running applications within about 5 s at p99; turning a flag off is no slower.
AvailabilityIf the service is slow or down, customers’ applications keep working normally.
ConsistencyA user in a percentage rollout gets the same variation on every request and every server.
SecurityNothing a browser or phone receives reveals other users’ targeting data or a server secret.
Durability

Scale

From the briefNumbers here are given in the practice brief.

ScaleAbout 5,000 customer companies; about 150 flags in a typical environment and about 20,000 in the largest.
ScaleAbout 2 million customer server processes run the SDK at peak; the largest customer runs 50,000 and redeploys them all within about 10 minutes.
ScaleAbout 20 million web and mobile sessions open at peak.
ScaleCustomer applications check flags about 10 million times a second in total.
ScaleAbout 500,000 flag changes a day, mostly in working hours.
ScaleAn environment’s rules are usually under 50 KB; the largest is about 5 MB.
Scale

Check yourself

Why can’t customers’ flag checks be the service’s load?
Show answerHide answer
Each would add a network hop to a customer’s request and make the service a hard dependency of every app. Checks run in the SDK; the service only sends rules.

The numbers that shape the design

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

Work out only the numbers this design depends on, with round figures and one line of working each. The biggest number turns out not to be the service’s at all.

Back of the envelope for the SaaS platform brief

Quantity	Value	Working
Flag checks	About 10 million a second	From the brief; all inside customers’ applications.
SDK instances at peak	About 22 million	2 million server processes plus 20 million web and mobile sessions.
One change, largest customer	About 250 GB whole, 50 MB as a diff	5 MB × 50,000 processes, against about 1 KB × 50,000.
Largest redeploy	About 85 new processes a second	50,000 over 10 minutes, each needing up to 5 MB of rules.
Flag changes	A few a second	500,000 a day, mostly in working hours.

What the numbers tell you

Checks are the biggest number but run inside customers’ applications, so they don’t reach the service and size nothing on our side.
Connections and bytes per change size the service; sending only what changed is a 5,000-fold saving.
Changes are rare, so writes are easy; the history grows by about 180 million rows a year and needs a retention plan.

Architecture diagramWhere the load lands

Try it: The numbers that shape the designGuided practice · Feature flags, SaaS platform · Mid-level

Check yourself

Which number rules out resending the whole environment on each change?
Show answerHide answer
The largest customer’s: 5 MB to 50,000 processes is about 250 GB for one edit, against about 50 MB for only the changed flag.

Core entities

Two version numbers do different jobs, and mixing them up is the classic slip. Each flag and segment has an edit version, so two people saving the same flag a second apart get a conflict instead of silently undoing each other. Each environment has one revision that rises with every change to any flag or segment in it, so SDKs can say exactly which changes they have.

Rules are checked in order and the first match wins; a user who matches none gets the default rule, and a flag that is off serves its off variation whatever the rules say.

The entities and their key fields

Entity	Key fields	Why it exists
Environment	`env_id`, name, SDK keys, `revision`	Development, staging, production; the revision orders every change in it.
Flag	`flag_key`, variations, ordered rules, default, off variation, `flag_version`	What a check reads; the edit version catches conflicting saves.
Segment	`segment_key`, included users, clauses, `segment_version`	Reusable lists of users, some thousands long.
EnvironmentChange	`revision`, flag or segment and its key, new edit version, change, who, when, why	One per change, in revision order: the history and what SDKs replay.
ChangeRequest	proposed change, requester, approver, state	Changes to protected flags wait here for a second person.

Where each part lives

Part	Store	Why
Flags, segments and changes	PostgreSQL	Relational, written a few times a second, one transaction per change; history partitioned by month and archived.
Changes on their way out	Kafka, fed from an outbox	Every committed change is passed on, none lost.
Copies SDKs start from	Object storage behind a CDN	Read by millions of processes, labelled by revision.

Architecture diagramFlags, segments and one ordered log of changes

Read:Learn: Data model design Learn: Storage choice justification Learn: PostgreSQL

Try it: Flags, rules and environmentsGuided practice · Feature flags, SaaS platform · Mid-level

Keep reading+1 min

Rollback as a new change

Rolling a flag back publishes the old version’s content as a new change with the next revision. SDKs apply it like any other change, and the history keeps both the mistake and the fix, which is what an auditor wants to see.

Check yourself

A segment of beta users changes. Which number goes up?

The API

Three kinds of caller need three credentials. People edit flags through the dashboard, signed in and acting under a role. Server SDKs read the environment’s rules with a secret that stays inside the customer’s network. Browsers and phones run on users’ devices, where anything can be read and any request copied, so they get neither the secret nor the rules, which hold other users’ emails and account ids.

A device sends a public id plus a short-lived token its customer’s server signed, naming the environment, the signed-in user, the attributes rules may use and an expiry, and gets back only that user’s values. One product signs just the user key for the same reason (LaunchDarkly docs (opens in a new tab)); signing the whole context also stops a device changing its own attributes.

The calls

Call	Request	Response	Who
`PUT /v1/…/flags/{key}`	`expected_flag_version`, the flag	`200` with the new versions, `409` on a conflict, `202` for a protected flag	People, by role
`POST /v1/change-requests/{id}/approve`	None	`200` with the new versions	A second person
`GET /sdk/v1/rules`	None	`revision`, flags, segments	Server SDKs, with the server secret
`POST /sdk/v1/client-flags`	`context_token`	This user’s values and the `revision`	Devices, with the public id

Architecture diagramThree callers, three credentials

Read:Learn: API contract design

Try it: The dashboard, server and browser callsGuided practice · Feature flags, SaaS platform · Mid-level

Check yourself

A curious user copies the browser’s request and changes the user id. What do they get?
Show answerHide answer
A 401: the token no longer matches what the customer’s server signed, so the edge refuses it.

High-level design

Build the design one flow at a time, following the brief: a check, a change, and a browser or phone. The deep dives then harden it.

An application checks a flag

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

The SDK downloads its environment’s rules at start-up and holds them in memory. isEnabled("new-checkout", user) runs the ordered rules inside the application in a few microseconds, with no network call. The service doesn’t see checks; its job is keeping that copy current.

The trap is the obvious design: an evaluation service the SDK calls per check. At 10 million checks a second it adds a network hop to every request, and when it is slow or down, every customer’s application is too. A per-user cache in front of it still sends every new user’s first check to the network.

The application calls isEnabled("new-checkout", user).
The SDK runs the rules it holds and returns a variation in microseconds.
Changes arrive in the background and replace the rules between checks.

Architecture diagramA flag check stays in the application

The SDK holds the rules and answers in microseconds; the service only keeps that copy current. Dashed lines are asynchronous.

Diagram as text

Components: Application code (Service); SDK (Service); Rules in memory (Cache); Stream tier (Service). Connections: Application code → SDK (isEnabled); SDK → Rules in memory (microseconds); Stream tier → Rules in memory (changes), asynchronous.

Read:Learn: Latency budgeting Learn: High-level architecture

Try it: From a rule change to a flag checkGuided practice · Feature flags, SaaS platform · Mid-level

Check yourself

A page checks 200 flags. How many network calls does that make?
Show answerHide answer
None. Every check runs against the rules the SDK already holds.

A change reaches running applications

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

The flags API saves the change, the environment’s next revision and an outbox row in one PostgreSQL transaction, so the change is published if and only if it commits (transactional outbox (opens in a new tab)). A publisher puts it on Kafka, and may send a row twice, so consumers and SDKs skip revisions they already have. A stream tier holding a long-lived connection to each server SDK pushes it, about 1–2 seconds end to end.

An SDK that was cut off reconnects with the last revision it has; server-sent events send it as Last-Event-ID (WHATWG (opens in a new tab)), and the tier replays the changes since, or a full copy if it is far behind. Polling every 2 seconds also works: an ETag with If-None-Match gets a 304 when nothing changed (RFC 9110 (opens in a new tab)), at the price of about a million requests a second for 2 million servers alone.

An engineer saves a changed rule.
The change, revision N+1 and an outbox row commit together.
The publisher sends it to Kafka; the stream tier pushes it to SDKs in that environment.
Each SDK applies revision N+1 before its next check.

Architecture diagramA change reaches running applications

A browser or phone gets its flags

Devices don’t hold rules. At the start of a session the web or mobile app sends its signed context token and gets back its user’s values from the edge, then reads them locally for every check. When the environment’s revision changes, a light notice on the stream tells it to fetch its values again; a phone app in the background refreshes when it returns to the foreground.

The customer’s server signs a context token for the signed-in user.
The app sends it to the edge and receives this user’s values.
A “revision N” notice prompts a fresh fetch after a change.

Architecture diagramBrowsers and phones get answers, not rules

Design a feature flag system

The answer in two minutes

Also asked as

What interviewers look for

The decisions that matter

Understanding the problem

Functional requirements

Non-functional requirements

Scale

The numbers that shape the design

Core entities

Rollback as a new change

The API

High-level design

An application checks a flag

A change reaches running applications

A browser or phone gets its flags

Sign in with Google to read the rest

Practise this

The answer in two minutes

Also asked as

What interviewers look for

The decisions that matter

01Understanding the problem

Functional requirements

Non-functional requirements

Scale

02The numbers that shape the design

03Core entities

Rollback as a new change

04The API

05High-level design

An application checks a flag

A change reaches running applications

A browser or phone gets its flags

Sign in with Google to read the rest

12Practise this

Understanding the problem

The numbers that shape the design

Core entities

The API

High-level design

Practise this