Loading…

Understanding the problem

A paste site takes some text, returns a link, and shows the text to anyone who opens the link. Here it takes about 5 million pastes a day, serves about 100 reads for each write, and holds text up to 10 MB. Some pastes are public and listed; others are unlisted, readable by anyone with the link and by nobody else.

It looks like a URL shortener, but the payload is the content itself, so where the bytes live matters more than a redirect. The rest of the difficulty is in the edges: links that must not be guessable, expiry that must hold, abuse that arrives as creates, and copies of one paste spread across caches and continents.

The practice brief gives the requirements; confirm them quickly. The first non-functional one is the invariant: only people with an unlisted paste’s link can read it.

Functional requirements

CoreAnyone can create a paste of text up to 10 MB, choosing public or unlisted, an optional expiry and an optional language, and gets a link to it.
CoreAnyone with a paste’s link can read it, with syntax highlighting or as raw text.
CorePublic pastes appear in a list of recent pastes; unlisted ones never do.
CoreThe creator can delete their paste, with or without an account.
Below the lineEditing pastes and keeping versions.
Below the linePrivate pastes that only the owner can read.
Below the linePastes that disappear after one read.
Below the lineFull-text search and comments.
Below the lineA view counter per paste.Easy to add later with a counter updated off the read path.
Below the lineCustom link names.A chosen name is guessable by design, so it would suit public pastes only.

Non-functional requirements

SecurityOnly people with an unlisted paste’s link can read it: nobody finds one by guessing links or browsing the site.
DurabilityA saved paste is not lost before it expires or is deleted.
LatencyReads take under 200 ms at p99 for readers everywhere, including a paste that suddenly becomes popular.
AvailabilityReads are available 99.9% of the time.
ConsistencyAn expired or deleted paste stops being served within a minute, and its storage is reclaimed within a few days.
SecurityNo sender can flood the site or use it to host malware.

Scale

From the briefNumbers here are given in the practice brief.

ScaleAbout 5 million new pastes a day, with the busiest hour about 3 times the average; reads are about 100 times writes, about 500 million a day.
ScalePastes average about 10 KB (median about 2 KB); about 1% are over 1 MB, and the limit is 10 MB.
ScaleAbout 2 billion pastes are live, and about half set an expiry.
ScaleOne paste drew 20,000 reads a second last month; 60% of reads come from other continents.

Out of scope

Out of scopeAccount sign-up and sign-in internals.
Out of scopePayments and paid plans.
Out of scopeThe moderators’ own tools.
Out of scopeThe paste editor’s own features.

Check yourself

Why is an unlisted paste’s id a security decision, not just a naming one?
Show answerHide answer
Nothing else stands between a stranger and the paste: no login, no list. If the id can be guessed, the paste can be found.

The numbers that shape the design

From the briefNumbers here are given in the practice brief.

Work out the few numbers the design depends on. The request rate is modest; the text kept and the bytes read are what size it.

Back of the envelope from the brief

Quantity	Value	Working
New pastes a second	About 58, about 175 at the peak	5 million ÷ 86,400, × 3 for the busiest hour
Reads a second	About 5,800, about 17,000 at the peak	100 times the writes
Text written a day	About 50 GB	5 million × 10 KB
Text a year if nothing expired	About 18 TB	50 GB × 365
Live text today	About 20 TB	2 billion pastes × 10 KB
Bytes sent at the peak	About 170 MB a second	17,000 reads × 10 KB, before a viral paste

Architecture diagramWhere the load lands

The requests are modest; the bytes read and the text kept size the design. Numbers are illustrative, worked out from the brief.

Diagram as text

Components: Writes: ~58/s, ~175/s at peak (Service); Reads: ~5,800/s, ~17,000/s at peak (Service); Text: ~50 GB a day, ~20 TB live (Object store); Bytes out: ~170 MB/s at peak (CDN); One viral paste: 20,000 reads/s (Client). Connections: Writes: ~58/s, ~175/s at peak → Text: ~50 GB a day, ~20 TB live (stored); Reads: ~5,800/s, ~17,000/s at peak → Bytes out: ~170 MB/s at peak (sent); One viral paste: 20,000 reads/s → Bytes out: ~170 MB/s at peak (on top), asynchronous.

Read:Learn: Numbers and capacity estimation

Try it: The numbers that shape the designGuided practice · Pastebin, Shared pastes · Entry level

Check yourself

About 58 writes a second is small. Why does storage still need planning?
Show answerHide answer
Each write carries text, about 50 GB a day, and half the pastes have no expiry, so the stored text grows by terabytes a year.

What a paste is, and where it lives

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

A paste has two parts with different shapes. Its details are a small row, about 200 bytes: the id, public or unlisted, when it expires, its size, its language, where its text is, an owner or a delete token, a status (live, deleted or taken down) and when it was made. About 2 billion rows is about 400 GB, one Postgres primary with read replicas, or DynamoDB.

The text is large, written once and never changed. It goes to object storage such as S3 under the paste’s id: cheap per GB, durable across machines, and out of the database’s backups, replicas and cache. Pastes under a small threshold, say 4 KB, can stay in the row to save a lookup. The 10 MB limit is checked when a paste is saved.

The two parts of a paste

Part	Fields	Where it lives
Row	id, visibility, `expires_at`, size, language, text key, owner or delete token, status, `created_at`	Postgres or DynamoDB
Text	The content, up to 10 MB	S3 or GCS, under the paste’s id

Where the text could live

Store	A 10 MB paste	At 50 GB a day
A Postgres column	Fits (values up to 1 GB)	Backups, replicas and cache carry all the text
A DynamoDB item	Refused (items up to 400 KB)	Not possible for large pastes
Redis only	Fits	Memory is costly, and a failover can lose pastes
S3 or GCS	Fits	Cheap per GB, and the database stays small

Architecture diagramA small row and a large text

Read:Learn: Large file upload & blob handling Learn: S3 / Blob Storage

Try it: What a paste is, and where it livesGuided practice · Pastebin, Shared pastes · Entry level

Keep reading+1 min

What the stores actually allow

Postgres moves large values out of line once a row passes about 2 kB, and a value can reach 1 GB (PostgreSQL docs (opens in a new tab)), so the objection to it is cost and load, not capacity. DynamoDB caps an item at 400 KB (AWS (opens in a new tab)), so a paste over that can’t be one item at all.

Check yourself

The API

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

Keep it small. POST /v1/pastes with the text, a visibility, an optional expiry and a language returns the link and, for anonymous users, a delete token. GET /v1/pastes/{id} returns the paste, GET /raw/{id} returns only the text, and DELETE /v1/pastes/{id} with the token or the owner’s session removes it.

Errors are few: 413 over 10 MB, 400 for bad input, and one 404 for a paste that is missing, expired or deleted, so a stranger can’t tell which links once worked. Large pastes are uploaded straight to storage, which a deep dive covers.

The calls

Call	Returns	Notes
`POST /v1/pastes`	The link and a delete token	`413` over 10 MB; `400` for a bad expiry or language
`GET /v1/pastes/{id}`	The paste, highlighted in the browser	One `404` for missing, expired and deleted
`GET /raw/{id}`	The text only	Served from a separate domain as plain text
`DELETE /v1/pastes/{id}`	`204`	With the delete token or the owner’s session

Architecture diagramThe calls, and what a reader gets back

Read:Learn: API contract design

Try it: The APIGuided practice · Pastebin, Shared pastes · Entry level

Keep reading+1 min

Deleting without an account

An anonymous creator gets a delete token once, when the paste is saved, and the site stores only a hash of it, like a password. Presenting the token later proves ownership without an account. Losing it means the paste lives until it expires, which the create page should say plainly.

Check yourself

Why return the same 404 for a deleted paste as for one that didn’t exist?
Show answerHide answer
A different answer would tell a stranger that a link once held a paste, which is itself information about unlisted pastes.

High-level design

Build the design in two walkthroughs: saving and sharing a paste, then reading it.

Saving a paste, and sharing an unlisted one

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

The link is the only lock on an unlisted paste, so the id must be impossible to guess. Each paste gets 128 bits from the operating system’s cryptographically secure generator, written as 22 letters and digits. With about 2 billion live pastes, a random guess finds one about once in 10^29 tries. A UUIDv4 from a secure generator, with 122 random bits, works too; RFC 9562 warns that UUIDs are not to be treated as hard to guess (RFC 9562 (opens in a new tab)), which is why the generator must be a secure one and every other check still applies.

What fails: a counter (the next link is the next paste), a timestamp plus a machine number, a hash of the text, and six characters from any generator: about 57 billion ids against 2 billion pastes means about one guess in 28 lands. OWASP’s advice for enumerable ids is the same: check access, and use long random values in URLs (OWASP (opens in a new tab)). Unlisted pastes also stay out of the recent list, search engines and the sitemap.

Ids against 2 billion live pastes

Id	Possible ids	A random guess hits
A counter in base62	Unlimited, but in order	The next paste, every time
Six base62 characters	About 57 billion	About once in 28
Eight base62 characters	About 218 trillion	About once in 100,000
128 random bits (22 characters)	About 3.4 × 10^38	About once in 10^29

A user saves a 3 KB log as unlisted; the app server makes a 128-bit id, stores the text and the row, and returns the link.
The user sends the link to a colleague, who opens it.
Someone tries made-up links one after another; each is a 404.
A public paste is added to the recent list; unlisted ones are not.

Architecture diagramSaving an unlisted paste

Read:Learn: Rate limiting and abuse prevention Learn: Data model design

Try it: Saving a paste, and sharing an unlisted oneGuided practice · Pastebin, Shared pastes · Entry level

Keep reading+1 min

Why not short codes, as in a link shortener?

A link shortener picks short codes and handles collisions because its links are meant to be shared publicly; Design a URL shortener covers that. An unlisted paste’s link is a secret, so length wins, and at 128 bits a collision is so unlikely that no counter or retry is needed. GitHub says the same of its own unlisted gists: they aren’t private, since anyone who discovers the URL can see them (GitHub Docs (opens in a new tab)).

Check yourself

Reading a paste, a hundred times over

Illustrative exampleNumbers here are illustrative: round figures to reason with, not measurements.

A paste’s text never changes after it is saved, so it caches well. A CDN sits in front of the site: the first reader of a paste in an area fetches it from the origin, and later readers get the CDN’s copy, kept for up to a day but never past the paste’s expiry. Behind the CDN, Redis holds hot paste details, misses go to Postgres, and the text comes from S3.

Large downloads stay off the app servers: for a 10 MB raw file, the app server checks the paste and returns a short-lived signed URL for S3, or the CDN serves it. Highlighting happens in the browser from the raw text and the stored language, so the servers never render anything.

Where a read is answered

Compared on	Every read to the origin	Cache in the app tier	CDN and cache in front
Reads reaching the database	All of them	Misses only	Few
A distant reader waits	An ocean crossing	An ocean crossing	A nearby hop, mostly
A 10 MB download	Ties up an app server	Ties up an app server	Served by the CDN or S3

A reader opens a link; the CDN has a copy and returns it.
Another reader opens a paste nobody nearby has read; the CDN asks the origin, which checks the row and returns the text.

Design Pastebin

The answer in two minutes

Also asked as

What interviewers look for

The decisions that matter

Understanding the problem

Functional requirements

Non-functional requirements

Scale

Out of scope

The numbers that shape the design

What a paste is, and where it lives

What the stores actually allow

The API

Deleting without an account

High-level design

Saving a paste, and sharing an unlisted one

Why not short codes, as in a link shortener?

Reading a paste, a hundred times over

Sign in with Google to read the rest

Practise this

Why highlighting belongs in the browser

What the cache holds

The answer in two minutes

Also asked as

What interviewers look for

The decisions that matter

01Understanding the problem

Functional requirements

Non-functional requirements

Scale

Out of scope

02The numbers that shape the design

03What a paste is, and where it lives

What the stores actually allow

04The API

Deleting without an account

05High-level design

Saving a paste, and sharing an unlisted one

Why not short codes, as in a link shortener?

Reading a paste, a hundred times over

Sign in with Google to read the rest

12Practise this

Why highlighting belongs in the browser

What the cache holds

Understanding the problem

The numbers that shape the design

What a paste is, and where it lives

The API

High-level design

Practise this