S3 / Blob Storage

Object storage is a service for storing large, unstructured blobs — images, videos, documents, backups, log archives, ML datasets — addressed by a key within a bucket. You PUT an object and get back a URL; you GET it by key. There are no rows, no schema, no queries — just durable, cheap, effectively unlimited file storage.

Its defining properties make it boring in the best way: it scales automatically (no capacity to provision; request rates scale per prefix, see below), extremely durable (S3 advertises eleven nines — 99.999999999% — via replication and erasure coding across devices and facilities), and cheap (cents per GB-month, far less than storing the same bytes in a database). In an interview you can treat its scalability and durability as given and spend your time on how data flows in and out.

The canonical pattern is store the blob in S3, store a pointer (URL/key) in your database. The database indexes and queries the metadata with low latency; S3 holds the heavy bytes cheaply. Uploads go direct from the client via pre-signed URLs so your app never proxies the bytes, and downloads are served through a CDN with the bucket as origin. Reach for object storage whenever a system stores files bigger than a few KB — and never use it as a primary database or for low-latency random access to structured data.

A few facts cover almost every interview use:

1. Objects, not files or rows. Each object is a blob plus metadata under a key in a flat namespace (the "folders" in a key like users/42/avatar.png are just a naming convention). Objects are written and read whole — there is no in-place edit and no partial append; to "change" an object you upload a new version. This is why it is great for media and terrible as a mutable database.

2. Durability and scale are givens; design around access, not capacity. Providers replicate and erasure-code across facilities for eleven-nines durability and present effectively unlimited capacity. So in a design you don't size S3 — you focus on the upload path, the download path, and lifecycle/cost.

3. Direct-to-blob uploads via pre-signed URLs. The client requests a short-lived signed URL scoped to one object; it then PUTs the bytes straight to S3, bypassing your app entirely. The app validates and records metadata but never proxies the file — which is what makes uploads scale.

4. Serve through a CDN, with the bucket as origin. For downloads, put a CDN in front so objects are cached at the edge near users; the origin bucket only sees cache misses. Private content uses signed URLs / signed cookies with short expiry so only authorised users can fetch.

5. Large uploads use multipart. Files over ~100 MB are uploaded in parallel chunks (S3 multipart / the tus protocol) so a network blip retries one part instead of the whole file — and a lifecycle rule aborts abandoned multipart uploads so they don't accrue cost.

6. Writes can be guarded. For create-only keys, complete the PUT or multipart upload with If-None-Match: *; for compare-and-swap overwrites, use If-Match with the ETag you previously read. Those checks keep concurrent clients from silently overwriting each other, but your metadata row still needs an explicit state machine.