Designing File Uploads with Object Storage and Pre-signed URLs

Photo sharing, document storage and video platforms all need file uploads. Sending large files through your application servers wastes bandwidth and threads. The standard design lets clients upload directly to object storage.

The flow

1. The client asks the API for permission to upload, sending file name, size and type.

2. The API checks auth and quotas, creates a metadata row with status "pending", and returns a pre-signed URL: a time-limited URL (for example 15 minutes) that allows exactly one PUT to one object key.

3. The client uploads the bytes straight to object storage (S3, GCS, Azure Blob or an S3-compatible store).

4. Storage emits an event, or the client calls "complete"; the backend verifies the object exists and its size matches, then marks it "uploaded" and triggers processing such as thumbnails, virus scanning or transcoding.

Large files

Use multipart uploads: split the file into parts (say 8 to 64 MB), upload them in parallel with separate pre-signed URLs, retry only failed parts, then call complete. Clients can resume after a network drop by asking which parts already arrived.

Security details

• Generate the object key on the server (for example user_id/uuid), never trust a client-supplied path.

• Restrict content type and maximum size in the signed policy.

• Keep the bucket private; serve downloads through short-lived pre-signed GET URLs or a CDN with signed cookies.

• Scan uploads before making them visible to other users.

Cleanup and consistency

Pending rows whose upload never completed should expire through a background job, and lifecycle rules should delete incomplete multipart uploads. Deduplicate by content hash if many users upload identical files.

Interview tip

Draw the three parties (client, API, object store) and make clear that file bytes never pass through the API servers. Then discuss multipart upload, post-processing and cleanup. That covers the usual follow-ups.

From the Linkjob AI team - Linkjob AI is an AI tool for job interviews: Linkjob AI interview assistant