A complete system design reference, from client-server basics to distributed systems theory. Covers caching, sharding, CAP theorem, and 6 worked designs.

System Design Notes: Basic to Advanced

PART 1: FOUNDATIONS

1.1 What System Design Actually Is

You’re translating a business need (“users need to message each other instantly”) into a technical blueprint (which servers, databases, protocols, and data flows make that true, reliably, at scale).

Two levels you’ll hear thrown around:

High Level Design (HLD): the boxes and arrows. Which services exist, which databases they use, how they talk to each other.
Low Level Design (LLD): what’s inside one box. Class structure, API contracts, schema, algorithms.

Most interviews and most real architecture reviews live at the HLD level. LLD matters once you’re actually writing the service.

1.2 A Request’s Journey

Here’s what happens the instant you hit enter on a URL. Every system design conversation assumes you know this cold.

HTTP itself is stateless. Every request stands alone unless you deliberately add state back in through cookies, tokens, or sessions.

1.3 Scaling Up vs Scaling Out

	Vertical (scale up)	Horizontal (scale out)
What you do	Add CPU/RAM to one machine	Add more machines
Ceiling	Hits hardware limits fast	Nearly unlimited
Complexity	Almost none	Real: you now need load balancing, coordination, consistent data
Failure mode	One machine, one point of failure	Distributed, fault tolerant if designed right

Every system I’ve worked on that mattered eventually scaled out. Scaling up buys you time early on, nothing more.

1.4 Latency, Throughput, Availability

Latency is how long one request takes.
Throughput is how many requests you handle per second.
You can trade one for the other. Batching improves throughput and hurts latency.
Availability is measured in nines. 99.9% is about 8.7 hours of downtime a year. 99.999% is about five minutes a year. Every extra nine costs real engineering effort, so pick a target that matches the actual business need, not the biggest number that sounds impressive.

PART 2: CORE BUILDING BLOCKS

2.1 Load Balancers

The traffic cop. Requests come in, the load balancer decides which backend server handles them.

Common algorithms: round robin, least connections, IP hash for session stickiness. Weighted versions of each exist for servers with uneven capacity.

L4 load balancers route on IP and port only, fast and dumb. L7 load balancers read the actual HTTP request (path, headers, cookies) and route smarter, at some CPU cost.

2.2 Caching

Store the answer somewhere fast so you don’t have to compute or fetch it again.

Cache strategies:

Cache aside: app checks the cache first, on a miss it reads the database and populates the cache. Simple, and the default choice for most systems.
Write through: every write goes to the cache and database together. Reads are always fresh, writes get slower.
Write back: writes hit the cache first and get flushed to the database later. Fast writes, but a cache crash can lose data.

Eviction policies you should know by name: LRU (evict what’s least recently used, the common default), LFU (evict what’s least frequently used), FIFO (evict oldest regardless of use).

Cache invalidation is genuinely one of the hardest problems in this field. TTLs, explicit invalidation on write, and versioned keys are the three tools you’ll reach for.

2.3 CDN

A network of edge servers that cache static content (images, video, JS, CSS) physically close to your users, cutting the distance data has to travel.

Pull CDN: fetches from your origin on first request, caches after.
Push CDN: you upload content to it proactively.

Most teams use pull CDNs. Simpler operationally, and you don’t have to think about it.

2.4 Proxies

Forward proxy sits in front of clients. It hides the client from the server. Corporate proxies and VPNs are forward proxies.
Reverse proxy sits in front of servers. It hides the server from the client, and typically also handles load balancing, TLS termination, and caching. NGINX in front of your app servers is a reverse proxy.

2.5 Rate Limiting

Protects a service from being overwhelmed, whether by abuse or just an unexpected traffic spike.

Token bucket: tokens refill at a fixed rate, each request consumes one, bursts allowed up to bucket size. Most commonly used in practice.
Leaky bucket: requests processed at a fixed, constant rate. Smooths bursts entirely, no spikes allowed through.
Fixed window counter: count requests in a time window, simple, but allows a burst right at the window edge.
Sliding window log: tracks requests over a rolling window, most accurate, costs more memory.

2.6 REST vs GraphQL vs gRPC

	REST	GraphQL	gRPC
Transport	JSON over HTTP	JSON over HTTP, single endpoint	Protobuf over HTTP/2
Strength	Simple, cacheable, everyone knows it	Client asks for exactly the fields it needs	Very fast, strongly typed, ideal for internal service to service calls
Weakness	Over fetching and under fetching are common	Harder to cache, more server complexity	Not human readable, not browser native

My rule of thumb: REST for public APIs, gRPC for internal service communication, GraphQL when your frontend teams are drowning in over fetching problems.

2.7 Message Queues and Async Processing

Producer —> [ Queue / Topic ] —> Consumer(s)

Decouples the sender from the processor. The sender doesn’t wait around for the work to finish.

Point to point queue: one message, one consumer. RabbitMQ, SQS.
Pub/Sub: one message, many subscribers. Kafka, Google Pub/Sub.

Kafka specifics worth knowing: messages live in topics, topics split into partitions for parallelism, data is retained on disk for a configurable window, and each consumer tracks its own offset independently.

This pattern is what lets you absorb traffic spikes, retry failed work safely, and build event driven systems where services react to what happened rather than being told what to do.

2.8 Monolith vs Microservices

Start monolithic. Split into microservices once the team and the scale actually demand it, not before. Microservices trade code complexity for operational complexity: service discovery, distributed debugging, network reliability all become your problem the moment you split.

PART 3: DATA LAYER

3.1 SQL vs NoSQL

	SQL	NoSQL
Schema	Fixed	Flexible
Examples	PostgreSQL, MySQL	MongoDB, Cassandra, Redis, Neo4j
Best for	Complex queries, relationships, transactions	High write throughput, flexible data shapes, horizontal scale
Consistency	ACID, strong	Usually BASE, eventual

ACID: atomicity (all or nothing), consistency (data always satisfies constraints), isolation (concurrent transactions don’t step on each other), durability (once committed, it survives a crash).

BASE: basically available, soft state, eventually consistent. The NoSQL answer to ACID, trading strict correctness for uptime and scale.

3.2 Indexing

An index is a data structure, usually a B Tree, that lets the database find rows without scanning the entire table. Reads get much faster. Writes get slightly slower because the index has to update too. Composite indexes across multiple columns help, but column order in the index matters a great deal.

3.3 Replication

Leader follower: all writes go to the leader, followers replicate and serve reads. Simple mental model. The leader is a bottleneck and a single point of failure unless you build automated failover.
Multi leader: several nodes accept writes and sync with each other. Useful across regions, painful for conflict resolution.
Leaderless: any node accepts reads and writes, relies on quorum reads and writes for consistency. This is the Dynamo model.

Synchronous replication waits for the follower to confirm before acknowledging the write. Safer, slower. Asynchronous doesn’t wait. Faster, with a real risk of losing recent writes if the leader dies.

3.4 Sharding

Splitting one large database across multiple machines because no single machine can hold or serve all of it.

Range based: split by key range. Simple, but easy to create hotspots.
Hash based: hash the key to pick a shard. Even distribution, but range queries become painful.
Directory based: a lookup service maps keys to shards. Flexible, adds a dependency.
Geo based: shard by region. Great for latency, harder for anything that spans regions.

The real cost of sharding shows up in cross shard joins and rebalancing when you add or remove shards. Plan for this before you need it, not after.

3.5 Consistent Hashing

Distribute keys across nodes so that adding or removing a node only remaps a small slice of the data, not everything.

key “user_42” hashes to a point on the ring,

gets assigned to the next node clockwise.

Add virtual nodes (each physical node claims several positions on the ring) to spread load more evenly. This is how Cassandra, DynamoDB, and most distributed cache client libraries handle node membership changes without a full data reshuffle.

PART 4: DISTRIBUTED SYSTEMS THEORY

4.1 CAP Theorem

During a network partition, you have to choose between:

Consistency: every read returns the most recent write.
Availability: every request gets a response, even if it’s not the freshest data.

Partition tolerance is assumed in any real distributed system, so in practice this comes down to CP versus AP. CP systems (traditional RDBMS clusters, HBase) favor correctness over uptime. AP systems (Cassandra, DynamoDB) favor uptime and resolve conflicts after the fact.

4.2 PACELC

The extension nobody skips once they’ve internalized CAP. Even without a partition, you still face a tradeoff: if there’s a partition, choose availability or consistency. Else, choose latency or consistency. This is the everyday tradeoff, not just the failure mode one, and it’s the one you’ll actually be tuning most of the time.

4.3 Consensus Algorithms

How do nodes in a distributed system agree on one value even when some of them fail?

Paxos: the original. Correct, and famously difficult to implement correctly.
Raft: designed specifically to be understandable. A leader is elected, handles all writes, replicates a log to followers. If the leader dies, a new election happens.

You’ll meet these inside tools like etcd and ZooKeeper, used for leader election, distributed locks, and configuration management.

4.4 Distributed Transactions

A transaction that spans multiple services can’t rely on a single database’s ACID guarantees.

Two Phase Commit: a coordinator asks every participant “can you commit?” then tells everyone to commit or roll back together. Strongly consistent, but blocking. If the coordinator dies mid process, participants can get stuck waiting.
Saga pattern: break the transaction into a chain of local transactions, each with a compensating action to undo it if something later fails.

Sagas can be choreographed (each service listens for events and reacts on its own) or orchestrated (one central coordinator tells each service what to do next). Orchestration is easier to reason about. Choreography scales better with fewer bottlenecks.

4.5 Idempotency

An operation is idempotent if doing it once or ten times produces the same result. This matters because in a distributed system, retries are guaranteed to happen eventually. “Set balance to 100” is idempotent. “Add 10 to balance” is not, unless you attach a unique idempotency key so the server can recognize and ignore a duplicate.

4.6 Eventual Consistency and Conflict Resolution

In AP systems, replicas can temporarily disagree. Ways to resolve that:

Last write wins: simplest option, can silently drop a legitimate update.
Vector clocks: track causality so you can tell a real conflict apart from a normal sequential update.
CRDTs: data structures built so concurrent updates always merge the same way, no conflict possible. Used in collaborative editors and distributed counters.

4.7 Observability

Logging: discrete events, “user 123 logged in at 10:03am.” Aggregated centrally with tools like the ELK stack or Loki.
Metrics: numeric time series, CPU percent, request rate, error rate. Prometheus and Grafana are the standard pairing.
Distributed tracing: follow one request as it crosses many services to find exactly where latency or errors are creeping in. Jaeger, Zipkin, OpenTelemetry.

The four golden signals from Google’s SRE playbook are worth memorizing: latency, traffic, errors, saturation.

4.8 Security Fundamentals

Authentication answers who you are. Authorization answers what you’re allowed to do. Keep these concepts separate in your design.
JWTs and OAuth 2.0 are the standard tools for stateless auth across distributed services.
TLS everywhere for data in transit, encryption at rest for anything sensitive.
An API gateway is a natural place to centralize auth, rate limiting, and routing so individual services don’t each reinvent it.

PART 5: A FRAMEWORK FOR DESIGNING ANYTHING

This is the sequence I actually use, whether it’s an interview or a real project kickoff.

Clarify requirements. Functional requirements first (what must it do), then non functional (scale, latency targets, consistency needs, availability target).
Estimate scale. Daily active users, requests per second, storage growth per year. Rough numbers, not precision.
Define the interface. What does the API actually look like from the outside.
Draw the high level design. Client, load balancer, application servers, cache, database, queue. Boxes and arrows.
Go deep on the hard part. Usually the schema and sharding strategy, the caching approach, and whatever is uniquely tricky about this particular system.
Find the bottlenecks and single points of failure. Fix them with redundancy, caching, sharding, or async processing.
Talk through the tradeoffs out loud. There’s rarely one right answer here. The value is in showing you understand what you’re giving up with each choice.

PART 6: WORKED EXAMPLES

6.1 URL Shortener

The core problem is generating a unique short code for each long URL and redirecting fast.

Storage is a perfect fit for a key value store. Reads are dominated by a small set of hot links, so cache aggressively. Scale writes by sharding on the short code hash.

6.2 Rate Limiter Service

A shared Redis backed counter, usually token bucket or sliding window log, per user or per IP. Deploy it once, inside the API gateway, so every downstream service benefits without reimplementing it.

6.3 Chat Application

The core problem is real time bidirectional delivery, plus reliable storage for offline users.

WebSockets replace polling with a persistent connection. Since users can land on different connection servers, a lookup layer routes messages to wherever the recipient actually is. Messages get stored in a write heavy, append only store like Cassandra, and undelivered messages queue until the recipient reconnects.

6.4 News Feed

The core problem is showing a fast, personalized feed even to users who follow millions of accounts.

Fan out on write: when someone posts, push it into every follower’s feed immediately. Reads are instant. Writes get expensive fast if the poster has millions of followers, the “celebrity problem.”
Fan out on read: assemble the feed at request time by pulling recent posts from everyone the user follows. Writes are cheap, reads are slower.
Hybrid, what most large systems actually run: fan out on write for normal accounts, fan out on read for celebrity accounts, merged together when the feed is built.

6.5 Distributed Unique ID Generator

The core problem is generating unique, roughly time sortable IDs across many machines with no central coordination.

Twitter’s Snowflake approach: an ID is built from a timestamp, a machine ID, and a per machine sequence number, all packed into one integer. Each machine generates IDs independently, and because the timestamp comes first, the IDs stay sortable by time.

6.6 Web Crawler

The core problem is crawling billions of pages without duplicating work or hammering any single site too hard.

Key pieces: a URL frontier (a prioritized queue of what to visit next), a Bloom filter to quickly check “have I already seen this URL,” a politeness policy that rate limits per domain, and a pool of distributed workers pulling from a shared queue, typically Kafka.

Glossary

QPS: queries per second.
SPOF: single point of failure.
Idempotent: same result no matter how many times you do it.
Hot key / hot partition: one key or shard taking disproportionate traffic.
Backpressure: telling upstream to slow down when a system is overwhelmed.
Bloom filter: a probabilistic structure for fast “definitely not here / maybe here” checks with very low memory cost.
Circuit breaker: stop calling a failing downstream service temporarily so its failure doesn’t cascade into yours.

Study Order

Foundations, Part 1. One day.
Core building blocks, Part 2. Two to three days. Build something small using at least caching and a load balancer.
Data layer, Part 3. Two to three days. Most real interview questions live here.
Distributed systems theory, Part 4. Take your time, this is what separates senior thinking from junior thinking.
Worked examples, Part 6. One per day. Sketch it yourself first, then check your version against the notes.

The fastest way to actually learn this material is to draw the diagram from memory and defend your tradeoffs out loud to another person. Reading alone won’t get you there.