Most engineers can code their way out of anything. Fewer can design their way out of a system that breaks at scale.
System design is not about memorizing architectures. It is about internalizing the 10 concepts that explain why every architecture is built the way it is. Get these right, and you can design any system confidently. Miss them, and you are always guessing.
1. Scalability: Vertical vs Horizontal
Scalability is how well your system grows under load without proportionally increasing failure. There are two ways to scale and choosing wrong is expensive.
Vertical scaling means buying a bigger machine. More CPU, more RAM. Simple to implement, but it has a hard ceiling and a single point of failure. Horizontal scaling means adding more machines. This is how every major system at scale operates today. YouTube, Netflix, Twitter — they all scale out, not up.
The classic mistake: building a system that can only vertical scale, then hitting the ceiling at 10x traffic. Design for horizontal from day one if you expect growth.
Every distributed system makes a promise it cannot fully keep. The CAP theorem states that during a network partition, you must choose between Consistency (all nodes return the same data) and Availability (every request gets a response, even if stale).
Partition tolerance is not optional — networks fail. So the real choice is always CP or AP.
System Type | Examples | Trade Off |
|---|---|---|
CP (Consistent + Partition Tolerant) | MongoDB, HBase | May reject requests during partition |
AP (Available + Partition Tolerant) | Cassandra, DynamoDB | May return stale data |
Ticket booking systems need CP. Two users cannot book the same seat. Social media feeds are fine with AP. Seeing a profile picture update a few minutes late is not a disaster.
[Diagram: CAP Theorem and Real World Database Choices]
3. Caching: The Single Biggest Performance Win
Caching is the most impactful optimization available in system design, and it is also the most misunderstood. The core problem it solves: reading from a database on every request does not scale.
Cache-aside is the most common pattern. The application checks the cache first. On a miss, it reads from the database and populates the cache. Write-through keeps cache and database always in sync by writing to both together. The tradeoff? Write latency doubles.
The hardest problem in caching is not choosing a pattern. It is cache invalidation: knowing when stale data has become a problem. Get comfortable with TTLs, version keys, and event-driven invalidation before you need them in production.
4. Consistent Hashing: How Distributed Systems Find Data
When you shard a database across N nodes, the obvious approach is hash(key) % N. The problem appears the moment you add or remove a node. Suddenly every key remaps. You either need to move all the data or serve the wrong data during migration.
Consistent hashing fixes this elegantly. Keys and nodes both live on a ring. Each key maps to the nearest node clockwise. When you add a node, only the keys between it and its predecessor move. Amazon's DynamoDB and Apache Cassandra both use consistent hashing at their core.
At its heart, consistent hashing is about minimizing data movement when your cluster topology changes. This is the concept that makes truly elastic distributed databases possible.
5. Load Balancing: Traffic Distribution Done Right
A load balancer sits in front of your fleet and distributes incoming requests across servers. The naive explanation ends there. The interesting part is the strategies.
Round robin distributes evenly, great for homogeneous servers. Least connections routes to the server with fewest active requests, better for variable workloads. IP hash ensures the same client always hits the same server, critical for session stickiness when you cannot use distributed session storage.
Health checks are what separate a real load balancer from a traffic splitter. If a server stops responding, a good load balancer stops sending it traffic within seconds. This is your first line of defense against partial outages becoming total ones.
6. SQL vs NoSQL: It Is Not About Which Is Better
Every senior engineer gets tired of this debate because the answer is always the same: it depends on your access patterns.
SQL databases give you ACID guarantees, rich queries, and relational integrity. When you need strong consistency and your data model is relational, SQL is the right choice. NoSQL trades those guarantees for horizontal scalability, flexible schemas, and often dramatically higher throughput for specific access patterns.
Criterion | SQL | NoSQL |
|---|---|---|
Consistency | Strong (ACID) | Eventual (BASE) |
Scaling | Vertical primary | Horizontal primary |
Schema | Fixed | Flexible |
Query flexibility | High | Depends on type |
Best for | Transactions, reports | High write throughput, variable schema |
The wrong answer in an interview is having a preference. The right answer is knowing which access pattern each optimizes for and choosing accordingly.
7. Replication: Surviving Failure Without Losing Data
Replication means keeping copies of your data on multiple nodes. This serves two goals: availability (if one node dies, another takes over) and read scalability (read replicas handle queries, the primary handles writes).
Primary-replica replication is the default model. The primary accepts all writes. Replicas stream those changes asynchronously. The tradeoff: if you read from a replica before a write propagates, you get stale data. This is eventual consistency in practice, not in theory. Synchronous replication eliminates staleness but adds write latency because every write must confirm to every replica before succeeding.
8. Message Queues and Pub/Sub: Decoupling at Scale
Tight coupling between services is how a spike in one place takes down your entire system. Message queues and pub/sub patterns exist to break that coupling.
A message queue is point to point. Service A sends a message. Service B processes it when ready. The queue absorbs traffic spikes and ensures no message is lost if B is temporarily down. Pub/Sub is one to many. A publisher emits events. Multiple subscribers consume them independently. One payment event can fan out to your billing service, your analytics service, and your fraud detection system simultaneously.
Dead Letter Queues deserve a mention here. When a message fails processing repeatedly, it moves to a DLQ instead of blocking the whole queue. Without a DLQ, one malformed message can stall your entire pipeline.
9. Circuit Breaker: Failing Fast on Purpose
When a downstream service starts failing, the naive approach is to keep retrying until it recovers. The result: your service also starts failing, connections pile up, timeouts cascade, and the failure spreads.
The circuit breaker pattern interrupts this cascade. It monitors failure rate on outbound calls. When failures cross a threshold, the breaker trips open: all calls fail immediately without even attempting the downstream. After a timeout period, it enters a half-open state and allows a test request through. If that succeeds, the breaker closes and normal traffic resumes.
Netflix's Chaos Monkey and the broader chaos engineering discipline exist because of exactly this pattern. You do not discover that your circuit breakers work by waiting for a real outage.
10. Observability: You Cannot Fix What You Cannot See
Modern systems must be observable. Engineers should be able to understand what the system is doing internally by looking at metrics, logs, and traces. This is the concept most junior engineers skip and most senior engineers wish they had implemented earlier.
Metrics tell you what is happening (request rate, error rate, latency percentiles). Logs tell you what happened in detail (specific error context, user IDs, payloads). Traces tell you where time was spent across the full request path through multiple services. All three together give you the full picture.
The practical advice: instrument from day one. Adding observability to a system in production under incident pressure is the worst possible time to learn that your logs have no structure.
[Diagram: The 10 Concepts and Where They Live in a System]
What This Means For Engineers
These 10 concepts are not independent. CAP theorem explains why you choose Cassandra over Postgres for a specific service. Consistent hashing explains how Cassandra distributes data across nodes. Replication explains how Cassandra stays available when a node dies. Learning them in isolation misses the point. Learn how they compose.
Interviews test reasoning, not recall. The question "design a URL shortener" is not asking you to recite architecture. It is asking you to apply scalability, caching, SQL vs NoSQL tradeoffs, and load balancing in sequence. The engineer who thinks in concepts can design anything. The engineer who memorized the URL shortener answer freezes at "design a pastebin."
These are not just interview concepts. Every production decision you make in your day job involves one of these 10 ideas. Choosing a cache TTL is a consistency tradeoff. Adding a retry is a circuit breaker decision in disguise. Adding a queue between two services is a decoupling call. Internalize the concepts and your production instincts sharpen automatically.
If this sparked something useful, hit like ❤️ For more simple explanations, useful insights on coding, system design and tech trends, Subscribe To My Newsletter! 🚀
See you next week with more exciting content!
Signing Off, Scortier