Design Systems from First Principles
Deep inside modern infrastructure.
Stop memorizing boxes and arrows. Start understanding how the systems actually work.
🧠 Concepts
Fundamentals
Availability, reliability, scalability, fault tolerance, durability, concurrency, security, state machines, NFRs.
Caching
Write strategies, eviction policies, cache invalidation, distributed caching, Redis, cache problems.
Storage & Databases
ACID, SQL, indexing, CDC, pagination, connection pooling, read/write splitting.
Database Types
Key-value, document, column-family, search engines, graph, blob storage, NewSQL, OLTP vs OLAP.
Messaging
Message queues, pub/sub, delivery guarantees, dead letter queues, SQS, RabbitMQ.
Event Broker
Kafka architecture, producers, consumers, offsets, backpressure, Kafka vs SQS vs RabbitMQ.
Event-Driven Patterns
Event sourcing, CQRS, outbox pattern, CDC, inbox + outbox combined.
Data Processing
Stream processing, batch processing, MapReduce, Spark, Lambda & Kappa architectures, schema evolution.
Distributed Systems
CAP theorem, PACELC, consistency models, replication, sharding, consensus, CRDTs, failure detection.
📐 Back of Envelope Estimation
⚔️ Apply the Concepts — Case Studies
01
Foundation ● ○ ○ ○ ○
First principles of scale
Unique ID schemes, sharding strategies, read-heavy caching, hot key mitigation, pre-generated key pools, tiered storage.
Snowflake ID UUID Clock skew Consistent hashing Bloom filters KGS Hot key problem Cold storage
Start →
02
Ascent ● ● ○ ○ ○
Async pipelines & multi-channel delivery
Kafka fan-out, per-channel workers, retry/DLQ, scheduled dispatch, content deduplication, async uploads, expiry at scale.
Kafka fan-out Retry / DLQ Bloom filter Circuit breaker Content-addressable storage Async S3 upload TTL sweeper Watchdog
Open →
03
Expedition ● ● ● ○ ○
Distributed rate control
Rate limiting algorithms from scratch, atomic Redis counters, distributed enforcement, rule management, DDoS layered defence.
Token bucket Leaky bucket Sliding window Fixed window Lua atomicity Redis cluster DDoS defence
Open →
04
Summit ● ● ● ● ○
Storage engine internals
Leaderless replication, quorum reads/writes, LSM tree internals, compaction strategies, gossip membership, conflict resolution.
LSM trees Compaction Bloom filters Quorum W/R Gossip protocol Hinted handoff Vector clocks Anti-entropy
Open →
05
Battleground ● ● ● ● ●
Real-time systems at scale
WebSocket connection management, per-conversation sequencing, offline queues, thundering herd, backpressure, hot partition salting.
WebSockets Per-conv. sequencing Offline delivery Thundering herd Backpressure Hot partition salting Reconnect storm Read receipts
Open →