07

Concepts

Event Broker

Kafka from the ground up — why it exists, how it stores data, and the design decisions behind partitions, offsets, and consumer groups.

Why Kafka exists, page cache writes, topics, partitions, brokers, replication, ISR, and hot partition problems.

TopicsPartitionsReplicationISR

Partitioner strategies, batching, and compression — how producers balance throughput against ordering guarantees.

PartitionersBatchingCompressionAcks

Pull model, offsets, offset storage, consumer groups, and the rebalancing problem when group membership changes.

OffsetsConsumer GroupsRebalancingLag

Retention policies and exactly-once processing — the hardest guarantees to get right in a distributed log.

RetentionExactly-OnceIdempotent ProducerTransactions

Consumer lag, scaling consumers, load shedding, and the signals that tell you a consumer is falling behind.

Consumer LagScalingLoad SheddingBackpressure Signals

Kafka vs SQS vs RabbitMQ

The fundamental difference between a log and a queue. Retention, routing, ordering, throughput — when to choose what.

Log vs QueueRetentionRoutingWhen To Choose