Data Processing

Windows, watermarks, statefulness, and crash recovery. Processing infinite streams of events without losing data or blowing memory.

MapReduce internals, Spark's DAG execution model, and how batch systems handle hot keys and skewed data.

Two architectures for mixing batch and stream. Lambda's complexity vs Kappa's simplicity — and when each one wins.

Schema registries, Protobuf, and Avro — how to change data formats in a live pipeline without breaking producers or consumers.