DD

Notification System · Deep Dives

Deep Dives

Seven deep dives — database, Kafka fan-out, per-channel workers, scheduled dispatch, retry/DLQ, rate limiting, and fault isolation.

DB selection for notification storage and schema design for high write throughput.

CassandraSchemaWrite Throughput

Queue selection rationale and Kafka fan-out design for multi-channel delivery.

KafkaFan-outQueue Selection

Per-Channel Workers

Push, SMS, and email worker architecture — isolation, third-party adapters, and failure containment.

PushSMSEmailWorker Isolation

Scheduled notifications, naive approaches, scheduler DB design, jitter, and dispatch flow.

Scheduler DBJitterDispatch Flow

Retry strategy with exponential backoff, dead letter queue design, and failure handling.

RetryDLQBackoffFailure Handling

Per-provider rate limiting to avoid throttling APNs, FCM, and SMS gateways.

APNsFCMProvider Throttle

Fault Isolation

APNs down, Cassandra down, Kafka down, Redis down, and Scheduler down — each failure isolated.

APNs DownKafka DownCassandra DownCircuit Breaker