Redis vs Kafka — When to Use Which
One concrete application of when to introduce what: when do you add Redis, when do you add Kafka, and when do you use both?
Decision table
Section titled “Decision table”| Decision signal | Choose Redis when… | Choose Kafka when… |
|---|---|---|
| Primary problem | You need faster reads / shared quick state | You need async workflows / decoupling |
| Data nature | Current state (latest value matters) | Event history (sequence matters) |
| Typical use | Cache, session store, rate limiting, locks, counters | Event bus, audit log, fan-out to many consumers |
| Latency goal | Sub-ms to a few ms for lookups | Low ms but not “cache-fast”; throughput-focused |
| Durability | Nice-to-have; can tolerate cache loss (or persist separately) | Often required; events must survive restarts/outages |
| Fan-out | Limited (pub/sub exists but not replay-friendly) | Built for many consumer groups independently |
| Replay / backfill | Not a replay system | Core feature: replay, reprocess, backfill |
| Ordering | Not a core guarantee across distributed usage | Ordering per partition is a core concept |
| Backpressure | Not the main strength | Built-in via consumer lag + offsets |
| Scale pattern | Memory-bound; scale by sharding/cluster | Disk + network-bound; scale via partitions + brokers |
| Failure mode if wrong tool | Cache turns into a fragile “mini-database” | Kafka becomes overkill for simple caching/state |
| Quick rule | ”Make reads cheap" | "Make work async + scalable” |
Common “both” scenario
Section titled “Common “both” scenario”Very typical pattern:
- Kafka = source of truth for events
- Redis = serving layer for fast reads / counters / rate limits / derived state
Quick reference: Redis
Section titled “Quick reference: Redis”One-liner: Redis is for fast access to shared state to reduce latency and offload databases.
Use cases: Caching hot reads, sessions, rate limiting, distributed locks, counters, leaderboards.
Key properties: In-memory, very low latency, supports TTL, simple data structures, optional persistence.
Tradeoffs / pitfalls: Cache invalidation, memory cost, data loss risk if treated as source-of-truth, hotkey issues.
Scaling lever: Sharding/cluster, TTL tuning, avoid large values, mitigate hot keys.
When to add it: DB is the bottleneck for reads or you need fast shared state (sessions/limits).
Quick reference: Kafka
Section titled “Quick reference: Kafka”One-liner: Kafka is for reliable event-driven systems: decouple services, scale consumers, and enable replay.
Use cases: Async processing, event bus, CDC pipelines, analytics streams, audit logs, fan-out to many teams.
Key properties: Durable log, partitions + offsets, consumer groups, replay, high throughput.
Tradeoffs / pitfalls: Operational complexity, partitioning strategy, schema evolution, exactly-once complexity, lag management.
Scaling lever: More partitions (parallelism), more brokers, tune batch size, manage consumer lag, right retention.
When to add it: Multiple consumers need the same stream, background workflows grow, or replay/backfill becomes a requirement.
One-line decision justifications
Section titled “One-line decision justifications”Use these for fast justification in design discussions or docs:
- Redis if the problem is latency / read amplification.
- Kafka if the problem is coupling / throughput / fan-out / replay.
- Kafka for history, Redis for state.
- Kafka moves work off the request path; Redis speeds up the request path.
Back to Optimization Quick Reference or Infrastructure building blocks.