Kafka won the event-streaming category so thoroughly that its protocol is now more durable than its implementation — half the interesting alternatives speak Kafka's wire format with a different engine underneath. This page maps the real choices: what each platform optimizes for, what it costs operationally, and the decision rules that actually separate them. Deep dives live on the per-platform pages.
| Platform | Model | Differentiator | Watch out for |
|---|---|---|---|
| Apache Kafka | Partitioned log | Ecosystem gravity: Connect, Streams, every integration ever written; KRaft-only since 4.0 (ZooKeeper gone) | Ops burden at scale; inter-AZ replication cost in cloud |
| Apache Pulsar | Log + segmented storage (BookKeeper) | Compute/storage separation, native multi-tenancy, tiered storage, queues and streams in one system | More moving parts; smaller ecosystem and talent pool |
| Redpanda | Kafka-compatible, C++ | Single binary, no JVM/ZooKeeper, thread-per-core performance, low tail latency | Source-available (BSL), not OSI open source |
| WarpStream | Kafka-compatible, object-storage-native | Stateless agents writing directly to S3 — ~10x cheaper for high-volume relaxed-latency workloads | Hundreds-of-ms latency floor; Confluent-owned (acquired 2024), BYOC-commercial |
| NATS + JetStream | Lightweight pub/sub + persistence | Tiny footprint, millions of subjects, edge/IoT-friendly; Apache-2.0 in CNCF | Not a data-pipeline ecosystem; smaller durability track record at Kafka scale |
| RabbitMQ | Broker with smart routing (AMQP) | Per-message acks, routing topology, task-queue semantics; quorum queues + a stream type | A queue at heart — replay/retention/ordering are not its native shape |
| Cloud-native (Kinesis, Pub/Sub, Event Hubs) | Managed logs | Zero ops, IAM-integrated | Vendor lock-in, per-throughput pricing, feature lag vs Kafka API |
1. Do you need the log abstraction at all? Replayable, ordered, retained history consumed independently by many readers — that is the log's exclusive value (queue vs log is the fundamental split). Task distribution, RPC, and routing-heavy messaging are queue problems; picking Kafka for them buys operational weight for semantics you'll fight.
2. What is your latency/cost point? The 2024-2026 architectural shift is object-storage-native streaming: WarpStream (and Confluent's Freight, Bufstream, and Kafka's own KIP work on tiered/diskless modes) trade end-to-end latency (hundreds of milliseconds) for an order-of-magnitude cost cut by replacing replicated local disks and inter-AZ traffic with S3. Logging, observability, and lake-feeding pipelines rarely need double-digit-millisecond delivery — that workload class is migrating. Interactive/transactional streams stay on disk-based brokers (Kafka, Redpanda — the low-tail-latency specialist).
3. Who operates it? Kafka's ecosystem is unmatched, but so is its operational surface. Honest orderings: smallest ops burden — managed cloud or NATS; smallest self-hosted Kafka-compatible burden — Redpanda (one binary) or post-4.0 KRaft Kafka; richest multi-tenant platform for a central team — Pulsar. The protocol-compatibility wave means you can defer the engine decision: write to the Kafka API, and the engine underneath becomes an infrastructure choice, swappable as economics change.