← Back to Blog

Kafka Explained: How It Works and Why It Matters

By

Modern systems rarely live in isolation. An order service may need to notify a payment service, update inventory, trigger analytics, send a confirmation email, and inform a fraud-detection system as part of a single business flow.

The simplest way to connect services is synchronous communication.

flowchart TD
    order["Order Service"]
    payment["Payment Service"]

    order -->|"  HTTP  "| payment

That works well when the caller needs an immediate response. But not every interaction needs to happen immediately, and not every downstream dependency should be able to affect the original request.

If the payment service is temporarily unavailable, should the order service fail too? If analytics processing takes a few seconds, should the user wait for it? Questions like these led distributed systems toward asynchronous messaging.

From Direct Calls to Messaging

Instead of one service directly calling another, the producer can publish a message and continue processing.

flowchart TD
    order["Order Service"]
    payment["Payment Service"]

    order -->|"  Message Queue  "| payment

Message queues solved several important problems. Producers and consumers became less tightly coupled, consumers could process work at their own pace, and temporary failures no longer had to immediately affect the producer.

For many systems, this model is still perfectly adequate. But as distributed architectures became larger and more event-driven, engineers started asking for more than simple message delivery.

What if several independent systems need the same event? What if we need to process yesterday’s events again? What if ordering matters? What if we need to process a very high volume of events while scaling consumers horizontally?

This is where Apache Kafka becomes particularly interesting.

A Different Mental Model

Kafka is often introduced as a messaging system. That description is not wrong, but it can lead to the wrong mental model.

A more useful way to think about Kafka is as a distributed, durable log of events.

Producers append events to that log, Kafka stores them for a configured period of time, and consumers independently move through the log at their own pace. Once this mental model makes sense, concepts such as partitions, offsets, and consumer groups become much easier to understand.

The Basic Kafka Architecture

At a high level, Kafka consists of a few important building blocks:

A simplified Kafka setup looks like this:

flowchart LR
    producer["Producer"]

    subgraph kafka["Kafka Cluster"]
        direction LR

        broker1["Broker 1<br/>Partition P0"]
        broker2["Broker 2<br/>Partition P1"]
        broker3["Broker 3<br/>Partition P2"]
    end

    consumerA["Consumer A"]
    consumerB["Consumer B"]
    consumerC["Consumer C"]

    producer -->|"&nbsp;&nbsp;Publish&nbsp;&nbsp;"| broker1
    producer -->|"&nbsp;&nbsp;Publish&nbsp;&nbsp;"| broker2
    producer -->|"&nbsp;&nbsp;Publish&nbsp;&nbsp;"| broker3

    broker1 -->|"&nbsp;&nbsp;Read&nbsp;&nbsp;"| consumerA
    broker2 -->|"&nbsp;&nbsp;Read&nbsp;&nbsp;"| consumerB
    broker3 -->|"&nbsp;&nbsp;Read&nbsp;&nbsp;"| consumerC

The important detail here is that Kafka is normally a cluster of brokers rather than one central server. Topics are split into partitions, and those partitions are distributed across the cluster.

What Exactly Is a Kafka Broker?

A broker is a Kafka server participating in a Kafka cluster. A production Kafka cluster usually contains multiple brokers so that storage and traffic can be distributed across several machines.

Imagine a topic called order-events with three partitions:

flowchart TB
    topic["Topic: order-events"]

    subgraph cluster["Kafka Cluster"]
        direction LR
        broker1["Broker 1<br/><br/>Partition P0"]
        broker2["Broker 2<br/><br/>Partition P1"]
        broker3["Broker 3<br/><br/>Partition P2"]
    end

    topic --> broker1
    topic --> broker2
    topic --> broker3

Each partition is stored on a broker, and in real production environments those partitions are normally replicated across other brokers for fault tolerance.

This distribution is one of the reasons Kafka scales well. Different partitions can be served by different brokers, so the entire workload does not have to pass through one central machine.

How Producers and Consumers Find the Right Broker

Kafka applications normally start with a small list of bootstrap servers:

bootstrap.servers=kafka-1:9092,kafka-2:9092,kafka-3:9092

These are broker addresses used as initial entry points to discover the Kafka cluster. After discovery, the application communicates directly with the brokers responsible for the partitions it needs.

For example, a producer may first connect to Broker 1. That broker returns metadata describing the cluster, including which broker is currently the leader for each partition.

sequenceDiagram
    participant P as Producer
    participant B1 as Broker 1
    participant B2 as Broker 2
    participant B3 as Broker 3

    P->>B1: Initial bootstrap connection

    B1-->>P: Cluster metadata<br/>P0 leader → Broker 1<br/>P1 leader → Broker 2<br/>P2 leader → Broker 3

    Note over P: Producer caches the metadata<br/>and determines the target partition

    P->>B2: Send event for Partition 1
    B2-->>P: Acknowledgement

    Note over P,B3: Metadata is periodically refreshed<br/>to discover broker or partition leader changes

Once the producer has this metadata, it can communicate with the broker responsible for the target partition. Consumers use the same general idea when discovering where they need to read from.

Kafka clients periodically refresh this metadata, which allows them to react when brokers or partition leaders change.

Topics and Partitions

A Kafka topic is a logical stream of related events. You might have topics such as order-events, payment-events, or customer-events.

Kafka does not store an entire topic as one giant ordered list. Instead, each topic is split into partitions.

flowchart TD
    topic["Topic: order-events"]

    p0["Partition 0"]
    p1["Partition 1"]
    p2["Partition 2"]

    topic --> p0
    topic --> p1
    topic --> p2

Each partition behaves like an ordered append-only log. New records are added to the end, and every record receives an offset that identifies its position within that partition.

flowchart LR
    r0["Offset 0<br/>Record 1"]
    r1["Offset 1<br/>Record 2"]
    r2["Offset 2<br/>Record 3"]
    r3["Offset 3<br/>Record 4"]
    r4["Offset 4<br/>Record 5"]
    next["New records append here"]

    r0 --> r1 --> r2 --> r3 --> r4 --> next

Offsets are local to a partition. For example, Partition 0 can have an offset 10, while Partition 1 can independently have its own offset 10.

Ordering in Kafka

One of Kafka’s most useful properties is ordered processing, but the guarantee is more specific than it first appears.

Kafka guarantees ordering within a partition, not across an entire topic.

Suppose a topic has three partitions. Kafka can process those partitions independently and in parallel, so there is no global ordering guarantee across all three.

Within a single partition, however, the record sequence is preserved. This becomes especially useful when combined with message keys.

Same Key, Same Partition

A Kafka producer can publish a record with a key.

ProducerRecord<String, OrderEvent> record =
        new ProducerRecord<>(
                "order-events",
                orderId,
                orderEvent
        );

producer.send(record);

Here, orderId is the key. Kafka’s partitioning logic uses that key when deciding which partition should receive the record.

With Kafka’s standard key-based partitioning, records with the same key are routed to the same partition as long as the topic’s partition count remains unchanged. So if all events for Order-123 use Order-123 as the key, the flow may look like this:

flowchart TB
    e1["OrderCreated<br/>Key: Order-123"]
    e2["PaymentCompleted<br/>Key: Order-123"]
    e3["OrderShipped<br/>Key: Order-123"]

    partitioner["Kafka Partitioner"]

    note["Same key → same partition"]

    subgraph p2["Partition 2"]
        direction LR
        r1["Offset 20<br/>OrderCreated"]
        r2["Offset 21<br/>PaymentCompleted"]
        r3["Offset 22<br/>OrderShipped"]

        r1 --> r2 --> r3
    end

    e1 --> partitioner
    e2 --> partitioner
    e3 --> partitioner

    partitioner --> note
    note --> p2

    style note stroke-dasharray: 5 5,stroke-width: 1.5px,fill: transparent

Because all three events are in the same partition, their sequence is preserved. This is extremely useful when later events only make sense after earlier events have been processed.

Using the order ID as the Kafka key helps preserve that ordering for all events belonging to the same order.

Ordering Comes With a Trade-Off

Choosing a Kafka key is not only a data-modelling decision. It also affects how load is distributed across partitions.

Imagine using country as the key. If most traffic comes from one country, one partition may receive a disproportionate amount of the workload.

flowchart TB
    key["Kafka Key: country"]

    subgraph partitions["Topic Partitions"]
        direction LR

        p0["Partition 0<br/>████████████████████████<br/>High traffic"]
        p1["Partition 1<br/>████<br/>Low traffic"]
        p2["Partition 2<br/>███<br/>Low traffic"]
        p3["Partition 3<br/>████<br/>Low traffic"]
    end

    hot["Hot Partition<br/>Potential bottleneck"]

    key --> partitions
    p0 --> hot

    style p0 stroke-width: 3px
    style hot stroke-dasharray: 5 5,fill: transparent

This is known as a hot partition.

Because a partition can only be processed by one consumer within a consumer group at a time, a hot partition can become a bottleneck even when the rest of the cluster has spare capacity.

A useful design rule is therefore:

Choose Kafka keys based on both ordering requirements and expected data distribution.

Consumers and Consumer Groups

Consumers read records from Kafka topics. Kafka becomes much more interesting when those consumers are organised into consumer groups.

Suppose an order-events topic has three partitions. With a single consumer, that consumer can be assigned all three partitions.

flowchart TB
    subgraph topic["Topic: order-events"]
        direction LR
        p0["Partition 0"]
        p1["Partition 1"]
        p2["Partition 2"]
    end

    consumer["Consumer A"]

    p0 --> consumer
    p1 --> consumer
    p2 --> consumer

If more consumers join the same group, Kafka distributes the partitions between them.

flowchart TB
    subgraph topic["Topic: order-events"]
        direction LR
        p0["Partition 0"]
        p1["Partition 1"]
        p2["Partition 2"]
    end

    subgraph group["Consumer Group"]
        direction LR
        c0["Consumer A"]
        c1["Consumer B"]
        c2["Consumer C"]
    end

    p0 --> c0
    p1 --> c1
    p2 --> c2

This is how Kafka achieves consumer-side parallelism.

Partition Count Controls Consumer Parallelism

There is one rule that is particularly important when designing Kafka consumers:

Inside the same consumer group, a partition can be assigned to only one consumer at a time.

Consider a topic with three partitions and a consumer group with four consumers.

flowchart TB
    subgraph topic["Topic: order-events"]
        direction LR
        p0["Partition 0"]
        p1["Partition 1"]
        p2["Partition 2"]
    end

    subgraph group["Consumer Group"]
        direction LR
        cA["Consumer A"]
        cB["Consumer B"]
        cC["Consumer C"]
        cD["Consumer D<br/>(Idle)"]
    end

    p0 --> cA
    p1 --> cB
    p2 --> cC

    style cD stroke-dasharray: 5 5,fill: transparent

The fourth consumer has nothing to process because all partitions are already assigned.

In practical terms:

Maximum active consumers for a topic within one consumer group = number of partitions

Adding more consumers than partitions does not increase consumption throughput. The partition count sets an upper bound on how much parallelism a single consumer group can achieve.

Kafka Consumer Concurrency in Spring

This becomes particularly relevant when using Spring Kafka.

@KafkaListener(
        topics = "order-events",
        groupId = "payment-service",
        concurrency = "3"
)
public void consume(OrderEvent event) {
    process(event);
}

Conceptually, Spring creates multiple Kafka consumer instances for the listener container.

If the topic has six partitions and the listener concurrency is three, Kafka may distribute the partitions roughly like this:

flowchart TB
    listener["Spring @KafkaListener<br/>concurrency = 3"]

    subgraph consumers["Listener Container"]
        direction LR
        c1["Consumer 1"]
        c2["Consumer 2"]
        c3["Consumer 3"]
    end

    subgraph topic["Topic: order-events — 6 Partitions"]
        direction LR
        p0["P0"]
        p1["P1"]
        p2["P2"]
        p3["P3"]
        p4["P4"]
        p5["P5"]
    end

    listener --> consumers

    c1 --> p0
    c1 --> p3

    c2 --> p1
    c2 --> p4

    c3 --> p2
    c3 --> p5

The exact assignment is controlled by Kafka and may differ, but the important point is that all six partitions can be shared across those three consumer instances.

Now imagine the topic has only three partitions while the listener is configured with a concurrency of ten. Only three consumers can actively receive partitions, while the rest remain idle.

flowchart TB
    listener["Spring @KafkaListener<br/>concurrency = 10"]

    subgraph topic["Topic: order-events | 3 Partitions"]
        direction LR
        p0["P0"]
        p1["P1"]
        p2["P2"]
    end

    subgraph active["Active Consumers"]
        direction LR
        c1["Consumer 1"]
        c2["Consumer 2"]
        c3["Consumer 3"]
    end

    idle["Consumers 4–10<br/>Idle"]

    listener --> active
    listener --> idle

    p0 --> c1
    p1 --> c2
    p2 --> c3

    style idle stroke-dasharray: 5 5,fill: transparent

So increasing the listener concurrency does not automatically make processing faster. The topic must have enough partitions to support the desired level of parallelism.

Multiple Consumer Groups

Kafka also allows several independent consumer groups to read the same topic.

Imagine an OrderPlaced event that is useful to several systems.

The payment service, analytics service, and notification service can each use their own consumer group.

Each group consumes independently and maintains its own position in the topic.

flowchart TB
    topic["Topic: order-events<br/>OrderPlaced"]

    subgraph groups["Consumer Groups"]
        direction LR

        pg["payment-service-group<br/>Offset 1500"]
        ag["analytics-service-group<br/>Offset 1432"]
        ng["notification-service-group<br/>Offset 1498"]
    end

    payment["Payment Service"]
    analytics["Analytics Service"]
    notification["Notification Service"]

    topic --> pg
    topic --> ag
    topic --> ng

    pg --> payment
    ag --> analytics
    ng --> notification

If the analytics service falls behind, it does not prevent the payment or notification services from continuing. Each group moves through the event stream at its own pace.

Kafka Does Not Remove an Event Because You Read It

This is one of the most important conceptual differences to understand.

Imagine a partition containing these offsets:

flowchart LR
    subgraph partition["Partition 1"]
        direction LR

        o100["Offset 100<br/>Record 1"]
        o101["Offset 101<br/>Record 2"]
        o102["Offset 102<br/>Record 3"]
        o103["Offset 103<br/>Record 4"]
        o104["Offset 104<br/>Record 5<br/>▲<br/>(Next to read)"]
        o105["Offset 105<br/>Record 6"]

        o100 --> o101 --> o102 --> o103 --> o104 --> o105
    end

Kafka does not remove the records at offsets 100 through 103 simply because that consumer has processed them. Those records remain according to the topic’s retention policy.

The consumer keeps track of the position from which it should continue consuming.

That design enables one of Kafka’s most powerful capabilities: replay.

Replaying Events

Imagine an analytics service contained a bug and calculated customer metrics incorrectly for six hours. The events have already been processed, but Kafka may still retain them.

If so, the consumer can move its position backwards and process those records again.

Reset the consumer position:

flowchart LR
    subgraph partition["Partition 1"]
        direction LR

        o100["Offset 100<br/>Record 1"]
        o101["Offset 101<br/>Record 2<br/>▲<br/>(Replay from here)"]
        o102["Offset 102<br/>Record 3"]
        o103["Offset 103<br/>Record 4"]
        o104["Offset 104<br/>Record 5"]
        o105["Offset 105<br/>Record 6<br/>△<br/>(Previous position)"]

        o100 --> o101 --> o102 --> o103 --> o104 --> o105
    end

The consumer can now replay those records.

The previous position at offset 105 represents the next record to read. Resetting the position to offset 101 causes the consumer to replay the records it had already processed from 101 through 104, then continue with 105.

Replay is useful for scenarios such as rebuilding projections, recalculating analytics, recovering from consumer bugs, rebuilding search indexes, and reprocessing failed events.

However, replay needs to be designed carefully. Reprocessing an analytics event is very different from reprocessing an event that sends an email, charges a payment method, or calls an external system.

Kafka gives you the ability to replay events, but your application still needs to ensure replay is safe. This is one reason idempotency is so important in event-driven systems.

Pause and Resume

Sometimes a Kafka consumer is healthy while one of its downstream dependencies is not.

Continuing to consume records aggressively may make the situation worse. Kafka consumers can temporarily pause assigned partitions.

At the low-level Kafka consumer API, the idea looks like this:

consumer.pause(consumer.assignment());

Pausing does not delete records or move the consumer forward. It simply prevents the consumer from fetching more records from those partitions temporarily.

Once the downstream dependency becomes available again, consumption can be resumed:

consumer.resume(consumer.assignment());
flowchart LR
    kafka["Kafka<br/>Partitions"]
    consumer["Consumer"]
    apiDown["External API<br/>Unavailable"]
    paused["⏸ Consumer Paused"]
    apiUp["External API<br/>Available"]
    resumed["▶ Consumer Resumed"]

    kafka --> consumer
    consumer -.-> apiDown
    apiDown --> paused
    paused --> apiUp
    apiUp --> resumed
    resumed --> consumer

    style apiDown stroke-dasharray: 5 5,fill: transparent
    style paused stroke-dasharray: 5 5,fill: transparent

This can be useful during downstream outages, temporary rate limiting, controlled maintenance, or backpressure situations. Frameworks such as Spring Kafka provide higher-level ways to manage pause and resume, so applications may not always interact with the raw consumer API directly.

What Happens When Consumers Come and Go?

Suppose a consumer group initially has two consumers.

If another consumer joins the group, Kafka redistributes the partitions.

This process is known as a consumer group rebalance.

The same thing happens when a consumer disappears. If Consumer B crashes, Kafka eventually detects that it is no longer part of the group and reassigns its partition to another active consumer.

flowchart LR
    subgraph before["Initial Consumer Group"]
        a1["Consumer A<br/>P0, P1"]
        b1["Consumer B<br/>P2, P3"]

        a1 ~~~ b1
    end

    join["Consumer C Joins<br/>Rebalance"]

    subgraph afterJoin["After Rebalance"]
        a2["Consumer A<br/>P0"]
        b2["Consumer B<br/>P1"]
        c2["Consumer C<br/>P2, P3"]

        a2 ~~~ b2
        b2 ~~~ c2
    end

    crash["Consumer B Crashes<br/>Rebalance"]

    subgraph afterCrash["After Rebalance"]
        a3["Consumer A<br/>P0, P1"]
        c3["Consumer C<br/>P2, P3"]

        a3 ~~~ c3
    end

    before --> join
    join --> afterJoin
    afterJoin --> crash
    crash --> afterCrash

    style join stroke-dasharray: 5 5,fill: transparent
    style crash stroke-dasharray: 5 5,fill: transparent

This is what allows Kafka consumer applications to scale horizontally and recover from individual consumer failures without manually redistributing work.

Rebalancing itself has many interesting behaviours and tuning options, but that is a topic for another article.

What Happens When a Broker Fails?

Kafka also distributes replicas of partitions across brokers for fault tolerance.

Producers and consumers normally communicate with the broker responsible for the relevant partition leader.

If Broker 1 fails, Kafka can elect an eligible replica as the new leader.

flowchart LR
    subgraph before["Before Broker Failure"]
        direction TB

        b1["Broker 1<br/><br/>P0 Leader"]
        b2["Broker 2<br/><br/>P0 Replica"]

        b1 ~~~ b2
    end

    failure["Broker 1 Fails<br/>Leader Election"]

    subgraph after["After Leader Election"]
        direction TB

        b1f["Broker 1<br/><br/>Unavailable"]
        b2l["Broker 2<br/><br/>P0 Leader"]

        b1f ~~~ b2l
    end

    metadata["Client Refreshes<br/>Cluster Metadata"]
    continue["Continue with<br/>New Leader"]

    before --> failure
    failure --> after
    after --> metadata
    metadata --> continue

    style failure stroke-dasharray: 5 5,fill: transparent
    style b1f stroke-dasharray: 5 5,fill: transparent

Kafka clients refresh their cluster metadata and discover that the partition leader has changed. This allows processing to continue without every application having to manually find another Kafka server.

Replication, leader election, acknowledgements, and Kafka’s internal fault-tolerance mechanisms go much deeper than this, but the important mental model is simple: partitions are distributed and replicated across brokers so that the Kafka cluster does not depend on one machine.

Partition Count Is an Architectural Decision

Suppose a topic has three partitions and your application has three consumers. All three consumers can be actively processing at the same time.

Now traffic increases significantly and you deploy ten consumers. If the topic still has only three partitions, only three of those consumers can actively consume from that topic.

Partitions are therefore not simply an internal Kafka detail. They directly influence maximum consumer parallelism, throughput, ordering, load distribution, and scalability.

It is possible to increase a topic’s partition count later, but that also deserves some thought. Kafka commonly chooses a partition by hashing the record key and then mapping that result across the topic’s available partitions. If the number of partitions changes, the same key may map to a different partition for future records.

For example, imagine all events for Order-123 have historically been written to Partition 2. After increasing the topic from three partitions to six, new records for Order-123 may start going to Partition 4 instead.

flowchart LR
    subgraph before["Before: 3 Partitions"]
        direction TB

        created["OrderCreated<br/>Key: Order-123"]
        paid["OrderPaid<br/>Key: Order-123"]
        p2a["Partition 2"]

        created --> p2a
        paid --> p2a
    end

    change["Partition Count<br/>3 → 6"]

    subgraph after["After: 6 Partitions"]
        direction TB

        shipped["OrderShipped<br/>Key: Order-123"]
        p4["Partition 4"]

        shipped --> p4
    end

    before --> change
    change --> after

    style change stroke-dasharray: 5 5,fill: transparent

The records inside each partition are still ordered, but Kafka does not provide ordering across Partition 2 and Partition 4. If consumers are processing both partitions in parallel, Order-123 Shipped could theoretically be processed before an older event for the same order that is still waiting in Partition 2.

This is why partition count should ideally be considered early when ordering by key is important. Increasing partitions is completely valid, but applications that depend on long-lived key-based ordering need to understand that the key-to-partition mapping may change.

When Kafka Starts to Make Sense

Kafka is powerful, but asynchronous communication alone is not necessarily a reason to introduce it.

Kafka becomes particularly useful when several requirements appear together: high event throughput, multiple independent consumers, replayability, ordering within a logical entity, horizontal consumer scaling, or event-driven processing.

For example, a single event may need to be consumed independently by payment processing, inventory, analytics, notifications, and fraud detection.

flowchart LR
    event["OrderPlaced"]

    subgraph consumers["Independent Consumers"]
        direction TB

        payment["Payment Processing"]
        inventory["Inventory"]
        analytics["Analytics"]
        notifications["Confirmation Email"]
        fraud["Fraud Detection"]
    end

    event --> payment
    event --> inventory
    event --> analytics
    event --> notifications
    event --> fraud

Kafka also fits naturally when systems react to facts that have already happened, such as OrderPlaced, PaymentCompleted, CustomerRegistered, or DocumentUploaded.

This is where Kafka’s model becomes especially valuable: producers publish facts, and interested systems decide independently how to process them.

Kafka Is Not Automatically the Right Answer

Kafka comes with operational and architectural complexity, so it should solve a real problem rather than simply be introduced because it is popular.

For a small application that only needs to execute background jobs, a traditional queue may be easier to operate and easier to reason about.

Kafka is also not a replacement for normal HTTP APIs. If a service needs an immediate answer, request-response communication is often still the natural choice.

GET /customer/123

Kafka becomes valuable when its event-stream model actually fits the architecture. Choosing it simply because “we use microservices” or “Kafka scales” can add complexity without providing much benefit.

The Kafka Mental Model to Remember

There are many Kafka concepts we have not covered here: replication factors, in-sync replicas, producer acknowledgements, delivery semantics, idempotent producers, transactions, dead-letter topics, schema evolution, Kafka Streams, KRaft, and different rebalancing strategies.

Those details matter, but they are much easier to understand once the basic model is clear.

Do not imagine Kafka primarily as messages travelling through a pipe. Imagine a distributed log.

flowchart LR
    producer["Producer"]

    subgraph kafka["Kafka Cluster"]
        direction TB

        topic["Topic"]

        subgraph partitions["Partitions"]
            direction LR
            p0["P0"]
            p1["P1"]
            p2["P2"]
        end

        topic --> partitions
    end

    subgraph group["Consumer Group"]
        direction TB
        c1["Consumer A"]
        c2["Consumer B"]
        c3["Consumer C"]
    end

    producer -->|"&nbsp;&nbsp;Append Events&nbsp;&nbsp;"| topic

    p0 --> c1
    p1 --> c2
    p2 --> c3

Producers append events, Kafka stores them, partitions provide ordering and scalability, and brokers distribute those partitions across a cluster.

Consumers independently keep track of where they are, while consumer groups allow processing to scale horizontally. Because consuming an event does not immediately destroy it, consumers can pause, resume, fall behind, recover, and replay history.

Kafka can feel complicated at first because there are many moving parts. But once topics, partitions, brokers, offsets, and consumer groups fit together, the architecture becomes much easier to reason about.

At its core, Kafka gives you a scalable way to publish events, distribute them across a cluster, preserve ordering where it matters, and let multiple consumers process those events independently.

The important part is not knowing every Kafka feature. It is understanding the model well enough to recognise when Kafka fits the problem you are trying to solve.

References