Kafka Interview Questions and Answers (2026)

Kafka interview questions test whether you understand the partitioned-log model underneath the producer/consumer API — ordering guarantees, rebalancing, and delivery semantics are where most candidates get tripped up. Here are the ones that come up most.

1. Explain topics, partitions, and offsets.

   A topic is a named, logical stream of records (e.g., order-events), analogous to a table or a category of messages. Each topic is split into one or more partitions, which are the actual unit of parallelism and ordering — each partition is an append-only, ordered log. An offset is a monotonically increasing integer identifying each record's position within its partition, unique per partition (not globally across the topic). Ordering is only guaranteed within a single partition, not across the whole topic, which is why the choice of partition count and partitioning strategy matters so much. More partitions allow more consumer parallelism (up to one active consumer per partition within a consumer group) but also mean more open file handles/replication overhead on the brokers and slightly higher end-to-end latency for certain operations, so partition count is a real capacity-planning decision, not just "more is always better."

2. Describe the producer/consumer model in Kafka.

   Producers publish records to a topic, optionally specifying a key that determines which partition the record lands in (via a hash of the key, by default). Consumers subscribe to topics and pull records (Kafka is pull-based, not push-based, unlike many traditional message brokers), tracking their own position via offsets, which lets a slow consumer fall behind without the broker needing to hold back or push at a forced rate. Producers can configure acknowledgment levels (acks=0 fire-and-forget, acks=1 leader-only ack, acks=all full ISR ack) trading off latency against durability.

   acks=all
   retries=3
   enable.idempotence=true

   Because Kafka retains messages for a configured period rather than deleting them on consumption, multiple independent consumers (or consumer groups) can read the same topic at their own pace without interfering with each other — a key difference from traditional queues where a message is typically consumed once and removed.

3. What are consumer groups, and what happens during a rebalance?

   A consumer group is a set of consumers that cooperatively consume a topic, with Kafka guaranteeing each partition is assigned to exactly one consumer within the group at a time — this is how Kafka achieves parallel consumption while preserving per-partition ordering. If you have more consumers than partitions, the excess consumers sit idle. A rebalance is triggered when group membership changes — a consumer joins, leaves, crashes (misses a heartbeat), or partition count changes — and it's the process of reassigning partitions across the current set of consumers. During a rebalance (with the traditional eager protocol), all consumers in the group stop processing while partitions are revoked and reassigned, causing a brief pause; newer cooperative/incremental rebalancing protocols reduce this by only reassigning the specific partitions that need to move, rather than a full stop-the-world reshuffle. Frequent rebalancing (e.g., from consumers falling behind and appearing dead) is a common production pain point worth tuning session.timeout.ms and max.poll.interval.ms for.

4. Explain replication factor and ISR (in-sync replicas).

   Replication factor is the number of copies of each partition maintained across different brokers for fault tolerance — a replication factor of 3 means each partition has one leader and two followers on separate brokers, so the topic survives up to 2 broker failures without data loss. The leader handles all reads/writes for the partition; followers replicate from the leader. ISR (In-Sync Replicas) is the subset of replicas (including the leader) that are fully caught up with the leader within a configured lag threshold — only ISR members are eligible for leader election if the current leader fails, which is what prevents data loss on failover (electing a stale replica would silently lose committed messages). The min.insync.replicas setting works with acks=all to define the durability guarantee: with replication factor 3 and min.insync.replicas=2, a write is only acknowledged once at least 2 replicas have it, tolerating one broker failure without data loss or write unavailability.

5. Compare exactly-once, at-least-once, and at-most-once delivery semantics.

   At-most-once means a message might be lost but is never processed twice — achieved by committing the consumer offset before processing, so a crash after commit but before processing loses that message. At-least-once means a message is never lost but might be processed more than once — achieved by processing first, then committing the offset, so a crash after processing but before commit causes reprocessing on restart; this is the most common default and requires idempotent downstream processing to be safe. Exactly-once semantics (EOS) guarantees each message affects the final state exactly once despite failures/retries, achieved in Kafka via idempotent producers (enable.idempotence=true, deduplicating retried sends via sequence numbers) combined with transactional writes across the read-process-write cycle (isolation.level=read_committed on the consumer side, transactional producer for output). EOS has real overhead and complexity, so many systems intentionally choose at-least-once plus idempotent consumers as a simpler, "good enough" alternative.

6. How does Kafka differ from a traditional message queue like RabbitMQ?

   RabbitMQ is a traditional broker built around the AMQP model of exchanges and queues — messages are typically pushed to consumers, and once acknowledged/consumed they're removed from the queue; it excels at complex routing (topic/fanout/direct exchanges), per-message priority, and low-latency task distribution for smaller-scale workloads. Kafka is a distributed, partitioned commit log designed for high-throughput streaming — messages persist for a configured retention period regardless of consumption, allowing multiple independent consumer groups to replay the same data at different times, and it scales horizontally to very high throughput (millions of messages/sec) via partitioning. Choose RabbitMQ for classic task-queue/work-distribution patterns needing flexible routing and lower operational complexity at moderate scale; choose Kafka when you need durable event history/replay, very high throughput, stream processing (joining/aggregating streams), or multiple downstream systems consuming the same event stream independently, such as feeding both a search index and an analytics pipeline from one event source.

7. How does the partition key affect ordering guarantees?

   Kafka only guarantees ordering within a single partition, not across an entire topic. When a producer sends a record with a key, Kafka's default partitioner hashes that key to consistently route all records with the same key to the same partition, which means all events for a given key (e.g., a specific orderId or userId) are strictly ordered relative to each other. Records with different keys may land on different partitions and have no ordering relationship to each other at all. This has a direct design implication: if you need ordering guarantees for a given entity's event stream (e.g., all state-change events for a specific order must be processed in order), you must key by that entity's ID; sending with no key (null) results in round-robin or sticky partitioning with no ordering guarantee at all across the topic. This tradeoff between parallelism (more partitions, more throughput) and ordering scope (per-key only) is one of the most important Kafka design decisions.

8. How does Kafka's retention policy work?

   Kafka retains messages in a partition log for a configured period or size, independent of whether they've been consumed — controlled via retention.ms (time-based, default 7 days) and/or retention.bytes (size-based per partition), whichever limit is hit first triggers deletion of the oldest segments. This is fundamentally different from a traditional queue where a message disappears once acknowledged. Alternatively, a topic can be configured with cleanup.policy=compact instead of (or alongside) time-based deletion, which retains only the latest value for each key indefinitely, deleting older records with the same key — ideal for topics representing current-state snapshots (e.g., a changelog topic backing a KTable) rather than an event history.

   retention.ms=604800000
   cleanup.policy=delete

   Retention design matters directly for replay capability (how far back can a new consumer or reprocessing job go) and storage cost, and should be tuned per topic based on its actual use case.

9. What is Kafka Connect, at a high level?

   Kafka Connect is a framework for reliably streaming data between Kafka and external systems (databases, search indexes, cloud storage, other message systems) without writing custom producer/consumer code for every integration. It runs Source connectors (pulling data into Kafka, e.g., Debezium capturing database change-data-capture events into a topic) and Sink connectors (pushing data out of Kafka into a target system, e.g., writing to Elasticsearch or S3), managed as a distributed, fault-tolerant cluster of workers that handles offset tracking, scaling, and failure recovery automatically. It's valuable because these integration patterns are common enough that a rich ecosystem of pre-built, configuration-driven connectors exists (Debezium, JDBC, S3, Elasticsearch, etc.), meaning most integrations require only a JSON configuration rather than custom code — for example, a JDBC source connector config just needs a connection string, table, and polling mode to start streaming database rows into a topic.

10. What is Kafka Streams, at a high level?

    Kafka Streams is a client library (not a separate cluster/infrastructure — it runs embedded within your own application) for building stream-processing applications directly on top of Kafka topics, supporting operations like filtering, mapping, aggregating, windowing, and joining streams, including stateful operations backed by local state stores (RocksDB) that are themselves backed by compacted Kafka topics for fault tolerance. It provides two APIs: the higher-level DSL (KStream/KTable abstractions for declarative stream/table operations) and a lower-level Processor API for full control. A KTable represents a continuously updated table view materialized from a stream's latest values per key, which is the bridge between "stream" and "table" thinking in Kafka Streams' stream-table duality. It's the right tool when you need real-time transformations or aggregations on Kafka data without standing up a separate processing cluster like Flink or Spark Streaming, keeping the entire pipeline within the Kafka ecosystem.

11. How does a consumer commit offsets, and what are the tradeoffs of different commit strategies?

    Committing an offset tells Kafka (via an internal __consumer_offsets topic) that the consumer has successfully processed up through that point, so on restart or rebalance it resumes from there rather than from the beginning or missing data. Auto-commit (enable.auto.commit=true, the default) periodically commits offsets in the background at a fixed interval, which is simple but risks reprocessing (if a crash happens between the last auto-commit and actually finishing processing a batch) or, less commonly configured, message loss. Manual commit gives explicit control: commitSync() blocks until the commit is acknowledged (safer, slower) while commitAsync() doesn't block but requires careful error handling since it won't automatically retry on failure (retrying blindly risks committing an out-of-order/stale offset). Best practice for at-least-once processing is to fully process a batch of records, then manually commit synchronously right after — trading a bit of throughput for a much clearer, controllable failure/reprocessing boundary than relying on auto-commit's fixed interval.

12. When is Kafka the wrong choice for a problem?

    Kafka is overkill and operationally heavy for simple, low-volume task queues where a lightweight broker like RabbitMQ or even a cloud queue (SQS, Azure Storage Queues) would be simpler to run and reason about — Kafka's operational complexity (Zookeeper/KRaft, partition rebalancing, broker sizing, monitoring) isn't justified without genuinely high throughput or replay/streaming needs. It's also a poor fit when you need strict, complex per-message routing logic (priority queues, content-based routing to many different consumers with different rules) since Kafka's model is fundamentally a partitioned log, not a flexible routing engine. It's not ideal for very large individual messages (multi-MB+ payloads) — better to put large payloads in blob storage and pass a reference through Kafka. And for use cases needing strict single-message request/reply patterns with low latency and simple point-to-point semantics, a simpler RPC or lightweight queue avoids Kafka's added complexity for no real benefit. In short: reach for Kafka when you specifically need durable replay, very high throughput, or multi-consumer event streaming — not by default.

More prep guides on the AI Interview Bot blog.