Absolutely. For a Java/Spring Boot senior architect interview, Kafka questions are often asked from basic concepts → architecture → delivery semantics → partitions → consumer groups → failures → performance → real-world design.
Below is an interview-focused set with examples and text-flow diagrams.
Kafka Interview Questions & Answers
1. What is Apache Kafka?
Answer:
Kafka is a distributed event-streaming platform used to publish, store, and consume streams of events/messages at high scale.
Typical architecture:
┌───────────────┐
│ Producer │
│ Spring Boot │
└───────┬───────┘
│
│ Event
▼
┌───────────────┐
│ Kafka │
│ Topic │
└───────┬───────┘
│
▼
┌───────────────┐
│ Consumer │
│ Spring Boot │
└───────────────┘Example:
Order Service
│
│ OrderCreated
▼
Kafka: order-events
│
├──────────────► Payment Service
│
├──────────────► Inventory Service
│
└──────────────► Notification ServiceThe important advantage is that producer and consumers are decoupled.
2. Kafka vs traditional message queue?
A common interview question.
Traditional queue
Producer
│
▼
Queue
│
▼
ConsumerUsually, once the consumer successfully processes the message, the message is removed/acknowledged.
Kafka
Producer
│
▼
Kafka Topic
│
├── Consumer Group A
├── Consumer Group B
└── Consumer Group CKafka stores events for a configured retention period.
For example:
Topic: orders
Offset
0 Order A
1 Order B
2 Order C
3 Order DConsumer A can read:
0 → 1 → 2 → 3Later, another consumer can independently read the same events.
Interview point:
Kafka is not simply a queue. It is a distributed, durable, partitioned event log.
3. What is a Kafka Topic?
A topic is a logical category/name to which producers publish messages.
Example:
order-events
payment-events
customer-events
notification-eventsFlow:
Order Service
│
│ publish()
▼
┌──────────────────────┐
│ Topic: order-events │
└──────────────────────┘
│
├── Payment Consumer
├── Inventory Consumer
└── Notification ConsumerA topic is divided into partitions.
order-events
Partition 0
[0] [1] [2] [3]
Partition 1
[0] [1] [2] [3]
Partition 2
[0] [1] [2] [3]4. What is a Kafka Partition?
A partition is the fundamental unit of parallelism and storage in Kafka.
Example:
Topic: orders
Partition 0
│
├── Order-101
├── Order-104
└── Order-108
Partition 1
│
├── Order-102
├── Order-105
└── Order-109
Partition 2
│
├── Order-103
├── Order-106
└── Order-110Each partition maintains message ordering.
Important interview statement
Kafka guarantees ordering within a partition, not across the entire topic.
5. Why does Kafka use partitions?
Mainly for:
1. Parallelism
Topic
│
┌──────────┼──────────┐
▼ ▼ ▼
P0 P1 P2
│ │ │
▼ ▼ ▼
Consumer-1 Consumer-2 Consumer-3Three consumers can process three partitions simultaneously.
2. Scalability
Partitions can be distributed across multiple Kafka brokers.
Broker 1
└── P0
Broker 2
└── P1
Broker 3
└── P26. What is a Kafka Broker?
A broker is a Kafka server.
A Kafka cluster can contain multiple brokers.
Kafka Cluster
┌────────────┼────────────┐
▼ ▼ ▼
Broker-1 Broker-2 Broker-3
│ │ │
P0 P1 P2Multiple brokers provide:
scalability
fault tolerance
replication
load distribution
7. What is Replication Factor?
Replication factor tells how many copies of a partition Kafka maintains.
Suppose:
Replication Factor = 3Then:
Partition P0
Broker 1 → Leader
Broker 2 → Replica
Broker 3 → ReplicaIf Broker 1 fails:
Broker 1 ❌
Broker 2 → becomes Leader
Broker 3 → ReplicaThis provides fault tolerance.
8. What is a Kafka Leader and Follower?
For each partition, one replica is the leader.
Other replicas are followers.
Partition P0
Broker 1
└── Leader
Broker 2
└── Follower
Broker 3
└── FollowerProducer normally sends writes to the leader.
Producer
│
▼
Broker 1
Leader
│
├────────► Broker 2
│ Follower
│
└────────► Broker 3
FollowerIf the leader fails, Kafka can elect another replica.
9. What is an Offset?
An offset uniquely identifies the position of a record within a partition.
Example:
Partition 0
Offset Message
0 Order-101
1 Order-102
2 Order-103
3 Order-104Offset is maintained per partition.
Important:
P0 → offset 10
P1 → offset 25
P2 → offset 17There is no single global offset for the entire topic.
10. What is a Consumer Group?
A consumer group is a group of consumers that collectively consume a topic.
Example:
Topic
├── P0
├── P1
└── P2
Consumer Group: payment-service
├── Consumer-1 → P0
├── Consumer-2 → P1
└── Consumer-3 → P2Each partition is assigned to only one consumer within a consumer group at a time.
11. Can two consumers in the same group consume the same partition?
Normally No.
Example:
Topic
P0
P1
P2
Consumer Group A
C1 → P0
C2 → P1
C3 → P2But different consumer groups can independently consume the same partition:
Topic
│
▼
P0
/ \
/ \
▼ ▼
Consumer Group A Consumer Group B
│ │
Payment Service Analytics ServiceThis is a very important interview concept.
12. What happens if consumers > partitions?
Suppose:
3 partitions
5 consumersOnly 3 consumers can actively consume.
P0 → Consumer-1
P1 → Consumer-2
P2 → Consumer-3
Consumer-4 → idle
Consumer-5 → idleTherefore:
Maximum active consumers in a consumer group = number of partitions.
13. What happens if consumers < partitions?
Suppose:
3 partitions
2 consumersKafka distributes multiple partitions to consumers.
Consumer-1 → P0 + P1
Consumer-2 → P2Therefore partition count is important when designing consumer scalability.
14. How does Kafka guarantee ordering?
Kafka guarantees ordering inside a partition.
Example:
P0
Offset 0 → Order Created
Offset 1 → Payment Started
Offset 2 → Payment CompletedA consumer reads:
0 → 1 → 2in order.
But:
P0:
Order A
Order B
P1:
Order C
Order DKafka does not guarantee:
A → B → C → Dacross partitions.
15. How do you guarantee ordering for a particular customer/order?
Use a key.
Example:
Producer
│
│ key = customerId
▼
KafkaKafka hashes the key to determine the partition.
customer-101 → P2
customer-101 → P2
customer-101 → P2Therefore:
Customer-101 events
Event 1 → P2
Event 2 → P2
Event 3 → P2Their relative order is preserved.
Interview answer
If ordering is required for an entity, use a stable key such as orderId or customerId so all events for that entity go to the same partition.
16. How does Kafka Producer decide which partition to use?
Common cases:
Key specified
key = customerId
│
▼
hash(customerId)
│
▼
partitionExample:
customer-100 → P0
customer-101 → P2
customer-102 → P1No key
Kafka can distribute records across partitions according to the producer's partitioning behavior/configuration.
17. What is Producer Acknowledgement (acks)?
One of the most common Kafka interview questions.
acks=0
Producer doesn't wait for broker acknowledgement.
Producer ─────────► Kafka
returns immediatelyFast but riskier.
acks=1
Leader acknowledges after writing the record.
Producer
│
▼
Leader
│
│ ACK
▼
ProducerGood balance of performance and durability.
acks=all
Leader waits for the required in-sync replicas to acknowledge.
Producer
│
▼
Leader
│
├────► Replica 1
│
└────► Replica 2
│
▼
ACKHighest durability among these settings, generally with more latency.
18. What is ISR?
ISR = In-Sync Replicas.
Suppose:
P0
Broker 1 → Leader
Broker 2 → ISR
Broker 3 → ISRAll are sufficiently caught up with the leader.
If Broker 3 becomes too far behind:
Broker 1 → Leader
Broker 2 → ISR
Broker 3 → Out of SyncISR is important for:
durability
leader election
replication health
19. What is Consumer Offset Commit?
Consumers need to remember which messages they have processed.
Example:
P0
0
1
2
3
4Consumer processed:
0
1
2It commits its position.
Conceptually:
Committed Offset = 3If consumer crashes:
Consumer
↓
❌New consumer can resume from the committed position.
20. What is Consumer Rebalancing?
Suppose:
P0 → C1
P1 → C2
P2 → C3C2 crashes.
Kafka detects the consumer group membership change and redistributes partitions.
Before:
C1 → P0
C2 → P1
C3 → P2
C2 ❌
After:
C1 → P0 + P1
C3 → P2This is called rebalance.
21. What is at-most-once delivery?
Message can be processed zero or one time, so messages may be lost but are not normally redelivered.
Typical approach:
Read message
│
▼
Commit offset
│
▼
Process messageIf application crashes after commit but before processing:
Commit ✓
Process ❌The message may effectively be lost.
22. What is at-least-once delivery?
Message is processed one or more times.
Typical flow:
Read
│
▼
Process
│
▼
Commit offsetIf processing succeeds but the application crashes before committing:
Process ✓
Commit ❌
CrashAfter restart:
Same message
↓
Processed againSo duplicates are possible.
Interview answer
At-least-once delivery provides stronger durability but requires consumers to be idempotent.
23. What is exactly-once processing?
Exactly-once semantics aim to ensure that an event's effect is applied exactly once within the supported Kafka processing/transaction model.
A simplified flow:
Consumer
│
▼
Process
│
├── Produce result
│
└── Commit offset
│
▼
TransactionKafka supports transactional mechanisms for atomic processing of Kafka records and produced records.
Senior-level interview point:
Don't simply say:
"Kafka guarantees exactly once."
Better:
Kafka provides exactly-once semantics for supported transactional Kafka workflows, but end-to-end exactly-once behavior with an external database or external API requires additional design, such as idempotency or transactional/outbox patterns.
24. What is Idempotency?
An operation is idempotent if executing it multiple times produces the same final result.
Example:
Kafka sends:
OrderId = 101
Payment = ₹1000Consumer receives it twice.
Without idempotency:
₹1000 charged
₹1000 charged
Total = ₹2000 ❌With idempotency:
Payment event 101
│
▼
Check processed_event table
│
├── Already processed → Ignore
│
└── New → ProcessResult:
₹1000 charged ✓25. Kafka Retry Pattern
Suppose payment processing fails.
Kafka
│
▼
Payment Consumer
│
▼
Payment Service
│
X
FailurePossible retry architecture:
┌──────────────┐
│ order-events │
└──────┬───────┘
│
▼
Payment Consumer
│
Failure
│
▼
retry-topic
│
▼
Payment Consumer
│
Failure
│
▼
DLQDLQ = Dead Letter Queue/Topic.
26. What is a Dead Letter Topic?
Messages that repeatedly fail processing can be sent to a DLQ/DLT.
Main Topic
│
▼
Consumer
│
X
Repeated Failure
│
▼
DLTExample:
order-events
│
▼
Order Consumer
│
X
Invalid message
│
▼
order-events.DLTOperations team can inspect and replay the failed events later.
27. How would you design Kafka for high throughput?
I would consider:
Kafka Cluster
┌────────────┼────────────┐
▼ ▼ ▼
Broker-1 Broker-2 Broker-3
│ │ │
P0/P3 P1/P4 P2/P5Key techniques:
Increase partitions appropriately.
Run multiple consumers in a consumer group.
Use batching.
Use compression.
Tune producer batch size/linger carefully.
Avoid unnecessary synchronous operations.
Scale brokers horizontally.
Monitor consumer lag.
Ensure adequate disk/network capacity.
28. What is Consumer Lag?
Consumer lag tells us how far behind a consumer is from the latest available records.
Example:
Latest offset = 1000
Consumer offset = 850
Lag = 150Flow:
Producer
│
▼
Kafka
│
│ 1000
▼
Consumer
│
│ processed till 850
▼
Lag = 150High lag could indicate:
slow consumer
insufficient consumer instances
downstream dependency latency
too few partitions
processing bottleneck
broker/network issues
29. How would you troubleshoot increasing Kafka consumer lag?
My interview answer would be:
Consumer Lag increasing
│
▼
Check consumer processing time
│
├── Slow DB/API?
│
├── CPU?
│
├── GC?
│
├── Network?
│
├── Partition imbalance?
│
└── Insufficient consumers?Then:
Check consumer lag per partition.
Check consumer CPU/memory.
Check processing latency.
Check downstream DB/API latency.
Check rebalance frequency.
Check partition distribution.
Increase consumers only if there are enough partitions.
Increase partitions if the architecture requires more parallelism.
30. Can we increase Kafka partitions later?
Yes.
For example:
Before:
Topic → 3 partitions
After:
Topic → 6 partitionsBut there is an important caveat:
Increasing partitions can affect key-based partition distribution and therefore ordering behavior for future records.
So partition count should be planned carefully.
31. What happens when a Kafka broker goes down?
Suppose:
P0
Broker-1 → Leader
Broker-2 → Replica
Broker-3 → ReplicaBroker-1 fails:
Broker-1 ❌
Broker-2 → New Leader
Broker-3 → ReplicaProducers and consumers reconnect to the new leader.
This is why replication is important.
32. Kafka Architecture — Complete Flow
This is a good diagram to remember for interviews:
┌─────────────────┐
│ Producer │
│ Order Service │
└────────┬────────┘
│
│ OrderCreated
▼
┌────────────────────┐
│ Kafka Cluster │
│ │
│ Topic: orders │
│ │
│ P0 P1 P2 │
└──┬────┬────┬───────┘
│ │ │
▼ ▼ ▼
C1 C2 C3
\ | /
\ | /
Consumer Group
Payment
│
▼
┌──────────────┐
│ Payment DB │
└──────────────┘Another consumer group can independently consume the same topic:
Kafka Topic
│
┌────────────┴────────────┐
│ │
▼ ▼
Consumer Group A Consumer Group B
Payment Service Analytics Service
│ │
▼ ▼
Payment DB Data Lake33. Kafka vs Redis Streams vs RabbitMQ
A senior architect may be asked this.
| Feature | Kafka | RabbitMQ | Redis Streams |
|---|---|---|---|
| Primary model | Distributed log | Message broker | Stream |
| High throughput | Excellent | Good | Excellent |
| Long retention | Excellent | Less typical | Depends on configuration |
| Replay | Excellent | Not primary strength | Yes |
| Partitioning | Core feature | Different model | Different model |
| Consumer groups | Yes | Consumer patterns | Yes |
| Event streaming | Excellent | Good | Good |
| Best fit | Large event streams | Messaging/work queues | Low-latency streams |
A good interview answer:
I would choose Kafka when durable event streaming, replay, high throughput and multiple independent consumers are important. RabbitMQ is often a better fit for traditional task/message routing patterns. Redis Streams can be attractive when very low latency and an existing Redis-centric architecture are priorities.
34. Real-world Spring Boot Kafka Flow
For your Java/Spring Boot background, explain it like this:
Spring Boot Order Service
│
│ KafkaTemplate
▼
Kafka Producer
│
▼
orders topic
│
▼
Payment Spring Boot Service
│
│ @KafkaListener
▼
Payment Processor
│
▼
Payment DBExample conceptually:
kafkaTemplate.send("orders", orderId, orderEvent);Consumer:
@KafkaListener(topics = "orders", groupId = "payment-service")
public void consume(OrderEvent event) {
paymentService.process(event);
}The important thing isn't memorizing annotations. In a senior interview, explain:
Producer
↓
Partition selection
↓
Leader
↓
Replication
↓
Consumer Group
↓
Partition assignment
↓
Processing
↓
Offset commit35. Senior Architect Question: Kafka + Database — How do you avoid data inconsistency?
This is very important.
Suppose:
Order Service
│
├── Save order to DB
│
└── Publish Kafka eventWhat happens?
DB commit ✓
Kafka publish ❌Now DB has the order but Kafka doesn't.
Solution: Transactional Outbox Pattern
Order Service
│
▼
┌─────────────────┐
│ DB Transaction│
│ │
│ Orders │
│ Outbox Events │
└────────┬────────┘
│
▼
Outbox Table
│
▼
CDC / Outbox
Publisher
│
▼
KafkaThe order and outbox event are committed in the same database transaction.
Then a CDC/outbox publisher sends the event to Kafka.
This is an excellent senior-level Kafka interview answer.
36. Most Important Kafka Questions to Prepare
If you have limited interview time, focus on these 15:
What is Kafka?
Kafka vs RabbitMQ?
Topic vs Partition?
What is a broker?
What is replication factor?
Leader vs follower?
What is ISR?
What is offset?
What is consumer group?
How does partition assignment work?
How is ordering guaranteed?
acks=0/1/all?At-most-once vs at-least-once vs exactly-once?
What is consumer lag and how do you troubleshoot it?
Kafka + DB consistency / Transactional Outbox?
One master flow to memorize
PRODUCER
│
│ Event + Key
▼
┌─────────────────┐
│ Kafka Cluster │
│ │
│ Topic │
│ ├─ P0 │
│ ├─ P1 │
│ └─ P2 │
└───────┬─────────┘
│
Partition Assignment
│
┌───────┼────────┐
▼ ▼ ▼
C1 C2 C3
└───────┼────────┘
│
Consumer Group
│
▼
Processing
│
▼
Offset Commit
│
┌───────┴────────┐
│ │
Success Failure
│ │
▼ ▼
Continue Retry / DLTFor a Senior Java Architect interview, the next level is not just Kafka definitions—it is scenario questions. The highest-value ones are: “Kafka consumer is slow,” “duplicate payment happened,” “broker went down,” “message ordering broke,” “DB commit succeeded but Kafka publish failed,” “10K TPS system design,” and “how to implement retry/DLT in Spring Kafka.”
No comments:
Post a Comment