Skip to content
LakeBench
ProblemsCommunityPricing
Sign inStart practicing
Back
  1. Home
  2. Interview prep
  3. Producer write flow

Kafka · Producers & Consumers

Producer write flow

Easykafka-11
producerwrite pathpartitionerbatching

Question

Walk through what happens when a Kafka producer sends a message.

Solution

Here is a fresher-friendly produce path:

Step by step

1. Create producer with bootstrap.servers, serializers, acks, etc. 2. Send a ProducerRecord (topic, optional key, value, optional headers/partition). 3. Producer serializes key/value (e.g. String, Avro, JSON). 4. Partitioner chooses a partition:

  • If partition set explicitly → use it
  • Else if key present → hash(key) % numPartitions
  • Else → sticky / round-robin style distribution

5. Producer may batch records for that partition (linger + batch size). 6. Send batch to the leader broker for that partition. 7. Leader appends to its log and replicates per acks policy. 8. Broker returns success/failure; producer callbacks / futures complete. 9. On success (depending on acks), the record has an assigned offset.

App → serialize → choose partition → batch → network
        → leader broker → (replicas if acks=all) → ack to producer

Tiny mental example

ProducerRecord(topic="orders", key="cust-9", value="{...}")
  → partition 2 (hash of cust-9)
  → leader of partition 2 on broker-3
  → offset 184422 assigned

Common producer settings to mention

  • acks
  • retries / delivery timeout
  • linger.ms and batch.size
  • enable.idempotence
  • compression (snappy, lz4, zstd)

Interview tip: Narrate key → partition → leader → ack. That shows you understand the path, not just the API.

PreviousNext