Kafka · Extra High-Value

What exactly is an offset?

Easyoffsetpartitionconsumerbasics

Interview question

In one crisp answer: what is a Kafka offset, and what is it not?

How to answer

The short answer

An offset is a per-partition, monotonically increasing identifier for a record in that 's log.

What it is

  • A bookmark position inside 1 partition
  • Assigned by the broker when the record is appended
  • Used by consumers to know what to read next
  • Committed by to mark progress

What it is not

  • Not global across a topic (partition 0 offset 5 ≠ partition 1 offset 5 as the "same" event)
  • Not a timestamp (though records also have timestamps)
  • Not a of your business event (your event should have its own id)
  • Log end offset / : newest offsets available to consumers
  • Committed offset: last progress saved for a group
  • Consumer position: where the consumer is currently reading

Interview tip: "Offset = index in a partition log. Always say which partition."