Azure Event Hubs partitions and Event Hubs Capture

Partitions and consumer groups

Event Hubs uses a partitioned consumer architecture similar to Apache Kafka:

• Partitions: Ordered event sequences that provide horizontal throughput scaling. Publishers pass a partition key so events for the same entity, such as an account or sensor, land on the same partition in strict sequential order.
• Consumer groups: Independent read views of the event stream. Multiple downstream consumers, such as an Azure Stream Analytics job and a Python consumer, read from the same Event Hub independently at their own offsets without affecting each other.

Producers (Telemetry / Logs)
         | (partition_key)
         v
Event Hub: [Partition 0] [Partition 1] [Partition 2] [Partition 3]
         |
         |---> Consumer Group A: Real-Time Stream Analytics (Alerts)
         |---> Consumer Group B: Python Worker Pool (Custom Processing)
         `---> Event Hubs Capture ---> ADLS Gen2 (Parquet / Avro Landing)

Serverless ingest with Event Hubs Capture

Writing custom worker microservices simply to read streaming events and write them to storage introduces unnecessary operational complexity.

Event Hubs Capture solves this by automatically batching streaming data and writing it directly into ADLS Gen2 or Blob Storage in Avro or Parquet format. Engineers configure a time window or size threshold, such as every 5 minutes or 300 megabytes, and Event Hubs handles landing files into object storage without executing consumer code.