Cloud · AWS

Amazon Kinesis (and vs Kafka)

awsMediumawskinesiskafkastreamingshards

Interview question

What is Amazon Kinesis, and how does it compare to Apache Kafka?

How to answer

The short answer

Amazon Kinesis is AWS's family of managed streaming services. For data engineers, Kinesis Data Streams is the closest cousin to Kafka: producers put records into a stream, consumers read them in near real time.

Kinesis Data Streams basics

  • Stream split into shards (throughput units; think )
  • Retention typically hours to days (configurable; shorter than many Kafka retention setups by default)
  • Consumers can use enhanced fan-out for dedicated throughput
  • Kinesis Data Firehose is a managed delivery pipe into S3/Redshift/etc. with less code

Kinesis vs Kafka

DimensionKinesis Data StreamsApache Kafka
OpsFully managed AWS serviceSelf-managed or managed (MSK / Confluent)
Scale unitShardPartition
OrderingPer shardPer partition
EcosystemAWS Lambda, Firehose, Broad: Connect, Streams, Flink, etc.
PortabilityAWS-lockedMulti-cloud / on-prem
RetentionHours–days (extendable)Often days–weeks+ by design

Tiny rule of thumb

  • Heavy AWS + Lambda consumers + short retention → Kinesis
  • Multi-cloud, long retention, rich Kafka tooling, or existing Kafka ops → Kafka / MSK

Interview tip: "Kinesis Streams ≈ managed AWS event log with shards; Kafka ≈ open streaming log with partitions." Mention Firehose as the easy land-to-S3 path.