Amazon Kinesis Data Streams versus Amazon MSK

04 Dec 2024

Learn how Amazon Kinesis Data Streams differs from Amazon MSK.

Amazon Kinesis Data Streams Amazon MSK
Scaling

1 MB or 1000 msg/sec per shard; up to 500 shards; you need to manually add/remove shards

30 brokers per cluster; you need add/remove brokers and reassign partitions manually

Max. message size

1 MB

configurable (default 1 MB)

Persistence

up to 365 days

forever (up to 16384 GiB per broker)

Replication

Multi-AZ

Multi-AZ (optional)

Order guarantee

within a shard

within a partition

Delivery guarantee

at least once

up to the consumer

Pricing

per shard hour

per broker hour + provisioned storage

Protocols

AWS Rest API

Kafka protocol

AWS Integrations

Lambda

Lambda

License

AWS only

open source (Apache Kafka)

Encryption at rest

yes

yes

Encryption in transit

yes

yes