Kinesis Data Streams
Use this page as a service-specific case-study starter for architecture, simulation, and optimization scenarios in pinpole.cloud.
Why this service matters
Kinesis Data Streams is the high-throughput ordered stream buffer. Use it when you need ordered processing per key, multiple independent consumers, or near-real-time analytics.
Power:
- Ordered per partition key (within a shard)
- Multiple consumers: shared throughput or Enhanced Fan-Out (EFO)
- Put-to-get delay typically < 1 second
Hard limits / constraints (from the Kinesis Developer Guide):
Important workflows
- Put Test Record - Publish a sample record to the stream
- Scale Shards - Increase/decrease shard count
- Check Hot Shards - Detect skew from partition keys
- Reshard Plan - Create a split/merge strategy for throughput changes
Configuration sections
- Why Kinesis (Power + Limits)
- Stream Mode, Retention & Core Limits
- Throughput Planning (Hard Numbers)
- Partitioning (Shard Skew Defense)
- Producers (PutRecord/PutRecords, KPL)
- Consumers (Shared vs EFO, Lambda / KCL)
- Resharding & Scaling Operations
- Security (Server-side encryption, KMS)
Key configuration points
| Point | Default / Value | Category |
|---|---|---|
| Enabled | true | Stream Mode, Retention & Core Limits |
| Capacity Mode | on-demand-standard | Stream Mode, Retention & Core Limits |
| Shard Count | 10 | Stream Mode, Retention & Core Limits |
| Retention (hours) | 24 | Stream Mode, Retention & Core Limits |
| Max Record Data Blob (bytes) | 1048576 | Stream Mode, Retention & Core Limits |
| Partition Key Max Length (chars) | 256 | Stream Mode, Retention & Core Limits |
| Partition Key Strategy | high-cardinality | Partitioning (Shard Skew Defense) |
| Producer Library | kpl | Producers (PutRecord/PutRecords, KPL) |
| Write API | PutRecords | Producers (PutRecord/PutRecords, KPL) |
| KPL Aggregation | true | Producers (PutRecord/PutRecords, KPL) |
| Max Payload Size (MiB) | 1 | Producers (PutRecord/PutRecords, KPL) |
| Retry Strategy | bounded-exponential | Producers (PutRecord/PutRecords, KPL) |