Kinesis examples using Amazon CLI

To add tags to a data stream

The following add-tags-to-stream example assigns a tag with the key samplekey and value example to the specified stream.

aws kinesis add-tags-to-stream \
    --stream-name samplestream \
    --tags samplekey=example

This command produces no output.

For more information, see Tagging Your Streams in the Amazon Kinesis Data Streams Developer Guide.

To create a data stream

The following create-stream example creates a data stream named samplestream with 3 shards.

aws kinesis create-stream \
    --stream-name samplestream \
    --shard-count 3

This command produces no output.

For more information, see Creating a Stream in the Amazon Kinesis Data Streams Developer Guide.

To decrease data stream retention period

The following decrease-stream-retention-period example decreases the retention period (the length of time data records are accessible after they are added to the stream) of a stream named samplestream to 48 hours.

aws kinesis decrease-stream-retention-period \
    --stream-name samplestream \
    --retention-period-hours 48

This command produces no output.

For more information, see Changing the Data Retention Period in the Amazon Kinesis Data Streams Developer Guide.

To delete a data stream

The following delete-stream example deletes the specified data stream.

aws kinesis delete-stream \
    --stream-name samplestream

This command produces no output.

For more information, see Deleting a Stream in the Amazon Kinesis Data Streams Developer Guide.

To deregister a data stream consumer

The following deregister-stream-consumer example deregisters the specified consumer from the specified data stream.

aws kinesis deregister-stream-consumer \
    --stream-arn arn:aws:kinesis:us-west-2:123456789012:stream/samplestream \
    --consumer-name KinesisConsumerApplication

This command produces no output.

For more information, see Developing Consumers with Enhanced Fan-Out Using the Kinesis Data Streams API in the Amazon Kinesis Data Streams Developer Guide.

To describe shard limits

The following describe-limits example displays the shard limits and usage for the current Amazon account.

aws kinesis describe-limits

Output:

{
    "ShardLimit": 500,
    "OpenShardCount": 29
}

For more information, see Resharding a Stream in the Amazon Kinesis Data Streams Developer Guide.

To describe a data stream consumer

The following describe-stream-consumer example returns the description of the specified consumer, registered with the specified data stream.

aws kinesis describe-stream-consumer \
    --stream-arn arn:aws:kinesis:us-west-2:012345678912:stream/samplestream \
    --consumer-name KinesisConsumerApplication

Output:

{
    "ConsumerDescription": {
        "ConsumerName": "KinesisConsumerApplication",
        "ConsumerARN": "arn:aws:kinesis:us-west-2:123456789012:stream/samplestream/consumer/KinesisConsumerApplication:1572383852",
        "ConsumerStatus": "ACTIVE",
        "ConsumerCreationTimestamp": 1572383852.0,
        "StreamARN": "arn:aws:kinesis:us-west-2:123456789012:stream/samplestream"
    }
}

For more information, see Reading Data from Amazon Kinesis Data Streams in the Amazon Kinesis Data Streams Developer Guide.

To describe a data stream summary

The following describe-stream-summary example provides a summarized description (without the shard list) of the specified data stream.

aws kinesis describe-stream-summary \
    --stream-name samplestream

Output:

{
    "StreamDescriptionSummary": {
        "StreamName": "samplestream",
        "StreamARN": "arn:aws:kinesis:us-west-2:123456789012:stream/samplestream",
        "StreamStatus": "ACTIVE",
        "RetentionPeriodHours": 48,
        "StreamCreationTimestamp": 1572297168.0,
        "EnhancedMonitoring": [
            {
                "ShardLevelMetrics": []
            }
        ],
        "EncryptionType": "NONE",
        "OpenShardCount": 3,
        "ConsumerCount": 0
    }
}

For more information, see Creating and Managing Streams in the Amazon Kinesis Data Streams Developer Guide.

To describe a data stream

The following describe-stream example returns the details of the specified data stream.

aws kinesis describe-stream \
    --stream-name samplestream

Output:

{
    "StreamDescription": {
        "Shards": [
            {
                "ShardId": "shardId-000000000000",
                "HashKeyRange": {
                    "StartingHashKey": "0",
                    "EndingHashKey": "113427455640312821154458202477256070484"
                },
                "SequenceNumberRange": {
                    "StartingSequenceNumber": "49600871682957036442365024926191073437251060580128653314"
                }
            },
            {
                "ShardId": "shardId-000000000001",
                "HashKeyRange": {
                    "StartingHashKey": "113427455640312821154458202477256070485",
                    "EndingHashKey": "226854911280625642308916404954512140969"
                },
                "SequenceNumberRange": {
                    "StartingSequenceNumber": "49600871682979337187563555549332609155523708941634633746"
                }
            },
            {
                "ShardId": "shardId-000000000002",
                "HashKeyRange": {
                    "StartingHashKey": "226854911280625642308916404954512140970",
                    "EndingHashKey": "340282366920938463463374607431768211455"
                },
                "SequenceNumberRange": {
                    "StartingSequenceNumber": "49600871683001637932762086172474144873796357303140614178"
                }
            }
        ],
        "StreamARN": "arn:aws:kinesis:us-west-2:123456789012:stream/samplestream",
        "StreamName": "samplestream",
        "StreamStatus": "ACTIVE",
        "RetentionPeriodHours": 24,
        "EnhancedMonitoring": [
            {
                "ShardLevelMetrics": []
            }
        ],
        "EncryptionType": "NONE",
        "KeyId": null,
        "StreamCreationTimestamp": 1572297168.0
    }
}

For more information, see Creating and Managing Streams in the Amazon Kinesis Data Streams Developer Guide.

To disable enhanced monitoring for shard-level metrics

The following disable-enhanced-monitoring example disables enhanced Kinesis data stream monitoring for shard-level metrics.

aws kinesis disable-enhanced-monitoring \
    --stream-name samplestream --shard-level-metrics ALL

Output:

{
    "StreamName": "samplestream",
    "CurrentShardLevelMetrics": [
        "IncomingBytes",
        "OutgoingRecords",
        "IteratorAgeMilliseconds",
        "IncomingRecords",
        "ReadProvisionedThroughputExceeded",
        "WriteProvisionedThroughputExceeded",
        "OutgoingBytes"
    ],
    "DesiredShardLevelMetrics": []
}

For more information, see Monitoring Streams in Amazon Kinesis Data Streams in the Amazon Kinesis Data Streams Developer Guide.

To enable enhanced monitoring for shard-level metrics

The following enable-enhanced-monitoring example enables enhanced Kinesis data stream monitoring for shard-level metrics.

aws kinesis enable-enhanced-monitoring \
    --stream-name samplestream \
    --shard-level-metrics ALL

Output:

{
    "StreamName": "samplestream",
    "CurrentShardLevelMetrics": [],
    "DesiredShardLevelMetrics": [
        "IncomingBytes",
        "OutgoingRecords",
        "IteratorAgeMilliseconds",
        "IncomingRecords",
        "ReadProvisionedThroughputExceeded",
        "WriteProvisionedThroughputExceeded",
        "OutgoingBytes"
    ]
}

For more information, see Monitoring Streams in Amazon Kinesis Data Streams in the Amazon Kinesis Data Streams Developer Guide.

To obtain records from a shard

The following get-records example gets data records from a Kinesis data stream's shard using the specified shard iterator.

aws kinesis get-records \
    --shard-iterator AAAAAAAAAAF7/0mWD7IuHj1yGv/TKuNgx2ukD5xipCY4cy4gU96orWwZwcSXh3K9tAmGYeOZyLZrvzzeOFVf9iN99hUPw/w/b0YWYeehfNvnf1DYt5XpDJghLKr3DzgznkTmMymDP3R+3wRKeuEw6/kdxY2yKJH0veaiekaVc4N2VwK/GvaGP2Hh9Fg7N++q0Adg6fIDQPt4p8RpavDbk+A4sL9SWGE1

Output:

{
    "Records": [],
    "MillisBehindLatest": 80742000
}

For more information, see Developing Consumers Using the Kinesis Data Streams API with the Amazon SDK for Java in the Amazon Kinesis Data Streams Developer Guide.

To obtain a shard iterator

The following get-shard-iterator example uses the AT_SEQUENCE_NUMBER shard iterator type and generates a shard iterator to start reading data records exactly from the position denoted by the specified sequence number.

aws kinesis get-shard-iterator \
    --stream-name samplestream \
    --shard-id shardId-000000000001 \
    --shard-iterator-type LATEST

Output:

{
    "ShardIterator": "AAAAAAAAAAFEvJjIYI+3jw/4aqgH9FifJ+n48XWTh/IFIsbILP6o5eDueD39NXNBfpZ10WL5K6ADXk8w+5H+Qhd9cFA9k268CPXCz/kebq1TGYI7Vy+lUkA9BuN3xvATxMBGxRY3zYK05gqgvaIRn94O8SqeEqwhigwZxNWxID3Ej7YYYcxQi8Q/fIrCjGAy/n2r5Z9G864YpWDfN9upNNQAR/iiOWKs"
}

For more information, see Developing Consumers Using the Kinesis Data Streams API with the Amazon SDK for Java in the Amazon Kinesis Data Streams Developer Guide.

To increase data stream retention period

The following increase-stream-retention-period example increases the retention period (the length of time data records are accessible after they are added to the stream) of the specified stream to 168 hours.

aws kinesis increase-stream-retention-period \
    --stream-name samplestream \
    --retention-period-hours 168

This command produces no output.

For more information, see Changing the Data Retention Period in the Amazon Kinesis Data Streams Developer Guide.

To list shards in a data stream

The following list-shards example lists all shards in the specified stream starting with the shard whose ID immediately follows the specified exclusive-start-shard-id of shardId-000000000000.

aws kinesis list-shards \
    --stream-name samplestream \
    --exclusive-start-shard-id shardId-000000000000

Output:

{
    "Shards": [
        {
            "ShardId": "shardId-000000000001",
            "HashKeyRange": {
                "StartingHashKey": "113427455640312821154458202477256070485",
                "EndingHashKey": "226854911280625642308916404954512140969"
            },
            "SequenceNumberRange": {
                "StartingSequenceNumber": "49600871682979337187563555549332609155523708941634633746"
            }
        },
        {
            "ShardId": "shardId-000000000002",
            "HashKeyRange": {
                "StartingHashKey": "226854911280625642308916404954512140970",
                "EndingHashKey": "340282366920938463463374607431768211455"
            },
            "SequenceNumberRange": {
                "StartingSequenceNumber": "49600871683001637932762086172474144873796357303140614178"
            }
        }
    ]
}

For more information, see Listing Shards in the Amazon Kinesis Data Streams Developer Guide.

To list data streams

The following list-streams example lists all active data streams in the current account and region.

aws kinesis list-streams

Output:

{
    "StreamNames": [
        "samplestream",
        "samplestream1"
    ]
}

For more information, see Listing Streams in the Amazon Kinesis Data Streams Developer Guide.

To list tags for a data stream

The following list-tags-for-stream example lists the tags attached to the specified data stream.

aws kinesis list-tags-for-stream \
    --stream-name samplestream

Output:

{
    "Tags": [
        {
            "Key": "samplekey",
            "Value": "example"
        }
    ],
    "HasMoreTags": false
}

For more information, see Tagging Your Streams in the Amazon Kinesis Data Streams Developer Guide.

To merge shards

The following merge-shards example merges two adjacent shards with IDs of shardId-000000000000 and shardId-000000000001 in the specified data stream and combines them into a single shard.

aws kinesis merge-shards \
    --stream-name samplestream \
    --shard-to-merge shardId-000000000000 \
    --adjacent-shard-to-merge shardId-000000000001

This command produces no output.

For more information, see Merging Two Shards in the Amazon Kinesis Data Streams Developer Guide.

To write a record into a data stream

The following put-record example writes a single data record into the specified data stream using the specified partition key.

aws kinesis put-record \
    --stream-name samplestream \
    --data sampledatarecord \
    --partition-key samplepartitionkey

Output:

{
    "ShardId": "shardId-000000000009",
    "SequenceNumber": "49600902273357540915989931256901506243878407835297513618",
    "EncryptionType": "KMS"
}

For more information, see Developing Producers Using the Amazon Kinesis Data Streams API with the Amazon SDK for Java in the Amazon Kinesis Data Streams Developer Guide.

To write multiple records into a data stream

The following put-records example writes a data record using the specified partition key and another data record using a different partition key in a single call.

aws kinesis put-records \
    --stream-name samplestream \
    --records Data=blob1,PartitionKey=partitionkey1 Data=blob2,PartitionKey=partitionkey2

Output:

{
    "FailedRecordCount": 0,
    "Records": [
        {
            "SequenceNumber": "49600883331171471519674795588238531498465399900093808706",
            "ShardId": "shardId-000000000004"
        },
        {
            "SequenceNumber": "49600902273357540915989931256902715169698037101720764562",
            "ShardId": "shardId-000000000009"
        }
    ],
    "EncryptionType": "KMS"
}

For more information, see Developing Producers Using the Amazon Kinesis Data Streams API with the Amazon SDK for Java in the Amazon Kinesis Data Streams Developer Guide.

To register a data stream consumer

The following register-stream-consumer example registers a consumer called KinesisConsumerApplication with the specified data stream.

aws kinesis register-stream-consumer \
    --stream-arn arn:aws:kinesis:us-west-2:012345678912:stream/samplestream \
    --consumer-name KinesisConsumerApplication

Output:

{
    "Consumer": {
        "ConsumerName": "KinesisConsumerApplication",
        "ConsumerARN": "arn:aws:kinesis:us-west-2: 123456789012:stream/samplestream/consumer/KinesisConsumerApplication:1572383852",
        "ConsumerStatus": "CREATING",
        "ConsumerCreationTimestamp": 1572383852.0
    }
}

For more information, see Developing Consumers with Enhanced Fan-Out Using the Kinesis Data Streams API in the Amazon Kinesis Data Streams Developer Guide.

To remove tags from a data stream

The following remove-tags-from-stream example removes the tag with the specified key from the specified data stream.

aws kinesis remove-tags-from-stream \
    --stream-name samplestream \
    --tag-keys samplekey

This command produces no output.

For more information, see Tagging Your Streams in the Amazon Kinesis Data Streams Developer Guide.

To split shards

The following split-shard example splits the specified shard into two new shards using a new starting hash key of 10.

aws kinesis split-shard \
    --stream-name samplestream \
    --shard-to-split shardId-000000000000 \
    --new-starting-hash-key 10

This command produces no output.

For more information, see Splitting a Shard in the Amazon Kinesis Data Streams Developer Guide.

To enable data stream encryption

The following start-stream-encryption example enables server-side encryption for the specified stream, using the specified Amazon KMS key.

aws kinesis start-stream-encryption \
    --encryption-type KMS \
    --key-id arn:aws:kms:us-west-2:012345678912:key/a3c4a7cd-728b-45dd-b334-4d3eb496e452 \
    --stream-name samplestream

This command produces no output.

For more information, see Data Protection in Amazon Kinesis Data Streams in the Amazon Kinesis Data Streams Developer Guide.

To disable data stream encryption

The following stop-stream-encryption example disables server-side encryption for the specified stream, using the specified Amazon KMS key.

aws kinesis start-stream-encryption \
    --encryption-type KMS \
    --key-id arn:aws:kms:us-west-2:012345678912:key/a3c4a7cd-728b-45dd-b334-4d3eb496e452 \
    --stream-name samplestream

This command produces no output.

For more information, see Data Protection in Amazon Kinesis Data Streams in the Amazon Kinesis Data Streams Developer Guide.

To update the shard count in a data stream

The following update-shard-count example updates the shard count of the specified data stream to 6. This example uses uniform scaling, which creates shards of equal size.

aws kinesis update-shard-count \
    --stream-name samplestream \
    --scaling-type UNIFORM_SCALING \
    --target-shard-count 6

Output:

{
    "StreamName": "samplestream",
    "CurrentShardCount": 3,
    "TargetShardCount": 6
}

For more information, see Resharding a Stream in the Amazon Kinesis Data Streams Developer Guide.