Getting started with Amazon S3 delivery
Currently, customers run and manage their own image transcoding pipeline to create images for various purposes like scrubbing, image preview, running ML models on images, and more. Kinesis Video Streams offers the capability to transcode and deliver the images. Kinesis Video Streams will automatically extract images from video data in real-time based on a tag, and deliver the images to a customer specified S3 bucket.
UpdateImageGenerationConfiguration
To set up a Kinesis video stream to enable image generation to Amazon S3:
-
Create an S3 Bucket for image generation based on the tags added in the SDK using the new API. Note the S3 URI, which is required in the next step, when updating the image generation configurations for the streams.
-
Create a JSON file called update-image-generation-input.json with the following content as input.
{ "StreamName": "TestStream", "ImageGenerationConfiguration": { "Status": "ENABLED", "DestinationConfig": { "DestinationRegion": "us-east-1", "Uri": "s3://bucket-name" }, "SamplingInterval": 200, "ImageSelectorType": "PRODUCER_TIMESTAMP", "Format": "JPEG", "FormatConfig": { "JPEGQuality": "80" }, "WidthPixels": 320, "HeightPixels": 240 } }
You can use the Amazon CLI to invoke the UpdateImageGenerationConfiguration
ENABLED
.
aws kinesisvideo update-image-generation-configuration \ --cli-input-json file://./update-image-generation-input.json \
Request:
UpdateImageGenerationConfiguration HTTP/1.1 Method: 'POST' Path: '/updateImageGenerationConfiguration' Body: { StreamName: 'String', // Optional. Either stream name or arn should be passed StreamArn: 'String', // Optional. Either stream name or arn should be passed ImageGenerationConfiguration : { // required Status: 'Enum', // ENABLED | DISABLED, ImageSelectorType: 'Enum', // SERVER_TIMESTAMP | PRODUCER_TIMESTAMP.. DestinationConfig: { DestinationRegion: 'String', Uri: string, }, SamplingInterval: 'Number'// Format: 'Enum', // JPEG | PNG // Optional parameters FormatConfig: { 'String': 'String', }, WidthPixels: 'Number', // 1 - 3840 (4k). HeightPixels: 'Number' // 1 - 2160 (4k). } }
Response:
HTTP/1.1 200 Content-type: application/json Body: { }
Note
It takes at least 1 minute to initiate the image generation workflow after updating the image
generation configuration. Wait at least 1 minute before invoking
PutMedia
after the update call.
DescribeImageGenerationConfiguration
To view image generation configurations that are already set for a stream, customers can make a
DescribeImageGenerationConfiguration
request, as follows.
Request:
DescribeImageGenerationConfiguration HTTP/1.1 Method: 'POST' Path: '/describeImageGenerationConfiguration' Body: { StreamName: 'String', // Optional. Either stream name or arn should be passed StreamArn: 'String', // Optional. Either stream name or arn should be passed }
Response:
HTTP/1.1 200 Content-type: application/json Body: { ImageGenerationConfiguration : { Status: 'Enum', ImageSelectorType: 'Enum', // SERVER_TIMESTAMP | PRODUCER_TIMESTAMP DestinationConfig: { DestinationRegion: 'String' Uri: 'string', }, SamplingInterval: 'Number', Format: 'Enum', FormatConfig: { 'String': 'String', }, WidthPixels: 'Number', HeightPixels: 'Number' } }
To learn more about the DescribeImageGenerationConfiguration
feature, see
DescribeImageGenerationConfiguration
Producer MKV tags
You can use the Kinesis Video Streams Producer SDK to tag specific fragments of interest by exposing an API operation in
the SDK. For an example of a tag, see this code
Any fragment metadata provided along with the Amazon S3 image generation tags will be saved as Amazon S3 metadata.
Syntax for Producer MKV tags
|+ Tags | + Tag | // MANDATORY: Predefined MKV tag to trigger image generation for the fragment | + Simple | + Name: AWS_KINESISVIDEO_IMAGE_GENERATION | // OPTIONAL: S3 prefix which will be set as prefix for generated image. | + Simple | + Name: AWS_KINESISVIDEO_IMAGE_PREFIX | + String:
image_prefix_in_s3
// 256 bytes max m | // OPTIONAL: Key value pairs that will be persisted as S3 Image object metadata. | + Simple | + Name:CUSTOM_KEY_1
// Max 128 bytes | + String:CUSTOM_VALUE_1
// Max 256 bytes | + Simple | + Name:CUSTOM_KEY_2
// Max 128 bytes | + String:CUSTOM_VALUE_2
// Max 256 bytes
Adding metadata tags in Producer SDK using PutEventMetaData
The PutEventMetaData
function appends an MKV file that's associated with an event.
PutEventMetaData
takes two parameters. The first parameter is an event whose value comes from the
STREAM_EVENT_TYPE
enum. The second parameter, pStreamEventMetadata
Limits
The following table lists the limitations associated with the metadata tags. If the metadata tag limit is adjustable, you can request an increase through your account manager.
Limit | Max value | Adjustable |
---|---|---|
Image prefix length |
256 |
no |
Optional metadata key length |
128 |
no |
Optional metadata value length |
256 |
no |
Max number of optional metadata |
10 |
yes |
S3 Object metadata
By default, Kinesis Video Streams will set the fragment number, producer, and server timestamp of the image generated as Amazon S3 object metadata. If any additional fragment data is specified in the MKV tags, those tags will also be added to the Amazon S3 object metadata. The following example shows the correct syntax for the Amazon S3 object metadata.
{ // KVS S3 object metadata x-amz-meta-aws_kinesisvideo_fragment_number : 'string', x-amz-meta-aws_kinesisvideo_producer_timestamp: 'number', x-amz-meta-aws_kinesisvideo_server_timestamp: 'number', // Optional key value pair sent as part of the MKV tags
custom_key_1
:custom_value_1
,custom_key_2
:custom_value_2
, }
S3 object path (image)
The following list shows the correct format of the object path and describes each element within the path.
Format:
ImagePrefix
_AccountID
_StreamName
_ImageTimecode
_RandomID
.file-extension
1.ImagePrefix
- Value of
AWS_KINESISVIDEO_IMAGE_PREFIX
.
2.AccountID
- Account ID under which the stream is created.
3.StreamName
- Name of the stream for which the image is
generated.
4.ImageTimecode
- Epoch timecode in the fragment at which the image
is generated.
5.RandomID
- Random GUID.
6.file-extension
- JPG or PNG based on the image format requested.
Amazon S3 URI recommendations to protect against throttling
If you write thousands of images to Amazon S3, there's a risk of throttling. For more
information, see S3 Prefix Put
Request Limits
An Amazon S3 prefix starts with a PUT limit of 3,500 PUT requests per second, and will gradually ramp up over time for unique prefixes. Avoid using dates and times as Amazon S3 prefixes. Time coded data will impact one prefix at a time, and will also change regularly, invalidating previous prefix scale ups. To enable faster, consistent Amazon S3 scaling, we recommend adding a random prefix, like a hex code or UUID to the Amazon S3 Destination URI. For example, hex code prefixes will naturally split your requests randomly among 16 different prefixes (a prefix for each unique hex character), which will allow a 56,000 PUT requests per second after Amazon S3 has auto-scaled.