Extract images from video streams
You can use Amazon Kinesis Video Streams APIs and SDKs to help you extract images from your video streams. You can use these images for enhanced playback applications such as thumbnails or enhanced scrubbing, or for use in machine learning pipelines. Kinesis Video Streams offers on-demand image extraction through an API, or automated image extraction from metadata tags in an ingested video.
For information about using Kinesis Video Streams managed support for images, see:
On-demand Image Generation (GetImages) - This API allows customers to extract a single image or multiple images from video stored in Kinesis Video Streams.
Automated image generation (Amazon S3 delivery) - Configure Kinesis Video Streams to automatically extract images from video data in real time based on tags in an uploaded video, and deliver the images to the customer-specified S3 bucket.
Automated image generation (Amazon S3 delivery)
Currently, customers run and manage their own image transcoding pipeline to create images for various purposes like scrubbing, image preview, running ML models on images, and more. Kinesis Video Streams offers the capability to transcode and deliver the images. Kinesis Video Streams will automatically extract images from video data in real-time based on a tag, and deliver the images to a customer specified S3 bucket.
Topics
UpdateImageGenerationConfiguration
To set up a Kinesis video stream to enable image generation to Amazon S3:
-
Create an S3 Bucket for image generation based on the tags added in the SDK using the new API. Note the S3 URI, which is required in the next step, when updating the image generation configurations for the streams.
-
Create a JSON file called update-image-generation-input.json with the following content as input.
{ "StreamName": "TestStream", "ImageGenerationConfiguration": { "Status": "ENABLED", "DestinationConfig": { "DestinationRegion": "us-east-1", "Uri": "s3://bucket-name" }, "SamplingInterval": 200, "ImageSelectorType": "PRODUCER_TIMESTAMP", "Format": "JPEG", "FormatConfig": { "JPEGQuality": "80" }, "WidthPixels": 320, "HeightPixels": 240 } }
You can use the Amazon CLI to invoke the UpdateImageGenerationConfiguration API operation to
add the Amazon S3 ARN created previously and change the status to
ENABLED
.
aws kinesisvideo update-image-generation-configuration \ --cli-input-json file://./update-image-generation-input.json \
Request:
UpdateImageGenerationConfiguration HTTP/1.1 Method: 'POST' Path: '/updateImageGenerationConfiguration' Body: { StreamName: 'String', // Optional. Either stream name or arn should be passed StreamArn: 'String', // Optional. Either stream name or arn should be passed ImageGenerationConfiguration : { // required Status: 'Enum', // ENABLED | DISABLED, ImageSelectorType: 'Enum', // SERVER_TIMESTAMP | PRODUCER_TIMESTAMP.. DestinationConfig: { DestinationRegion: 'String', Uri: string, }, SamplingInterval: 'Number'// Format: 'Enum', // JPEG | PNG // Optional parameters FormatConfig: { 'String': 'String', }, WidthPixels: 'Number', // 1 - 3840 (4k). HeightPixels: 'Number' // 1 - 2160 (4k). } }
Response:
HTTP/1.1 200 Content-type: application/json Body: { }
Note
It takes at least 1 minute to initiate the image generation workflow after updating the image
generation configuration. Wait at least 1 minute before invoking
PutMedia
after the update call.
DescribeImageGenerationConfiguration
To view image generation configurations that are already set for a stream, customers can make a
DescribeImageGenerationConfiguration
request, as follows.
Request:
DescribeImageGenerationConfiguration HTTP/1.1 Method: 'POST' Path: '/describeImageGenerationConfiguration' Body: { StreamName: 'String', // Optional. Either stream name or arn should be passed StreamArn: 'String', // Optional. Either stream name or arn should be passed }
Response:
HTTP/1.1 200 Content-type: application/json Body: { ImageGenerationConfiguration : { Status: 'Enum', ImageSelectorType: 'Enum', // SERVER_TIMESTAMP | PRODUCER_TIMESTAMP DestinationConfig: { DestinationRegion: 'String' Uri: 'string', }, SamplingInterval: 'Number', Format: 'Enum', FormatConfig: { 'String': 'String', }, WidthPixels: 'Number', HeightPixels: 'Number' } }
To learn more about the DescribeImageGenerationConfiguration
feature, see
DescribeImageGenerationConfiguration in the
Amazon Kinesis Video Streams Developer Guide.
Producer MKV tags
You can use the Kinesis Video Streams producer SDK to tag specific fragments of interest by exposing
an API operation in the SDK. For an example of a tag, see this code
Any fragment metadata provided along with the Amazon S3 image generation tags will be saved as Amazon S3 metadata.
Syntax for producer MKV tags
|+ Tags | + Tag | // MANDATORY: Predefined MKV tag to trigger image generation for the fragment | + Simple | + Name: AWS_KINESISVIDEO_IMAGE_GENERATION | // OPTIONAL: S3 prefix which will be set as prefix for generated image. | + Simple | + Name: AWS_KINESISVIDEO_IMAGE_PREFIX | + String:
image_prefix_in_s3
// 256 bytes max m | // OPTIONAL: Key value pairs that will be persisted as S3 Image object metadata. | + Simple | + Name:CUSTOM_KEY_1
// Max 128 bytes | + String:CUSTOM_VALUE_1
// Max 256 bytes | + Simple | + Name:CUSTOM_KEY_2
// Max 128 bytes | + String:CUSTOM_VALUE_2
// Max 256 bytes
Add metadata tags in producer SDK using PutEventMetaData
The PutEventMetaData
function appends an MKV file that's associated with an event.
PutEventMetaData
takes two parameters. The first parameter is an event whose value comes from the
STREAM_EVENT_TYPE
enum. The second parameter, pStreamEventMetadata
Limits
The following table lists the limitations associated with the metadata tags. If the metadata tag limit is adjustable, you can request an increase through your account manager.
Limit | Max value | Adjustable |
---|---|---|
Image prefix length |
256 |
no |
Optional metadata key length |
128 |
no |
Optional metadata value length |
256 |
no |
Max number of optional metadata |
10 |
yes |
S3 Object metadata
By default, Kinesis Video Streams will set the fragment number, producer, and server timestamp of the image generated as Amazon S3 object metadata. If any additional fragment data is specified in the MKV tags, those tags will also be added to the Amazon S3 object metadata. The following example shows the correct syntax for the Amazon S3 object metadata.
{ // KVS S3 object metadata x-amz-meta-aws_kinesisvideo_fragment_number : 'string', x-amz-meta-aws_kinesisvideo_producer_timestamp: 'number', x-amz-meta-aws_kinesisvideo_server_timestamp: 'number', // Optional key value pair sent as part of the MKV tags
custom_key_1
:custom_value_1
,custom_key_2
:custom_value_2
, }
Amazon S3 object path (image)
The following list shows the correct format of the object path and describes each element within the path.
Format:
ImagePrefix
_AccountID
_StreamName
_ImageTimecode
_RandomID
.file-extension
1.ImagePrefix
- Value of
AWS_KINESISVIDEO_IMAGE_PREFIX
.
2.AccountID
- Account ID under which the stream is created.
3.StreamName
- Name of the stream for which the image is
generated.
4.ImageTimecode
- Epoch timecode in the fragment at which the image
is generated.
5.RandomID
- Random GUID.
6.file-extension
- JPG or PNG based on the image format requested.
Amazon S3 URI recommendations to protect against throttling
If you write thousands of images to Amazon S3, there's a risk of throttling. For more
information, see S3 Prefix Put
Request Limits
An Amazon S3 prefix starts with a PUT limit of 3,500 PUT requests per second, and will gradually ramp up over time for unique prefixes. Avoid using dates and times as Amazon S3 prefixes. Time coded data will impact one prefix at a time, and will also change regularly, invalidating previous prefix scale ups. To enable faster, consistent Amazon S3 scaling, we recommend adding a random prefix, like a hex code or UUID to the Amazon S3 Destination URI. For example, hex code prefixes will naturally split your requests randomly among 16 different prefixes (a prefix for each unique hex character), which will allow a 56,000 PUT requests per second after Amazon S3 has auto-scaled.