Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Using streaming metadata with Kinesis Video Streams

You can use the Amazon Kinesis Video Streams Producer SDK to embed metadata at the individual fragment level in a Kinesis video stream. Metadata in Kinesis Video Streams is a mutable key-value pair. You can use it to describe the content of the fragment, embed associated sensor readings that must be transferred along with the actual fragment, or meet other custom needs. The metadata is made available as part of the GetMedia or GetMediaForFragmentList API operations. It's stored along with the fragments for the entire duration of the stream's retention period. Your consuming applications can read, process, and react based on the metadata using the Kinesis video stream Parser Library.

There are two modes in which the metadata can be embedded with fragments in a stream:

You can affix metadata in both of these modes to the same fragment simultaneously, based on your application's needs. The embedded metadata might include objects detected, activity tracked, GPS coordinates, or any other custom data that you want to associate with the fragments in the stream. Metadata is encoded as key-value string pairs.

Adding metadata to a Kinesis video stream

Metadata that you add to a Kinesis video stream is modeled as MKV tags, which are implemented as key-value pairs.

Metadata can either be transient, such as to mark an event within the stream, or persistent, such as to identify fragments where a given event is taking place. A persistent metadata item remains, and is applied to each consecutive fragment, until it's canceled.

Streaming metadata API

You can use the following operations in the Producer SDK to implement streaming metadata.

PIC

PUBLIC_API STATUS putKinesisVideoFragmentMetadata(STREAM_HANDLE streamHandle, 
    PCHAR name, 
    PCHAR value, 
    BOOL persistent);

C++ Producer SDK

/**
 * Appends a "tag" or metadata - a key/value string pair into the stream.
 */
bool putFragmentMetadata(const std::string& name, const std::string& value, bool persistent = true);

Java Producer SDK

You can use the Java Producer SDK, to add metadata to a MediaSource using MediaSourceSink.onCodecPrivateData:

void onFragmentMetadata(final @Nonnull String metadataName, final @Nonnull String metadataValue, final boolean persistent)
throws KinesisVideoException;

Persistent and nonpersistent metadata

For nonpersistent metadata, you can add multiple metadata items with the same name. The Producer SDK collects the metadata items in the metadata queue until they are prepended to the next fragment. The metadata queue is cleared as the metadata items are applied to the stream. To repeat the metadata, call putKinesisVideoFragmentMetadata or putFragmentMetadata again.

For persistent metadata, the Producer SDK collects the metadata items in the metadata queue in the same way as for nonpersistent metadata. However, the metadata items aren't removed from the queue when they are prepended to the next fragment.

Calling putKinesisVideoFragmentMetadata or putFragmentMetadata with persistent set to true has the following behavior:

Consuming metadata embedded in a Kinesis video stream

To consume the metadata in a Kinesis video stream, use an implementation of MkvTagProcessor:

public interface MkvTagProcessor {
        default void process(MkvTag mkvTag, Optional<FragmentMetadata> currentFragmentMetadata) {
            throw new NotImplementedException("Default FragmentMetadataVisitor.MkvTagProcessor");
        }
        default void clear() {
            throw new NotImplementedException("Default FragmentMetadataVisitor.MkvTagProcessor");
	    }
    }
}

This interface is found in the FragmentMetadataVisitor class in the Kinesis video stream Parser Library.

The FragmentMetadataVisitor class contains an implementation of MkvTagProcessor:

public static final class BasicMkvTagProcessor implements FragmentMetadataVisitor.MkvTagProcessor {
    @Getter
    private List<MkvTag> tags = new ArrayList<>();

    @Override
    public void process(MkvTag mkvTag, Optional<FragmentMetadata> currentFragmentMetadata) {
        tags.add(mkvTag);
    }
    
    @Override
    public void clear() {
        tags.clear();
	}
}

The KinesisVideoRendererExample class contains an example of how to use a BasicMkvTagProcessor. In the following example, a BasicMkvTagProcessor is added to the MediaProcessingArguments of an application:

if (renderFragmentMetadata) {
    getMediaProcessingArguments = KinesisVideoRendererExample.GetMediaProcessingArguments.create(
        Optional.of(new FragmentMetadataVisitor.BasicMkvTagProcessor()));

The BasicMkvTagProcessor.process method is called when fragment metadata arrives. You can retrieve the accumulated metadata with GetTags. To retrieve a single metadata item, first call clear to clear the collected metadata, and then retrieve the metadata items again.

Streaming metadata limits

See Fragment metadata quotas for more information about the limits that apply to adding streaming metadata to a Kinesis video stream