Writing to Kinesis Data Firehose Using Kinesis Data Streams - Amazon Kinesis Data Firehose
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Writing to Kinesis Data Firehose Using Kinesis Data Streams

You can configure Amazon Kinesis Data Streams to send information to a Kinesis Data Firehose delivery stream.

Important

If you use the Kinesis Producer Library (KPL) to write data to a Kinesis data stream, you can use aggregation to combine the records that you write to that Kinesis data stream. If you then use that data stream as a source for your Kinesis Data Firehose delivery stream, Kinesis Data Firehose de-aggregates the records before it delivers them to the destination. If you configure your delivery stream to transform the data, Kinesis Data Firehose de-aggregates the records before it delivers them to Amazon Lambda. For more information, see Developing Amazon Kinesis Data Streams Producers Using the Kinesis Producer Library and Aggregation.

  1. Sign in to the Amazon Web Services Management Console and open the Kinesis Data Firehose console at https://console.amazonaws.cn/firehose/.

  2. Choose Create Delivery Stream. On the Name and source page, provide values for the following fields:

    Delivery stream name

    The name of your Kinesis Data Firehose delivery stream.

    Source

    Choose Kinesis stream to configure a Kinesis Data Firehose delivery stream that uses a Kinesis data stream as a data source. You can then use Kinesis Data Firehose to read data easily from an existing data stream and load it into destinations.

    To use a Kinesis data stream as a source, choose an existing stream in the Kinesis stream list, or choose Create new to create a new Kinesis data stream. After you create a new stream, choose Refresh to update the Kinesis stream list. If you have a large number of streams, filter the list using Filter by name.

    Note

    When you configure a Kinesis data stream as the source of a Kinesis Data Firehose delivery stream, the Kinesis Data Firehose PutRecord and PutRecordBatch operations are disabled. To add data to your Kinesis Data Firehose delivery stream in this case, use the Kinesis Data Streams PutRecord and PutRecords operations.

    Kinesis Data Firehose starts reading data from the LATEST position of your Kinesis stream. For more information about Kinesis Data Streams positions, see GetShardIterator. Kinesis Data Firehose calls the Kinesis Data Streams GetRecords operation once per second for each shard.

    More than one Kinesis Data Firehose delivery stream can read from the same Kinesis stream. Other Kinesis applications (consumers) can also read from the same stream. Each call from any Kinesis Data Firehose delivery stream or other consumer application counts against the overall throttling limit for the shard. To avoid getting throttled, plan your applications carefully. For more information about Kinesis Data Streams limits, see Amazon Kinesis Streams Limits.

  3. Choose Next to advance to the Record Transformation and Format Conversion page.