Amazon Data Firehose was previously known as Amazon Kinesis Data Firehose
Writing to Amazon Data Firehose Using Kinesis Data Streams
You can configure Amazon Kinesis Data Streams to send information to a Firehose stream.
Important
If you use the Kinesis Producer Library (KPL) to write data to a Kinesis data stream, you can use aggregation to combine the records that you write to that Kinesis data stream. If you then use that data stream as a source for your Firehose stream, Amazon Data Firehose de-aggregates the records before it delivers them to the destination. If you configure your delivery stream to transform the data, Amazon Data Firehose de-aggregates the records before it delivers them to Amazon Lambda. For more information, see Developing Amazon Kinesis Data Streams Producers Using the Kinesis Producer Library and Aggregation.
-
Sign in to the Amazon Web Services Management Console and open the Amazon Data Firehose console at https://console.amazonaws.cn/firehose/
. -
Choose Create Firehose stream. On the Name and source page, provide values for the following fields:
- Firehose stream name
-
The name of your Firehose stream.
- Source
-
Choose Kinesis stream to configure a Firehose stream that uses a Kinesis data stream as a data source. You can then use Amazon Data Firehose to read data easily from an existing data stream and load it into destinations.
To use a Kinesis data stream as a source, choose an existing stream in the Kinesis stream list, or choose Create new to create a new Kinesis data stream. After you create a new stream, choose Refresh to update the Kinesis stream list. If you have a large number of streams, filter the list using Filter by name.
Note
When you configure a Kinesis data stream as the source of a Firehose stream, the Amazon Data Firehose
PutRecord
andPutRecordBatch
operations are disabled. To add data to your Firehose stream in this case, use the Kinesis Data StreamsPutRecord
andPutRecords
operations.Amazon Data Firehose starts reading data from the
LATEST
position of your Kinesis stream. For more information about Kinesis Data Streams positions, see GetShardIterator.Amazon Data Firehose calls the Kinesis Data Streams GetRecords operation once per second for each shard. However, when full backup is enabled, Firehose calls the Kinesis Data Streams
GetRecords
operation twice per second for each shard, one for primary delivery destination and another for full backup.More than one Firehose stream can read from the same Kinesis stream. Other Kinesis applications (consumers) can also read from the same stream. Each call from any Firehose stream or other consumer application counts against the overall throttling limit for the shard. To avoid getting throttled, plan your applications carefully. For more information about Kinesis Data Streams limits, see Amazon Kinesis Streams Limits.
Choose Next to advance to the Record Transformation and Format Conversion page.