Troubleshooting Amazon Kinesis Data Firehose

If Kinesis Data Firehose encounters errors while delivering or processing data, it retries until the configured retry duration expires. If the retry duration ends before the data is delivered successfully, Kinesis Data Firehose backs up the data to the configured S3 backup bucket. If the destination is Amazon S3 and delivery fails or if delivery to the backup S3 bucket fails, Kinesis Data Firehose keeps retrying until the retention period ends. For DirectPut delivery streams, Kinesis Data Firehose retains the records for 24 hours. For a delivery stream whose data source is a Kinesis data stream, you can change the retention period as described in Changing the Data Retention Period.

If the data source is a Kinesis data stream, Kinesis Data Firehose retries the following operations indefinitely: DescribeStream, GetRecords, and GetShardIterator.

If the delivery stream uses DirectPut, check the IncomingBytes and IncomingRecords metrics to see if there's incoming traffic. If you are using the PutRecord or PutRecordBatch, make sure you catch exceptions and retry. We recommend a retry policy with exponential back-off with jitter and several retries. Also, if you use the PutRecordBatch API, make sure your code checks the value of FailedPutCount in the response even when the API call succeeds.

If the delivery stream uses a Kinesis data stream as its source, check the IncomingBytes and IncomingRecords metrics for the source data stream. Additionally, ensure that the DataReadFromKinesisStream.Bytes and DataReadFromKinesisStream.Records metrics are being emitted for the delivery stream.

For information about tracking delivery errors using CloudWatch, see Monitoring Kinesis Data Firehose Using CloudWatch Logs.