Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

CreateDeliveryStream

Creates a Kinesis Data Firehose delivery stream.

By default, you can create up to 50 delivery streams per Amazon Region.

This is an asynchronous operation that immediately returns. The initial status of the delivery stream is CREATING. After the delivery stream is created, its status is ACTIVE and it now accepts data. If the delivery stream creation fails, the status transitions to CREATING_FAILED. Attempts to send data to a delivery stream that is not in the ACTIVE state cause an exception. To check the state of a delivery stream, use DescribeDeliveryStream.

If the status of a delivery stream is CREATING_FAILED, this status doesn't change, and you can't invoke CreateDeliveryStream again on it. However, you can invoke the DeleteDeliveryStream operation to delete it.

A Kinesis Data Firehose delivery stream can be configured to receive records directly from providers using PutRecord or PutRecordBatch, or it can be configured to use an existing Kinesis stream as its source. To specify a Kinesis data stream as input, set the DeliveryStreamType parameter to KinesisStreamAsSource, and provide the Kinesis stream Amazon Resource Name (ARN) and role ARN in the KinesisStreamSourceConfiguration parameter.

To create a delivery stream with server-side encryption (SSE) enabled, include DeliveryStreamEncryptionConfigurationInput in your request. This is optional. You can also invoke StartDeliveryStreamEncryption to turn on SSE for an existing delivery stream that doesn't have SSE enabled.

A delivery stream is configured with a single destination, such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon OpenSearch Service, Amazon OpenSearch Serverless, Splunk, and any custom HTTP endpoint or HTTP endpoints owned by or supported by third-party service providers, including Datadog, Dynatrace, LogicMonitor, MongoDB, New Relic, and Sumo Logic. You must specify only one of the following destination configuration parameters: ExtendedS3DestinationConfiguration, S3DestinationConfiguration, ElasticsearchDestinationConfiguration, RedshiftDestinationConfiguration, or SplunkDestinationConfiguration.

When you specify S3DestinationConfiguration, you can also provide the following optional values: BufferingHints, EncryptionConfiguration, and CompressionFormat. By default, if no BufferingHints value is provided, Kinesis Data Firehose buffers data up to 5 MB or for 5 minutes, whichever condition is satisfied first. BufferingHints is a hint, so there are some cases where the service cannot adhere to these conditions strictly. For example, record boundaries might be such that the size is a little over or under the configured buffering size. By default, no encryption is performed. We strongly recommend that you enable encryption to ensure secure data storage in Amazon S3.

A few notes about Amazon Redshift as a destination:

Kinesis Data Firehose assumes the IAM role that is configured as part of the destination. The role should allow the Kinesis Data Firehose principal to assume the role, and the role should have permissions that allow the service to deliver the data. For more information, see Grant Kinesis Data Firehose Access to an Amazon S3 Destination in the Amazon Kinesis Data Firehose Developer Guide.

Request Syntax

{
   "AmazonOpenSearchServerlessDestinationConfiguration": { 
      "BufferingHints": { 
         "IntervalInSeconds": number,
         "SizeInMBs": number
      },
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "CollectionEndpoint": "string",
      "IndexName": "string",
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RetryOptions": { 
         "DurationInSeconds": number
      },
      "RoleARN": "string",
      "S3BackupMode": "string",
      "S3Configuration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      },
      "VpcConfiguration": { 
         "RoleARN": "string",
         "SecurityGroupIds": [ "string" ],
         "SubnetIds": [ "string" ]
      }
   },
   "AmazonopensearchserviceDestinationConfiguration": { 
      "BufferingHints": { 
         "IntervalInSeconds": number,
         "SizeInMBs": number
      },
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "ClusterEndpoint": "string",
      "DocumentIdOptions": { 
         "DefaultDocumentIdFormat": "string"
      },
      "DomainARN": "string",
      "IndexName": "string",
      "IndexRotationPeriod": "string",
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RetryOptions": { 
         "DurationInSeconds": number
      },
      "RoleARN": "string",
      "S3BackupMode": "string",
      "S3Configuration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      },
      "TypeName": "string",
      "VpcConfiguration": { 
         "RoleARN": "string",
         "SecurityGroupIds": [ "string" ],
         "SubnetIds": [ "string" ]
      }
   },
   "DeliveryStreamEncryptionConfigurationInput": { 
      "KeyARN": "string",
      "KeyType": "string"
   },
   "DeliveryStreamName": "string",
   "DeliveryStreamType": "string",
   "ElasticsearchDestinationConfiguration": { 
      "BufferingHints": { 
         "IntervalInSeconds": number,
         "SizeInMBs": number
      },
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "ClusterEndpoint": "string",
      "DocumentIdOptions": { 
         "DefaultDocumentIdFormat": "string"
      },
      "DomainARN": "string",
      "IndexName": "string",
      "IndexRotationPeriod": "string",
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RetryOptions": { 
         "DurationInSeconds": number
      },
      "RoleARN": "string",
      "S3BackupMode": "string",
      "S3Configuration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      },
      "TypeName": "string",
      "VpcConfiguration": { 
         "RoleARN": "string",
         "SecurityGroupIds": [ "string" ],
         "SubnetIds": [ "string" ]
      }
   },
   "ExtendedS3DestinationConfiguration": { 
      "BucketARN": "string",
      "BufferingHints": { 
         "IntervalInSeconds": number,
         "SizeInMBs": number
      },
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "CompressionFormat": "string",
      "DataFormatConversionConfiguration": { 
         "Enabled": boolean,
         "InputFormatConfiguration": { 
            "Deserializer": { 
               "HiveJsonSerDe": { 
                  "TimestampFormats": [ "string" ]
               },
               "OpenXJsonSerDe": { 
                  "CaseInsensitive": boolean,
                  "ColumnToJsonKeyMappings": { 
                     "string" : "string" 
                  },
                  "ConvertDotsInJsonKeysToUnderscores": boolean
               }
            }
         },
         "OutputFormatConfiguration": { 
            "Serializer": { 
               "OrcSerDe": { 
                  "BlockSizeBytes": number,
                  "BloomFilterColumns": [ "string" ],
                  "BloomFilterFalsePositiveProbability": number,
                  "Compression": "string",
                  "DictionaryKeyThreshold": number,
                  "EnablePadding": boolean,
                  "FormatVersion": "string",
                  "PaddingTolerance": number,
                  "RowIndexStride": number,
                  "StripeSizeBytes": number
               },
               "ParquetSerDe": { 
                  "BlockSizeBytes": number,
                  "Compression": "string",
                  "EnableDictionaryCompression": boolean,
                  "MaxPaddingBytes": number,
                  "PageSizeBytes": number,
                  "WriterVersion": "string"
               }
            }
         },
         "SchemaConfiguration": { 
            "CatalogId": "string",
            "DatabaseName": "string",
            "Region": "string",
            "RoleARN": "string",
            "TableName": "string",
            "VersionId": "string"
         }
      },
      "DynamicPartitioningConfiguration": { 
         "Enabled": boolean,
         "RetryOptions": { 
            "DurationInSeconds": number
         }
      },
      "EncryptionConfiguration": { 
         "KMSEncryptionConfig": { 
            "AWSKMSKeyARN": "string"
         },
         "NoEncryptionConfig": "string"
      },
      "ErrorOutputPrefix": "string",
      "Prefix": "string",
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RoleARN": "string",
      "S3BackupConfiguration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      },
      "S3BackupMode": "string"
   },
   "HttpEndpointDestinationConfiguration": { 
      "BufferingHints": { 
         "IntervalInSeconds": number,
         "SizeInMBs": number
      },
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "EndpointConfiguration": { 
         "AccessKey": "string",
         "Name": "string",
         "Url": "string"
      },
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RequestConfiguration": { 
         "CommonAttributes": [ 
            { 
               "AttributeName": "string",
               "AttributeValue": "string"
            }
         ],
         "ContentEncoding": "string"
      },
      "RetryOptions": { 
         "DurationInSeconds": number
      },
      "RoleARN": "string",
      "S3BackupMode": "string",
      "S3Configuration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      }
   },
   "KinesisStreamSourceConfiguration": { 
      "KinesisStreamARN": "string",
      "RoleARN": "string"
   },
   "MSKSourceConfiguration": { 
      "AuthenticationConfiguration": { 
         "Connectivity": "string",
         "RoleARN": "string"
      },
      "MSKClusterARN": "string",
      "TopicName": "string"
   },
   "RedshiftDestinationConfiguration": { 
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "ClusterJDBCURL": "string",
      "CopyCommand": { 
         "CopyOptions": "string",
         "DataTableColumns": "string",
         "DataTableName": "string"
      },
      "Password": "string",
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RetryOptions": { 
         "DurationInSeconds": number
      },
      "RoleARN": "string",
      "S3BackupConfiguration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      },
      "S3BackupMode": "string",
      "S3Configuration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      },
      "Username": "string"
   },
   "S3DestinationConfiguration": { 
      "BucketARN": "string",
      "BufferingHints": { 
         "IntervalInSeconds": number,
         "SizeInMBs": number
      },
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "CompressionFormat": "string",
      "EncryptionConfiguration": { 
         "KMSEncryptionConfig": { 
            "AWSKMSKeyARN": "string"
         },
         "NoEncryptionConfig": "string"
      },
      "ErrorOutputPrefix": "string",
      "Prefix": "string",
      "RoleARN": "string"
   },
   "SplunkDestinationConfiguration": { 
      "CloudWatchLoggingOptions": { 
         "Enabled": boolean,
         "LogGroupName": "string",
         "LogStreamName": "string"
      },
      "HECAcknowledgmentTimeoutInSeconds": number,
      "HECEndpoint": "string",
      "HECEndpointType": "string",
      "HECToken": "string",
      "ProcessingConfiguration": { 
         "Enabled": boolean,
         "Processors": [ 
            { 
               "Parameters": [ 
                  { 
                     "ParameterName": "string",
                     "ParameterValue": "string"
                  }
               ],
               "Type": "string"
            }
         ]
      },
      "RetryOptions": { 
         "DurationInSeconds": number
      },
      "S3BackupMode": "string",
      "S3Configuration": { 
         "BucketARN": "string",
         "BufferingHints": { 
            "IntervalInSeconds": number,
            "SizeInMBs": number
         },
         "CloudWatchLoggingOptions": { 
            "Enabled": boolean,
            "LogGroupName": "string",
            "LogStreamName": "string"
         },
         "CompressionFormat": "string",
         "EncryptionConfiguration": { 
            "KMSEncryptionConfig": { 
               "AWSKMSKeyARN": "string"
            },
            "NoEncryptionConfig": "string"
         },
         "ErrorOutputPrefix": "string",
         "Prefix": "string",
         "RoleARN": "string"
      }
   },
   "Tags": [ 
      { 
         "Key": "string",
         "Value": "string"
      }
   ]
}

Request Parameters

The request accepts the following data in JSON format.

Response Syntax

{
   "DeliveryStreamARN": "string"
}

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

Errors

For information about the errors that are common to all actions, see Common Errors.

Examples

Example

The following JSON example creates a delivery stream named exampleStreamName with an Amazon S3 destination. To use this example, first replace the placeholders for the RoleARN and BucketARN keys with valid strings. For more information, see Amazon Resource Names (ARNs) and Amazon Service Namespaces.

Sample Request

POST / HTTP/1.1
Host: firehose.<region>.<domain>
Content-Length: <PayloadSizeBytes>
User-Agent: <UserAgentString>
Content-Type: application/x-amz-json-1.1
Authorization: <AuthParams>
Connection: Keep-Alive 
X-Amz-Date: <Date>
X-Amz-Target: Firehose_20150804.CreateDeliveryStream 
{
    "DeliveryStreamName": "exampleStreamName",
    "S3DestinationConfiguration": {
        "RoleARN": "insert-role-ARN",
        "BucketARN": "insert-bucket-ARN",
        "BufferingHints": {
            "SizeInMBs": 3,
            "IntervalInSeconds": 60
        },
        "CompressionFormat": "ZIP"
    }
}

Sample Response

HTTP/1.1 200 OK
x-amzn-RequestId: <RequestId>
Content-Type: application/x-amz-json-1.1
Content-Length: <PayloadSizeBytes>
Date: <Date>
{
    "DeliveryStreamARN": "arn:aws:firehose:us-east-1:814985986679:deliverystream/exampleStreamName"
}

See Also

For more information about using this API in one of the language-specific Amazon SDKs, see the following: