Publish flow logs to Firehose - Amazon VPC
IAM roles for cross account deliveryCreate a flow log that publishes to Firehose
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Publish flow logs to Firehose

Flow logs can publish flow log data directly to Firehose. You can choose to publish flow logs to the same account as the resource monitor or to a different account.

Prerequisities

When publishing to Firehose, flow log data is published to a Firehose delivery stream, in plain text format. You must first have created a Firehose delivery stream. For the steps to create a delivery stream, see Creating an Amazon Data Firehose Delivery Stream in the Amazon Data Firehose Developer Guide.

Pricing

Standard ingestion and delivery charges apply. For more information, open Amazon CloudWatch Pricing, select Logs and find Vended Logs.

IAM roles for cross account delivery

When you publish to Kinesis Data Firehose, you can choose a delivery stream that's in the same account as the resource to monitor (the source account), or in a different account (the destination account). To enable cross account delivery of flow logs to Firehose, you must create an IAM role in the source account and an IAM role in the destination account.

Source account role

In the source account, create a role that grants the following permissions. In this example, the name of the role is mySourceRole, but you can choose a different name for this role. The last statement allows the role in the destination account to assume this role. The condition statements ensure that this role is passed only to the log delivery service, and only when monitoring the specified resource. When you create your policy, specify the VPCs, network interfaces, or subnets that you're monitoring with the condition key iam:AssociatedResourceARN.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "arn:aws:iam::source-account:role/mySourceRole",
      "Condition": {
          "StringEquals": {
              "iam:PassedToService": "delivery.logs.amazonaws.com"
          },
          "StringLike": {
              "iam:AssociatedResourceARN": [
                  "arn:aws:ec2:region:source-account:transit-gateway/tgw-0fb8421e2da853bf"
              ]
          }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
          "logs:CreateLogDelivery",
          "logs:DeleteLogDelivery",
          "logs:ListLogDeliveries",
          "logs:GetLogDelivery"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::destination-account:role/AWSLogDeliveryFirehoseCrossAccountRole"      
    }
  ]
}

Ensure that this role has the following trust policy, which allows the log delivery service to assume the role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "delivery.logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

From the source account, use the following procedure to create the role.

To create the source account role

  1. Sign in to the Amazon Web Services Management Console and open the IAM console at https://console.amazonaws.cn/iam/.

  2. In the navigation pane, choose Policies.

  3. Choose Create policy.

  4. On the Create policy page, do the following:

    1. Choose JSON.

    2. Replace the contents of this window with the permissions policy at the start of this section.

    3. Choose Next: Tags and Next: Review.

    4. Enter a name for your policy and an optional description, and then choose Create policy.

  5. In the navigation pane, choose Roles.

  6. Choose Create role.

  7. For the Trusted entity type, choose Custom trust policy. For Custom trust policy, replace "Principal": {}, with the following, which specifies the log delivery service. Choose Next.

    "Principal": {
   "Service": "delivery.logs.amazonaws.com"
},

  8. On the Add permissions page, select the checkbox for the policy that you created earlier in this procedure, and then choose Next.

  9. Enter a name for your role and optionally provide a description.

  10. Choose Create role.

Destination account role

In the destination account, create a role with a name that starts with AWSLogDeliveryFirehoseCrossAccountRole. This role must grant the following permissions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
          "iam:CreateServiceLinkedRole",
          "firehose:TagDeliveryStream"
      ],
      "Resource": "*"
    }
  ]
}

Ensure that this role has the following trust policy, which allows the role that you created in the source account to assume this role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
          "AWS": "arn:aws:iam::source-account:role/mySourceRole"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

From the destination account, use the following procedure to create the role.

To create the destination account role

  1. Sign in to the Amazon Web Services Management Console and open the IAM console at https://console.amazonaws.cn/iam/.

  2. In the navigation pane, choose Policies.

  3. Choose Create policy.

  4. On the Create policy page, do the following:

    1. Choose JSON.

    2. Replace the contents of this window with the permissions policy at the start of this section.

    3. Choose Next: Tags and Next: Review.

    4. Enter a name for your policy that starts with AWSLogDeliveryFirehoseCrossAccountRole, and then choose Create policy.

  5. In the navigation pane, choose Roles.

  6. Choose Create role.

  7. For the Trusted entity type, choose Custom trust policy. For Custom trust policy, replace "Principal": {}, with the following, which specifies the log delivery service. Choose Next.

    "Principal": {
   "AWS": "arn:aws:iam::source-account:role/mySourceRole"
},

  8. On the Add permissions page, select the checkbox for the policy that you created earlier in this procedure, and then choose Next.

  9. Enter a name for your role and optionally provide a description.

  10. Choose Create role.

Create a flow log that publishes to Firehose

To create a transit gateway flow log that publishes to Firehose using the console

  1. Open the Amazon VPC console at https://console.amazonaws.cn/vpc/.

  2. In the navigation pane, choose Transit gateways or Transit gateway attachments.

  3. Select the checkboxes for one or more transit gateways or transit gateway attachments.

  4. Choose Actions, Create flow log.

  5. For Destination choose Send to a Firehose Delivery System.

  6. For the Firehose Delivery Stream ARN, choose the ARN of a delivery stream you created where the flow log is to be published.

  7. For Log record format, specify the format for the flow log record.

    • To use the default flow log record format, choose Amazon default format.

    • To create a custom format, choose Custom format. For Log format, choose the fields to include in the flow log record.

  8. (Optional) To add a tag to the flow log, choose Add new tag and specify the tag key and value.

  9. Choose Create flow log.

To create a flow log that publishes to Firehose using the command line tool

Use one of the following commands:

The following Amazon CLI example creates a flow log that captures transit gateway information and delivers the flow log to the specified Firehose delivery stream.

aws ec2 create-flow-logs \ 
                --resource-type TransitGateway \ 
                --resource-ids tgw-1a2b3c4d \ 
                --log-destination-type kinesis-data-firehose \
                --log-destination arn:aws:firehose:us-east-1:123456789012:deliverystream:flowlogs_stream

The following Amazon CLI example creates a flow log that captures transit gateway information and delivers the flow log to a different Firehose delivery stream from the source account.

aws ec2 create-flow-logs  \
  --resource-type TransitGateway \
  --resource-ids gw-1a2b3c4d \
  --log-destination-type kinesis-data-firehose \
  --log-destination arn:aws:firehose:us-east-1:123456789012:deliverystream:flowlogs_stream \
  --deliver-logs-permission-arn arn:aws:iam::source-account:role/mySourceRole \ 
  --deliver-cross-account-role arn:aws:iam::destination-account:role/AWSLogDeliveryFirehoseCrossAccountRole