Publish flow logs to CloudWatch Logs - Amazon Virtual Private Cloud
IAM role for publishing flow logs to CloudWatch LogsPermissions for IAM principals that publish flow logs to CloudWatch LogsCreate a flow log that publishes to CloudWatch LogsView flow log recordsSearch flow log recordsProcess flow log records in CloudWatch Logs
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Publish flow logs to CloudWatch Logs

Flow logs can publish flow log data directly to Amazon CloudWatch.

When publishing to CloudWatch Logs, flow log data is published to a log group, and each network interface has a unique log stream in the log group. Log streams contain flow log records. You can create multiple flow logs that publish data to the same log group. If the same network interface is present in one or more flow logs in the same log group, it has one combined log stream. If you've specified that one flow log should capture rejected traffic, and the other flow log should capture accepted traffic, then the combined log stream captures all traffic.

In CloudWatch Logs, the timestamp field corresponds to the start time that's captured in the flow log record. The ingestionTime field indicates the date and time when the flow log record was received by CloudWatch Logs. This timestamp is later than the end time that's captured in the flow log record.

For more information about CloudWatch Logs, see Logs sent to CloudWatch Logs in the Amazon CloudWatch Logs User Guide.

Pricing

Data ingestion and archival charges for vended logs apply when you publish flow logs to CloudWatch Logs. For more information, open Amazon CloudWatch Pricing, select Logs and find Vended Logs.

IAM role for publishing flow logs to CloudWatch Logs

The IAM role that's associated with your flow log must have sufficient permissions to publish flow logs to the specified log group in CloudWatch Logs. The IAM role must belong to your Amazon account.

The IAM policy that's attached to your IAM role must include at least the following permissions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams"
      ],
      "Resource": "*"
    }
  ]
}

Ensure that your role has the following trust policy, which allows the flow logs service to assume the role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "vpc-flow-logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

We recommend that you use the aws:SourceAccount and aws:SourceArn condition keys to protect yourself against the confused deputy problem. For example, you could add the following condition block to the previous trust policy. The source account is the owner of the flow log and the source ARN is the flow log ARN. If you don't know the flow log ID, you can replace that portion of the ARN with a wildcard (*) and then update the policy after you create the flow log.

"Condition": {
    "StringEquals": {
        "aws:SourceAccount": "account_id"
    },
    "ArnLike": {
        "aws:SourceArn": "arn:aws-cn:ec2:region:account_id:vpc-flow-log/flow-log-id"
    }
}

Create an IAM role for flow logs

You can update an existing role as described above. Alternatively, you can use the following procedure to create a new role for use with flow logs. You'll specify this role when you create the flow log.

To create an IAM role for flow logs

  1. Open the IAM console at https://console.amazonaws.cn/iam/.

  2. In the navigation pane, choose Policies.

  3. Choose Create policy.

  4. On the Create policy page, do the following:

    1. Choose JSON.

    2. Replace the contents of this window with the permissions policy at the start of this section.

    3. Choose Next.

    4. Enter a name for your policy and an optional description and tags, and then choose Create policy.

  5. In the navigation pane, choose Roles.

  6. Choose Create role.

  7. For Trusted entity type, choose Custom trust policy. For Custom trust policy, replace "Principal": {}, with the following, then and choose Next.

    "Principal": {
   "Service": "vpc-flow-logs.amazonaws.com"
},

  8. On the Add permissions page, select the checkbox for the policy that you created earlier in this procedure, and then choose Next.

  9. Enter a name for your role and optionally provide a description.

  10. Choose Create role.

Permissions for IAM principals that publish flow logs to CloudWatch Logs

Users must have permissions to use the iam:PassRole action for the IAM role that's associated with the flow log.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["iam:PassRole"],
      "Resource": "arn:aws-cn:iam::account-id:role/flow-log-role-name"
    }
  ]
}

Create a flow log that publishes to CloudWatch Logs

You can create flow logs for your VPCs, subnets, or network interfaces. If you perform these steps as a user using a particular IAM role, ensure that the role has permissions to use the iam:PassRole action. For more information, see Permissions for IAM principals that publish flow logs to CloudWatch Logs.

Prerequisite
To create a flow log using the console

  1. Do one of the following:

  2. Choose Actions, Create flow log.

  3. For Filter, specify the type of traffic to log. Choose All to log accepted and rejected traffic, Reject to log only rejected traffic, or Accept to log only accepted traffic.

  4. For Maximum aggregation interval, choose the maximum period of time during which a flow is captured and aggregated into one flow log record.

  5. For Destination, choose Send to CloudWatch Logs.

  6. For Destination log group, choose the name of an existing log group or enter the name of a new log group that will be created when you create this flow log.

  7. For IAM role, specify the name of the role that has permissions to publish logs to CloudWatch Logs.

  8. For Log record format, select the format for the flow log record.

    • To use the default format, choose Amazon default format.

    • To use a custom format, choose Custom format and then select fields from Log format.

  9. (Optional) Choose Add new tag to apply tags to the flow log.

  10. Choose Create flow log.

To create a flow log using the command line

Use one of the following commands.

The following Amazon CLI example creates a flow log that captures all accepted traffic for the specified subnet. The flow logs are delivered to the specified log group. The --deliver-logs-permission-arn parameter specifies the IAM role required to publish to CloudWatch Logs.

aws ec2 create-flow-logs --resource-type Subnet --resource-ids subnet-1a2b3c4d --traffic-type ACCEPT --log-group-name my-flow-logs --deliver-logs-permission-arn arn:aws-cn:iam::123456789101:role/publishFlowLogs

View flow log records

You can view your flow log records using the CloudWatch Logs console. After you create your flow log, it might take a few minutes for it to be visible in the console.

To view flow log records published to CloudWatch Logs using the console

  1. Open the CloudWatch console at https://console.amazonaws.cn/cloudwatch/.

  2. In the navigation pane, choose Logs, Log groups.

  3. Select the name of the log group that contains your flow logs to open its details page.

  4. Select the name of the log stream that contains the flow log records. For more information, see Flow log records.

To view flow log records published to CloudWatch Logs using the command line

Search flow log records

You can search your flow log records that are published to CloudWatch Logs using the CloudWatch Logs console. You can use metric filters to filter flow log records. Flow log records are space delimited.

To search flow log records using the CloudWatch Logs console

  1. Open the CloudWatch console at https://console.amazonaws.cn/cloudwatch/.

  2. In the navigation pane, choose Logs, Log groups.

  3. Select the log group that contains your flow log, and then select the log stream, if you know the network interface that you are searching for. Alternatively, choose Search log group. This might take some time if there are many network interfaces in your log group, or depending on the time range that you select.

  4. Under Filter events, enter the string below. This assumes that the flow log record uses the default format.

    [version, accountid, interfaceid, srcaddr, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action, logstatus]

  5. Modify the filter as needed by specifying values for the fields. The following examples filter by specific source IP addresses.

    [version, accountid, interfaceid, srcaddr = 10.0.0.1, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action, logstatus]
[version, accountid, interfaceid, srcaddr = 10.0.2.*, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action, logstatus]

    The following examples filter by destination port, the number of bytes, and whether the traffic was rejected.

    [version, accountid, interfaceid, srcaddr, dstaddr, srcport, dstport = 80 || dstport = 8080, protocol, packets, bytes, start, end, action, logstatus]
[version, accountid, interfaceid, srcaddr, dstaddr, srcport, dstport = 80 || dstport = 8080, protocol, packets, bytes >= 400, start, end, action = REJECT, logstatus]

Process flow log records in CloudWatch Logs

You can work with flow log records as you would with any other log events collected by CloudWatch Logs. For more information about monitoring log data and metric filters, see Searching and Filtering Log Data in the Amazon CloudWatch User Guide.

Example: Create a CloudWatch metric filter and alarm for a flow log

In this example, you have a flow log for eni-1a2b3c4d. You want to create an alarm that alerts you if there have been 10 or more rejected attempts to connect to your instance over TCP port 22 (SSH) within a 1-hour time period. First, you must create a metric filter that matches the pattern of the traffic for which to create the alarm. Then, you can create an alarm for the metric filter.

To create a metric filter for rejected SSH traffic and create an alarm for the filter

  1. Open the CloudWatch console at https://console.amazonaws.cn/cloudwatch/.

  2. In the navigation pane, choose Logs, Log groups.

  3. Select the check box for the log group, and then choose Actions, Create metric filter.

  4. For Filter pattern, enter the following string.

    [version, account, eni, source, destination, srcport, destport="22", protocol="6", packets, bytes, windowstart, windowend, action="REJECT", flowlogstatus]

  5. For Select log data to test, select the log stream for your network interface. (Optional) To view the lines of log data that match the filter pattern, choose Test pattern.

  6. When you're ready, choose Next.

  7. Enter a filter name, metric namespace, and metric name. Set the metric value to 1. When you're done, choose Next and then choose Create metric filter.

  8. In the navigation pane, choose Alarms, All alarms.

  9. Choose Create alarm.

  10. Select the metric name that you created and then choose Select metric.

  11. Configure the alarm as follows, and then choose Next:

    • For Statistic, choose Sum. This ensure that you capture the total number of data points for the specified time period.

    • For Period, choose 1 hour.

    • For Whenever TimeSinceLastActive is..., choose Greater/Equal and enter 10 for the threshold.

    • For Additional configuration, Datapoints to alarm, leave the default of 1.

  12. Choose Next.

  13. For Notification, select an existing SNS topic or choose Create new topic to create a new one. Choose Next.

  14. Enter a name and description for the alarm and choose Next.

  15. When you are done previewing the alarm, choose Create alarm.