Create an event data store for CloudTrail Insights events with the console - Amazon CloudTrail
To create a destination event data store that logs Insights eventsTo create a source event data store that enables Insights events
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Create an event data store for CloudTrail Insights events with the console

Amazon CloudTrail Insights help Amazon users identify and respond to unusual activity associated with API calls and API error rates by continuously analyzing CloudTrail management events. CloudTrail Insights analyze your normal patterns of API call volume and API error rates, also called the baseline, and generate Insights events when the call volume or error rates are outside normal patterns. Insights events on API call volume are generated for write management APIs, and Insights events on API error rate are generated for both read and write management APIs.

To log Insights events in CloudTrail Lake, you need a destination event data store that logs Insights events and a source event data store that enables Insights and logs management events.

Note

To log Insights events on API call volume, the source event data store must log write management events. To log Insights events on API error rate, the source event data store must log read or write management events.

If you have CloudTrail Insights enabled on a source event data store and CloudTrail detects unusual activity, CloudTrail delivers Insights events to your destination event data store. Unlike other types of events captured in a CloudTrail event data store, Insights events are logged only when CloudTrail detects changes in your account's API usage that differ significantly from the account's typical usage patterns.

After you enable CloudTrail Insights for the first time on an event data store, it can take up to 7 days for CloudTrail to deliver the first Insights event, if unusual activity is detected.

CloudTrail Insights analyzes management events that occur in a single Region, not globally. A CloudTrail Insights event is generated in the same Region as its supporting management events are generated.

For an organization event data store, CloudTrail analyzes management events from each member's account instead of analyzing the aggregation of all management events for the organization.

Additional charges apply for ingesting Insights events in CloudTrail Lake. You will be charged separately if you enable Insights for both trails and CloudTrail Lake event data stores. For information about CloudTrail pricing, see Amazon CloudTrail Pricing.

To create a destination event data store that logs Insights events

When you create an Insights event data store, you have the option to choose an existing source event data store that logs management events and then specify the Insights types you want to receive. Or, you can alternatively enable Insights on a new or existing event data store after you create your Insights event data store and then choose this event data store as the destination event data store.

This procedure shows you how to create a destination event data store that logs Insights events.

  1. Sign in to the Amazon Web Services Management Console and open the CloudTrail console at https://console.amazonaws.cn/cloudtrail/.

  2. From the navigation pane, open the Lake submenu, then choose Event data stores.

  3. Choose Create event data store.

  4. On the Configure event data store page, in General details, enter a name for the event data store. A name is required.

  5. Choose the Pricing option that you want to use for your event data store. The pricing option determines the cost for ingesting and storing events, and the default and maximum retention periods for your event data store. For more information, see Amazon CloudTrail Pricing and Managing CloudTrail Lake costs.

    The following are the available options:

    • One-year extendable retention pricing - Generally recommended if you expect to ingest less than 25 TB of event data per month and want a flexible retention period of up to 10 years. For the first 366 days (the default retention period), storage is included at no additional charge with ingestion pricing. After 366 days, extended retention is available at pay-as-you-go pricing. This is the default option.

      • Default retention period: 366 days

      • Maximum retention period: 3,653 days

    • Seven-year retention pricing - Recommended if you expect to ingest more than 25 TB of event data per month and need a retention period of up to 7 years. Retention is included with ingestion pricing at no additional charge.

      • Default retention period: 2,557 days

      • Maximum retention period: 2,557 days

  6. Specify a retention period for the event data store in days. Retention periods can be between 7 days and 3,653 days (about 10 years) for the One-year extendable retention pricing option, or between 7 days and 2,557 days (about seven years) for the Seven-year retention pricing option. The event data store retains event data for the specified number of days.

  7. (Optional) To enable encryption using Amazon Key Management Service, choose Use my own Amazon KMS key. Choose New to have an Amazon KMS key created for you, or choose Existing to use an existing KMS key. In Enter KMS alias, specify an alias, in the format alias/MyAliasName. Using your own KMS key requires that you edit your KMS key policy to allow CloudTrail logs to be encrypted and decrypted. For more information, see Configure Amazon KMS key policies for CloudTrail. CloudTrail also supports Amazon KMS multi-Region keys. For more information about multi-Region keys, see Using multi-Region keys in the Amazon Key Management Service Developer Guide.

    Using your own KMS key incurs Amazon KMS costs for encryption and decryption. After you associate an event data store with a KMS key, the KMS key cannot be removed or changed.

    Note

    To enable Amazon Key Management Service encryption for an organization event data store, you must use an existing KMS key for the management account.

  8. (Optional) If you want to query against your event data using Amazon Athena, choose Enable in Lake query federation. Federation lets you view the metadata associated with the event data store in the Amazon Glue Data Catalog and run SQL queries against the event data in Athena. The table metadata stored in the Amazon Glue Data Catalog lets the Athena query engine know how to find, read, and process the data that you want to query. For more information, see Federate an event data store.

    To enable Lake query federation, choose Enable and then do the following:

    1. Choose whether you want to create a new role or use an existing IAM role. Amazon Lake Formation uses this role to manage permissions for the federated event data store. When you create a new role using the CloudTrail console, CloudTrail automatically creates a role with the required permissions. If you choose an existing role, be sure the policy for the role provides the required minimum permissions.

    2. If you are creating a new role, enter a name to identify the role.

    3. If you are using an existing role, choose the role you want to use. The role must exist in your account.

  9. (Optional) In the Tags section, you can add up to 50 tag key pairs to help you identify, sort, and control access to your event data store. For more information about how to use IAM policies to authorize access to an event data store based on tags, see Examples: Denying access to create or delete event data stores based on tags. For more information about how you can use tags in Amazon, see Tagging Amazon resources in the Amazon Web Services General Reference.

  10. Choose Next to configure the event data store.

  11. On the Choose events page, choose Amazon events, and then choose CloudTrail Insights events.

  12. In CloudTrail Insights events, do the following.

    1. Choose Allow delegated administrator access if you want to give your organization's delegated administrator access to this event data store. This option is only available if you are signed in with the management account for an Amazon Organizations organization.

    2. (Optional) Choose an existing source event data store that logs management events and specify the Insights types you want to receive.

      To add a source event data store, do the following.

      1. Choose Add source event data store.

      2. Choose the source event data store.

      3. Choose the Insights type that you want to receive.

        • ApiCallRateInsight – The ApiCallRateInsight Insights type analyzes write-only management API calls that are aggregated per minute against a baseline API call volume. To receives Insights on ApiCallRateInsight, the source event data store must log Write management events.

        • ApiErrorRateInsight – The ApiErrorRateInsight Insights type analyzes management API calls that result in error codes. The error is shown if the API call is unsuccessful. To receive Insights on ApiErrorRateInsight, the source event data store must log Write or Read management events.

      4. Repeat the previous two steps (ii and iii) to add any additional Insights types you want to receive.

  13. Choose Next to review your choices.

  14. On the Review and create page, review your choices. Choose Edit to make changes to a section. When you're ready to create the event data store, choose Create event data store.

  15. The new event data store is visible in the Event data stores table on the Event data stores page.

  16. If you did not choose a source event data store in step 10, follow the steps in To create a source event data store that enables Insights events to create a source event data store.

To create a source event data store that enables Insights events

This procedure shows you how to create a source event data store that enables Insights events and logs management events.

  1. Sign in to the Amazon Web Services Management Console and open the CloudTrail console at https://console.amazonaws.cn/cloudtrail/.

  2. From the navigation pane, open the Lake submenu, then choose Event data stores.

  3. Choose Create event data store.

  4. On the Configure event data store page, in General details, enter a name for the event data store. A name is required.

  5. Choose the Pricing option that you want to use for your event data store. The pricing option determines the cost for ingesting and storing events, and the default and maximum retention periods for your event data store. For more information, see Amazon CloudTrail Pricing and Managing CloudTrail Lake costs.

    The following are the available options:

    • One-year extendable retention pricing - Generally recommended if you expect to ingest less than 25 TB of event data per month and want a flexible retention period of up to 10 years. For the first 366 days (the default retention period), storage is included at no additional charge with ingestion pricing. After 366 days, extended retention is available at pay-as-you-go pricing. This is the default option.

      • Default retention period: 366 days

      • Maximum retention period: 3,653 days

    • Seven-year retention pricing - Recommended if you expect to ingest more than 25 TB of event data per month and need a retention period of up to 7 years. Retention is included with ingestion pricing at no additional charge.

      • Default retention period: 2,557 days

      • Maximum retention period: 2,557 days

  6. Specify a retention period for the event data store. Retention periods can be between 7 days and 3,653 days (about 10 years) for the One-year extendable retention pricing option, or between 7 days and 2,557 days (about seven years) for the Seven-year retention pricing option.

    CloudTrail Lake determines whether to retain an event by checking if the eventTime of the event is within the specified retention period. For example, if you specify a retention period of 90 days, CloudTrail will remove events when their eventTime is older than 90 days.

  7. (Optional) To enable encryption using Amazon Key Management Service, choose Use my own Amazon KMS key. Choose New to have an Amazon KMS key created for you, or choose Existing to use an existing KMS key. In Enter KMS alias, specify an alias, in the format alias/MyAliasName. Using your own KMS key requires that you edit your KMS key policy to allow CloudTrail logs to be encrypted and decrypted. For more information, see Configure Amazon KMS key policies for CloudTrail. CloudTrail also supports Amazon KMS multi-Region keys. For more information about multi-Region keys, see Using multi-Region keys in the Amazon Key Management Service Developer Guide.

    Using your own KMS key incurs Amazon KMS costs for encryption and decryption. After you associate an event data store with a KMS key, the KMS key cannot be removed or changed.

    Note

    To enable Amazon Key Management Service encryption for an organization event data store, you must use an existing KMS key for the management account.

  8. (Optional) If you want to query against your event data using Amazon Athena, choose Enable in Lake query federation. Federation lets you view the metadata associated with the event data store in the Amazon Glue Data Catalog and run SQL queries against the event data in Athena. The table metadata stored in the Amazon Glue Data Catalog lets the Athena query engine know how to find, read, and process the data that you want to query. For more information, see Federate an event data store.

    To enable Lake query federation, choose Enable and then do the following:

    1. Choose whether you want to create a new role or use an existing IAM role. Amazon Lake Formation uses this role to manage permissions for the federated event data store. When you create a new role using the CloudTrail console, CloudTrail automatically creates a role with the required permissions. If you choose an existing role, be sure the policy for the role provides the required minimum permissions.

    2. If you are creating a new role, enter a name to identify the role.

    3. If you are using an existing role, choose the role you want to use. The role must exist in your account.

  9. (Optional) In the Tags section, you can add up to 50 tag key pairs to help you identify, sort, and control access to your event data store. For more information about how to use IAM policies to authorize access to an event data store based on tags, see Examples: Denying access to create or delete event data stores based on tags. For more information about how you can use tags in Amazon, see Tagging Amazon resources in the Amazon Web Services General Reference.

  10. Choose Next to configure the event data store.

  11. On the Choose events page, choose Amazon events, and then choose CloudTrail events.

  12. In CloudTrail events, leave Management events selected.

  13. To have your event data store collect events from all accounts in an Amazon Organizations organization, select Enable for all accounts in my organization. You must be signed in to the management account for the organization to create an event data store that enables Insights.

  14. Expand Additional settings to choose whether you want your event data store to collect events for all Amazon Web Services Regions, or only the current Amazon Web Services Region, and choose whether the event data store ingests events. By default, your event data store collects events from all Regions in your account and starts ingesting events when it's created.

    1. Choose Include only the current region in my event data store if you want to include only events that are logged in the current Region. If you do not choose this option, your event data store includes events from all Regions.

    2. Leave Ingest events selected.

  15. Choose the type of management events you want to include in your event data store. You can choose Read, Write, or both. At least one is required.

    Note

    To log Insights events on API call volume, the event data store must log write management events. To log Insights events on API error rate, the event data store must log read or write management events.

  16. You can choose to exclude Amazon Key Management Service or Amazon RDS Data API events from your event data store. For more information about these options, see Logging management events.

  17. Choose Enable Insights.

  18. In Enable Insights, choose the destination event store that will log Insights events. The destination event data store will collect Insights events based upon the management event activity in this event data store. For information about how to create the destination event data store, see To create a destination event data store that logs Insights events.

  19. Choose the Insights types. You can choose API call rate, API error rate, or both. You must be logging Write management events to log Insights events for API call rate. You must be logging Read or Write management events to log Insights events for API error rate.

  20. Choose Next to review your choices.

  21. On the Review and create page, review your choices. Choose Edit to make changes to a section. When you're ready to create the event data store, choose Create event data store.

  22. The new event data store is visible in the Event data stores table on the Event data stores page.

    From this point forward, the event data store captures events that match its advanced event selectors. After you enable CloudTrail Insights for the first time on your source event data store, it can take up to 7 days for CloudTrail to deliver the first Insights event to your destination event data store, if unusual activity is detected.

    You can view the CloudTrail Lake dashboard to visualize the Insights events in your destination event data store. For more information about Lake dashboards, see View CloudTrail Lake dashboards.

Additional charges apply for ingesting Insights events in CloudTrail Lake. You will be charged separately if you enable Insights for both trails and event data stores. For information about CloudTrail pricing, see Amazon CloudTrail Pricing.