Managing CloudTrail Lake costs
Amazon CloudTrail Lake event data stores and queries incur charges. As a best practice, we recommend using Amazon Web Services and tools that can help you manage
CloudTrail costs. You can also configure event data stores in ways that capture the data you need
while remaining cost-effective. For information about
CloudTrail pricing, see Amazon CloudTrail
Pricing
Topics
Event data store pricing options
When you create an event data store, you choose the pricing option that you want to use for the event data store. The pricing option determines the cost for ingesting and storing events, and the default and maximum retention periods for the event data store.
The following table describes the available pricing options. The table shows the
Pricing option in the console and the corresponding
BillingMode
value for the API, and lists the default and maximum
retention period for each option.
Pricing option (console) | BillingMode (API) | Description |
---|---|---|
One-year extendable retention pricing |
|
Recommended if you expect to ingest less than 25 TB of event data per month and want a flexible retention period of up to 10 years. This option is also recommended if your event data store collects Amazon Config configuration items, Audit Manager evidence, and events from outside of Amazon. For the first 366 days (the default retention period), storage is included at no additional cost with ingestion pricing. After 366 days, extended retention is available at pay-as-you-go pricing. This is the default option. Default retention period: 366 days Maximum retention period: 3,653 days |
Seven-year retention pricing |
|
Recommended if expect to ingest more than 25 TB of event data per month and need a retention period of up to 7 years. Retention is included with ingestion pricing at no additional charge. Default retention period: 2,557 days Maximum retention period: 2,557 days |
Understanding CloudTrail Lake charges
The following tables provides information about how CloudTrail Lake event data stores and queries incur charges. For information about
CloudTrail pricing, see Amazon CloudTrail
Pricing
Charge type | How you incur charges |
---|---|
Data ingestion (uncompressed data) |
For CloudTrail Lake, you pay based on the uncompressed data ingested. The pricing option for the event data store determines the cost of ingesting events:
Copying trail events When you copy trail events to CloudTrail Lake, CloudTrail unzips the logs that are stored in gzip (compressed) format. Then CloudTrail copies the events contained in the logs to your event data store. The size of the uncompressed data could be greater than the actual Amazon S3 storage size. To get a general estimate of the size of the uncompressed data, multiply the size of the logs in the S3 bucket by 10. NoteCloudTrail will not copy an event if its event time is older than the specified retention period. To determine the appropriate retention period, take the sum of the oldest event you want to copy in days and the number of days you want to retain the events in the event data store as demonstrated in this equation: Retention period =
For example, if the oldest event you're copying is 45 days old and you want to keep the events in the event data store for a further 45 days, you would set the retention period to 90 days. |
Data retention (optimized and compressed data) |
CloudTrail Lake converts existing
events in row-based JSON format to Apache ORC An event data store’s retention period determines how long event data is kept in the event data store. CloudTrail Lake determines whether to retain an event by checking if an event's event time is within the specified retention period. For example, if you specify a retention period of 90 days, CloudTrail will remove events when their event time is older than 90 days. For event data stores using the Seven-year retention pricing option, storage is included with ingestion pricing at no additional charge. For event data stores using the One-year extendable retention pricing option, storage is included at no charge with ingestion pricing for the first 366 days (the default retention period). After 366 days, storage is offered at pay-as-you-pricing and is charged based on the optimized and compressed data in the event data store. |
Running queries in CloudTrail Lake (optimized and compressed data) |
When you run queries in CloudTrail Lake, you pay based on the amount of optimized and compressed data scanned. |
Recommendations for how you can reduce costs
This section provides recommendations for how you can reduce costs when working with CloudTrail Lake.
- Choose a pricing option based on the type of events your event data store will collect and your expected monthly ingestion
-
When creating an event data store, choose a pricing option based on the type of events your event data store will collect and your expected monthly ingestion.
If you expect to ingest less than 25 TB of event data on a monthly basis and want a flexible retention period of up to 10 years, choose the One-year extendable retention pricing option. We also generally recommend this option for event data stores that collect Amazon Config configuration items, Audit Manager evidence, and events from outside of Amazon.
If you expect to ingest more than 25 TB of event data on a monthly basis and need a 7-year retention period, choose the Seven-year retention pricing option.
- Evaluate your event data store's monthly ingestion over time
-
Evaluate the historical monthly ingestion of your event data store to see if there's a pricing option better suited to your needs.
If you have an existing event data store that uses the Seven-year retention pricing option and you ingest less than 25 TB of data on a monthly basis, consider updating the event data store to use One-year extendable retention pricing. For event data stores using the Seven-year retention pricing option, you can change the pricing option using the CloudTrail console, Amazon CLI, or UpdateEventDataStore API operation.
If you have an existing event data store that uses the One-year extendable retention pricing option and you ingest more than 25 TB of event data on a monthly basis, consider whether Seven-year retention pricing would better suit your needs. To use the new pricing option, stop ingestion on your event data store and create a new event data store with the Seven-year retention pricing option.
- Use advanced event selectors to filter out events that aren't of interest
-
When configuring an event data store for CloudTrail management or data events, filter out events that aren't of interest by using advanced event selectors.
If you're creating an event data store to collect management events, you can filter out Amazon Key Management Service (Amazon KMS) or Amazon Relational Database Service (Amazon RDS ) Data API events. Typically, Amazon KMS actions such as
Encrypt
,Decrypt
, andGenerateDataKey
generate more than 99 percent of events.If you're creating an event data store to collect data events, you can use advanced event selectors to filter on the
eventName
,resources.type
,resources.ARN
, andreadOnly
fields. For an example, see Example: Create an event data store for S3 data events. - Choose a narrower time range when copying trail events
-
When copying trail events to CloudTrail Lake, specify a narrower start event time and end event time to reduce the amount of data ingested.
If you are copying trail events to CloudTrail Lake for historical analysis and do not want to ingest future events, deselect the option to ingest events so that you do not incur charges on ingesting any additional events.
- Format queries to use a starting and ending
eventTime
-
When you run queries in Lake, you pay based upon the amount of data scanned. You can constrain costs by specifying a starting and ending
eventTime
for the query.
Tools to help manage costs
Amazon Budgets, a feature of Amazon Billing and Cost Management, lets you set custom budgets that alert you when your costs or usage exceed (or are forecasted to exceed) your budgeted amount.
As you create event data stores, creating a budget for CloudTrail by using Amazon Budgets is a recommended best practice, and can help you track your CloudTrail spending. Cost-based budgets help promote awareness of how much you might be billed for your CloudTrail use. Budget alerts notify you when your bill reaches a threshold that you define. When you receive a budget alert, you can make changes before the end of the billing cycle to manage your costs.
After you create a budget, you can use Amazon Cost Explorer to see how your CloudTrail costs are influencing your overall Amazon bill. In Amazon Cost Explorer, after adding CloudTrail to the Service filter, you can compare your historical CloudTrail spending to that of your current month-to-date (MTD) spending, by both Region and account. This feature helps you monitor and detect unexpected costs in your monthly CloudTrail spending. Additional features in Cost Explorer let you compare CloudTrail spending to monthly spending at the specific resource level, providing information about what might be driving cost increases or decreases in your bill.
To get started with Amazon Budgets, open Amazon Billing and Cost Management
Creating user-defined cost allocation tags for CloudTrail Lake event data stores
You can create user-defined cost allocation tags to track the query and ingestion costs for your CloudTrail Lake event data stores. A user-defined cost allocation tag is a key-value pair that you can associate with an event data store. After you activate cost allocation tags, Amazon uses the tags to organize your resource costs on your cost allocation report.
-
To create tags in the console, see step 9 of the To create an event data store for CloudTrail management or data events procedure.
-
To create tags using the CloudTrail API, see CreateEventDataStore and AddTags in the Amazon CloudTrail API Reference.
-
To create tags using the Amazon CLI, see create-event-data-store
and add-tags in the Amazon CLI Command Reference.
For more information about activating tags, see Activating user-defined cost allocation tags.