Understand CloudTrail logs and Athena tables
Before you begin creating tables, you should understand a little more about CloudTrail and how it stores data. This can help you create the tables that you need, whether you create them from the CloudTrail console or from Athena.
CloudTrail saves logs as JSON text files in compressed gzip format (*.json.gzip). The location of the log files depends on how you set up trails, the Amazon Web Services Region or Regions in which you are logging, and other factors.
For more information about where logs are stored, the JSON structure, and the record file contents, see the following topics in the Amazon CloudTrail User Guide:
To collect logs and save them to Amazon S3, enable CloudTrail from the Amazon Web Services Management Console. For more information, see Creating a trail in the Amazon CloudTrail User Guide.
Note the destination Amazon S3 bucket where you save the logs. Replace the
LOCATION
clause with the path to the CloudTrail log location and the set of
objects with which to work. The example uses a LOCATION
value of logs for a
particular account, but you can use the degree of specificity that suits your
application.
For example:
-
To analyze data from multiple accounts, you can roll back the
LOCATION
specifier to indicate allAWSLogs
by usingLOCATION 's3://amzn-s3-demo-bucket/AWSLogs/'
. -
To analyze data from a specific date, account, and Region, use
LOCATION 's3://amzn-s3-demo-bucket/123456789012/CloudTrail/us-east-1/2016/03/14/'.
Using the highest level in the object hierarchy gives you the greatest flexibility when you query using Athena.