Enabling Amazon S3 server access logging
Server access logging provides detailed records for the requests that are made to an Amazon S3 bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits. This information can also help you learn about your customer base and understand your Amazon S3 bill.
By default, Amazon S3 doesn't collect server access logs. When you enable logging, Amazon S3 delivers access logs for a source bucket to a destination bucket (also known as a target bucket) that you choose. The destination bucket must be in the same Amazon Web Services Region and Amazon Web Services account as the source bucket.
An access log record contains details about the requests that are made to a bucket. This information can include the request type, the resources that are specified in the request, and the time and date that the request was processed. For more information about logging basics, see Logging requests with server access logging.
Important
-
There is no extra charge for enabling server access logging on an Amazon S3 bucket. However, any log files that the system delivers to you will accrue the usual charges for storage. (You can delete the log files at any time.) We do not assess data-transfer charges for log file delivery, but we do charge the normal data-transfer rate for accessing the log files.
-
Your destination bucket should not have server access logging enabled. You can have logs delivered to any bucket that you own that is in the same Region as the source bucket, including the source bucket itself. However, delivering logs to the source bucket will cause an infinite loop of logs and is not recommended. For simpler log management, we recommend that you save access logs in a different bucket. For more information, see How do I enable log delivery?
-
S3 buckets that have S3 Object Lock enabled can't be used as destination buckets for server access logs. Your destination bucket must not have a default retention period configuration.
-
The destination bucket must not have Requester Pays enabled.
-
You can use default bucket encryption on the destination bucket only if you use server-side encryption with Amazon S3 managed keys (SSE-S3), which uses the 256-bit Advanced Encryption Standard (AES-256). Default server-side encryption with Amazon Key Management Service (Amazon KMS) keys (SSE-KMS) is not supported.
You can enable or disable server access logging by using the Amazon S3 console, Amazon S3 API, the Amazon Command Line Interface (Amazon CLI), or Amazon SDKs.
Permissions for log delivery
Amazon S3 uses a special log delivery account to write server access logs. These writes are
subject to the usual access control restrictions. For access log delivery, you must grant the
logging service principal (logging.s3.amazonaws.com
) access to your destination
bucket.
To grant permissions to Amazon S3 for log delivery, you can use either a bucket policy or bucket access control lists (ACLs), depending on your destination bucket's S3 Object Ownership settings. However, we recommend that you use a bucket policy instead of ACLs.
Bucket owner enforced setting for S3 Object Ownership
If the destination bucket uses the Bucket owner enforced setting for Object Ownership, ACLs are disabled and no longer affect permissions. In this case, you must update the bucket policy for the destination bucket to grant access to the logging service principal. You can't update your bucket ACL to grant access to the S3 log delivery group. You also can't include destination grants (also known as target grants) in your PutBucketLogging configuration.
For information about migrating existing bucket ACLs for access log delivery to a bucket policy, see Grant access to the S3 log delivery group for server access logging. For more information about Object Ownership, see Controlling ownership of objects and disabling ACLs for your bucket. When you create new buckets, ACLs are disabled by default.
Granting access by using a bucket policy
To grant access by using the bucket policy on the destination bucket, update the bucket
policy to grant the s3:PutObject
permission to the logging service principal.
If you use the Amazon S3 console to enable server access logging, the console automatically
updates the bucket policy on the destination bucket to grant this permission to the logging
service principal. If you enable server access logging programmatically, you must manually
update the bucket policy for the destination bucket to grant access to the logging service
principal.
For an example bucket policy that grants access to the logging service principal, see Grant permissions to the logging service principal by using a bucket policy.
Granting access by using bucket ACLs
You can alternately use bucket ACLs to grant access for access log delivery. You add a
grant entry to the bucket ACL that grants WRITE
and READ_ACP
permissions to the S3 log delivery group. However, granting access to the S3 log delivery
group by using bucket ACLs is not recommended. For more information, see Controlling ownership of objects and disabling ACLs
for your bucket. For information
about migrating existing bucket ACLs for access log delivery to a bucket policy, see Grant access to
the
S3 log delivery group for server access logging. For an example ACL that grants
access to the logging service principal, see Grant permissions to the log delivery group
by using a bucket ACL.
Grant permissions to the logging service principal by using a bucket policy
This example bucket policy grants the s3:PutObject
permission to the
logging service principal (logging.s3.amazonaws.com
). To use this bucket
policy, replace the
with
your own information. In the following policy, user input placeholders
is the destination bucket where server access logs will be delivered, and
amzn-s3-demo-destination-bucket
is the source bucket.
amzn-s3-demo-source-bucket
is the optional
destination prefix (also known as a target prefix) that
you want to use for your log objects.
EXAMPLE-LOGGING-PREFIX
is the Amazon Web Services account that owns
the source bucket. SOURCE-ACCOUNT-ID
Note
If there are Deny
statements in your bucket policy, make sure that they
don't prevent Amazon S3 from delivering access logs.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "S3ServerAccessLogsPolicy", "Effect": "Allow", "Principal": { "Service": "logging.s3.amazonaws.com" }, "Action": [ "s3:PutObject" ], "Resource": "arn:aws-cn:s3:::
amzn-s3-demo-destination-bucket
/EXAMPLE-LOGGING-PREFIX
*", "Condition": { "ArnLike": { "aws:SourceArn": "arn:aws-cn:s3:::amzn-s3-demo-source-bucket
" }, "StringEquals": { "aws:SourceAccount": "SOURCE-ACCOUNT-ID
" } } } ] }
Grant permissions to the log delivery group by using a bucket ACL
Note
As a security best practice, Amazon S3 disables access control lists (ACLs) by default in all new buckets. For more information about ACL permissions in the Amazon S3 console, see Configuring ACLs.
Although we do not recommend this approach, you can grant permissions to the log
delivery group by using a bucket ACL. However, if the destination bucket uses the Bucket
owner enforced setting for Object Ownership, you can't set bucket or object ACLs. You also
can't include destination grants (also known as target
grants) in your PutBucketLogging configuration. Instead, you must use a bucket
policy to grant access to the logging service principal
(logging.s3.amazonaws.com
). For more information, see Permissions for log delivery.
In the bucket ACL, the log delivery group is represented by the following URL:
http://acs.amazonaws.com/groups/s3/LogDelivery
To grant WRITE
and READ_ACP
(ACL read) permissions, add the
following grants to the destination bucket ACL:
<Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group"> <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI> </Grantee> <Permission>WRITE</Permission> </Grant> <Grant> <Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Group"> <URI>http://acs.amazonaws.com/groups/s3/LogDelivery</URI> </Grantee> <Permission>READ_ACP</Permission> </Grant>
For examples of adding ACL grants programmatically, see Configuring ACLs.
Important
When you enable Amazon S3 server access logging by using Amazon CloudFormation on a bucket and you're
using ACLs to grant access to the S3 log delivery group, you must also add
"AccessControl": "LogDeliveryWrite"
to your CloudFormation template. Doing so
is important because you can grant those permissions only by creating an ACL for the
bucket, but you can't create custom ACLs for buckets in CloudFormation. You can use only
canned ACLs with CloudFormation.
To enable server access logging
To enable server access logging by using the Amazon S3 console, Amazon S3 REST API, Amazon SDKs, and Amazon CLI, use the following procedures.
Sign in to the Amazon Web Services Management Console and open the Amazon S3 console at https://console.amazonaws.cn/s3/
. -
In the Buckets list, choose the name of the bucket that you want to enable server access logging for.
-
Choose Properties.
-
In the Server access logging section, choose Edit.
-
Under Server access logging, choose Enable.
-
Under Destination bucket, specify a bucket and an optional prefix. If you specify a prefix, we recommend including a forward slash (
/
) after the prefix to make it easier to find your logs.Note
Specifying a prefix with a slash (
/
) makes it simpler for you to locate the log objects. For example, if you specify the prefix valuelogs/
, each log object that Amazon S3 creates begins with thelogs/
prefix in its key, as follows:logs/2013-11-01-21-32-16-E568B2907131C0C0
If you specify the prefix value
logs
, the log object appears as follows:logs2013-11-01-21-32-16-E568B2907131C0C0
-
Under Log object key format, do one of the following:
-
To choose non-date-based partitioning, choose [DestinationPrefix][YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString].
-
To choose date-based partitioning, choose [DestinationPrefix][SourceAccountId]/[SourceRegion]/[SourceBucket]/[YYYY]/[MM]/[DD]/[YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString], then choose S3 event time or Log file delivery time.
-
-
Choose Save changes.
When you enable server access logging on a bucket, the console both enables logging on the source bucket and updates the bucket policy for the destination bucket to grant the
s3:PutObject
permission to the logging service principal (logging.s3.amazonaws.com
). For more information about this bucket policy, see Grant permissions to the logging service principal by using a bucket policy.You can view the logs in the destination bucket. After you enable server access logging, it might take a few hours before the logs are delivered to the target bucket. For more information about how and when logs are delivered, see How are logs delivered?
For more information, see Viewing the properties for an S3 bucket.
To enable logging, you submit a PutBucketLogging request to add the logging configuration on the source bucket. The request specifies the destination bucket (also known as a target bucket) and, optionally, the prefix to be used with all log object keys.
The following example identifies
as the
destination bucket and amzn-s3-demo-destination-bucket
as the prefix. logs/
<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01"> <LoggingEnabled> <TargetBucket>
amzn-s3-demo-destination-bucket
</TargetBucket> <TargetPrefix>logs/
</TargetPrefix> </LoggingEnabled> </BucketLoggingStatus>
The following example identifies
as the
destination bucket, amzn-s3-demo-destination-bucket
as the prefix, and
logs/
EventTime
as the log object key format.
<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01"> <LoggingEnabled> <TargetBucket>
amzn-s3-demo-destination-bucket
</TargetBucket> <TargetPrefix>logs/
</TargetPrefix> <TargetObjectKeyFormat> <PartitionedPrefix> <PartitionDateSource>EventTime</PartitionDateSource> </PartitionedPrefix> </TargetObjectKeyFormat> </LoggingEnabled> </BucketLoggingStatus>
The log objects are written and owned by the S3 log delivery account, and the bucket owner is granted full permissions on the log objects. You can optionally use destination grants (also known as target grants) to grant permissions to other users so that they can access the logs. For more information, see PutBucketLogging.
Note
If the destination bucket uses the Bucket owner enforced setting for Object Ownership, you can't use destination grants to grant permissions to other users. To grant permissions to others, you can update the bucket policy on the destination bucket. For more information, see Permissions for log delivery.
To retrieve the logging configuration on a bucket, use the GetBucketLogging API operation.
To delete the logging configuration, you send a PutBucketLogging
request
with an empty BucketLoggingStatus
:
<BucketLoggingStatus xmlns="http://doc.s3.amazonaws.com/2006-03-01"> </BucketLoggingStatus>
To enable logging on a bucket, you can use either the Amazon S3 API or the Amazon SDK wrapper libraries.
The following examples enable logging on a bucket. You must create two buckets, a source bucket and a destination (target) bucket. The examples update the bucket ACL on the destination bucket first. They then grant the log delivery group the necessary permissions to write logs to the destination bucket, and then they enable logging on the source bucket.
These examples won't work on destination buckets that use the Bucket owner enforced setting for Object Ownership.
If the destination (target) bucket uses the Bucket owner enforced setting for Object Ownership, you can't set bucket or
object ACLs. You also can't include destination (target) grants in your
PutBucketLogging configuration.
You must use a bucket policy to grant access to the logging service principal (logging.s3.amazonaws.com
).
For more information, see Permissions for log delivery.
We recommend that you create a dedicated logging bucket in each Amazon Web Services Region that you
have S3 buckets in. Then have your Amazon S3 access logs delivered to that S3 bucket. For more
information and examples, see put-bucket-logging
If the destination (target) bucket uses the Bucket owner enforced setting for Object Ownership, you can't set bucket or
object ACLs. You also can't include destination (target) grants in your
PutBucketLogging configuration.
You must use a bucket policy to grant access to the logging service principal (logging.s3.amazonaws.com
).
For more information, see Permissions for log delivery.
Example — Enable access logs with five buckets across two Regions
In this example, you have the following five buckets:
-
amzn-s3-demo-source-bucket
-us-east-1 -
amzn-s3-demo-source-bucket1
-us-east-1 -
amzn-s3-demo-source-bucket2
-us-east-1 -
amzn-s3-demo-bucket1
-us-west-2 -
amzn-s3-demo-bucket2
-us-west-2
Note
The final step of the following procedure provides example bash scripts that you
can use to create your logging buckets and enable server access logging on these
buckets. To use those scripts, you must create the policy.json
and logging.json
files, as described in the following
procedure.
-
Create two logging destination buckets in the US West (Oregon) and US East (N. Virginia) Regions and give them the following names:
-
amzn-s3-demo-destination-bucket
-logs-us-east-1 -
amzn-s3-demo-destination-bucket1
-logs-us-west-2
-
-
Later in these steps, you will enable server access logging as follows:
-
logs to the S3 bucketamzn-s3-demo-source-bucket
-us-east-1
with the prefixamzn-s3-demo-destination-bucket
-logs-us-east-1amzn-s3-demo-source-bucket
-us-east-1 -
logs to the S3 bucketamzn-s3-demo-source-bucket1
-us-east-1
with the prefixamzn-s3-demo-destination-bucket
-logs-us-east-1amzn-s3-demo-source-bucket1
-us-east-1 -
logs to the S3 bucketamzn-s3-demo-source-bucket2
-us-east-1
with the prefixamzn-s3-demo-destination-bucket
-logs-us-east-1amzn-s3-demo-source-bucket2
-us-east-1 -
logs to the S3 bucketamzn-s3-demo-bucket1
-us-west-2
with the prefixamzn-s3-demo-destination-bucket1
-logs-us-west-2amzn-s3-demo-bucket1
-us-west-2 -
logs to the S3 bucketamzn-s3-demo-bucket2
-us-west-2
with the prefixamzn-s3-demo-destination-bucket1
-logs-us-west-2amzn-s3-demo-bucket2
-us-west-2
-
-
For each destination logging bucket, grant permissions for server access log delivery by using a bucket ACL or a bucket policy:
-
Update the bucket policy (Recommended) – To grant permissions to the logging service principal, use the following
put-bucket-policy
command. Replace
with the name of your destination bucket.amzn-s3-demo-destination-bucket
-logsaws s3api put-bucket-policy --bucket
--policy file://policy.jsonamzn-s3-demo-destination-bucket
-logsPolicy.json
is a JSON document in the current folder that contains the following bucket policy. To use this bucket policy, replace the
with your own information. In the following policy,user input placeholders
is the destination bucket where server access logs will be delivered, andamzn-s3-demo-destination-bucket
-logs
is the source bucket.amzn-s3-demo-source-bucket
is the Amazon Web Services account that owns the source bucket.SOURCE-ACCOUNT-ID
{ "Version": "2012-10-17", "Statement": [ { "Sid": "S3ServerAccessLogsPolicy", "Effect": "Allow", "Principal": { "Service": "logging.s3.amazonaws.com" }, "Action": [ "s3:PutObject" ], "Resource": "arn:aws-cn:s3:::
amzn-s3-demo-destination-bucket
-logs
/*", "Condition": { "ArnLike": { "aws:SourceArn": "arn:aws-cn:s3:::amzn-s3-demo-source-bucket
" }, "StringEquals": { "aws:SourceAccount": "SOURCE-ACCOUNT-ID
" } } } ] } -
Update the bucket ACL – To grant permissions to the S3 log delivery group, use the following
put-bucket-acl
command. Replace
with the name of your destination (target) bucket.amzn-s3-demo-destination-bucket
-logsaws s3api put-bucket-acl --bucket
--grant-write URI=http://acs.amazonaws.com/groups/s3/LogDelivery --grant-read-acp URI=http://acs.amazonaws.com/groups/s3/LogDeliveryamzn-s3-demo-destination-bucket
-logs
-
-
Then, create a
logging.json
file that contains your logging configuration (based on one of the three examples that follow). After you create thelogging.json
file, you can apply the logging configuration by using the followingput-bucket-logging
command. Replace
with the name of your destination (target) bucket.amzn-s3-demo-destination-bucket
-logsaws s3api put-bucket-logging --bucket
--bucket-logging-status file://logging.jsonamzn-s3-demo-destination-bucket
-logsNote
Instead of using this
put-bucket-logging
command to apply the logging configuration on each destination bucket, you can use one of the bash scripts provided in the next step. To use those scripts, you must create thepolicy.json
andlogging.json
files, as described in this procedure.The
logging.json
file is a JSON document in the current folder that contains your logging configuration. If a destination bucket uses the Bucket owner enforced setting for Object Ownership, your logging configuration can't contain destination (target) grants. For more information, see Permissions for log delivery.Example –
logging.json
without destination (target) grantsThe following example
logging.json
file doesn't contain destination (target) grants. Therefore, you can apply this configuration to a destination (target) bucket that uses the Bucket owner enforced setting for Object Ownership.{ "LoggingEnabled": { "TargetBucket": "
amzn-s3-demo-destination-bucket
-logs
", "TargetPrefix": "amzn-s3-demo-destination-bucket
/" } }Example –
logging.json
with destination (target) grantsThe following example
logging.json
file contains destination (target) grants.If the destination bucket uses the Bucket owner enforced setting for Object Ownership, you can't include destination (target) grants in your PutBucketLogging configuration. For more information, see Permissions for log delivery.
{ "LoggingEnabled": { "TargetBucket": "
amzn-s3-demo-destination-bucket
-logs
", "TargetPrefix": "amzn-s3-demo-destination-bucket
/", "TargetGrants": [ { "Grantee": { "Type": "AmazonCustomerByEmail", "EmailAddress": "user@example.com
" }, "Permission": "FULL_CONTROL" } ] } }Example –
logging.json
with the log object key format set to S3 event timeThe following
logging.json
file changes the log object key format to S3 event time. For more information about setting the log object key format, see How do I enable log delivery?{ "LoggingEnabled": { "TargetBucket": "
amzn-s3-demo-destination-bucket
-logs
", "TargetPrefix": "amzn-s3-demo-destination-bucket
/", "TargetObjectKeyFormat": { "PartitionedPrefix": { "PartitionDateSource": "EventTime" } } } } -
Use one of the following bash scripts to add access logging for all the buckets in your account. Replace
with the name of your destination (target) bucket, and replaceamzn-s3-demo-destination-bucket
-logs
with the name of the Region that your buckets are located in.us-west-2
Note
This script works only if all of your buckets are in the same Region. If you have buckets in multiple Regions, you must adjust the script.
Example – Grant access with bucket policies and add logging for the buckets in your account
loggingBucket='
amzn-s3-demo-destination-bucket
-logs
' region='us-west-2
' # Create the logging bucket. aws s3 mb s3://$loggingBucket --region $region aws s3api put-bucket-policy --bucket $loggingBucket --policy file://policy.json # List the buckets in this account. buckets="$(aws s3 ls | awk '{print $3}')" # Put a bucket logging configuration on each bucket. for bucket in $buckets do # This if statement excludes the logging bucket. if [ "$bucket" != "$loggingBucket" ] ; then continue; fi printf '{ "LoggingEnabled": { "TargetBucket": "%s", "TargetPrefix": "%s/" } }' "$loggingBucket" "$bucket" > logging.json aws s3api put-bucket-logging --bucket $bucket --bucket-logging-status file://logging.json echo "$bucket done" done rm logging.json echo "Complete"Example – Grant access with bucket ACLs and add logging for the buckets in your account
loggingBucket='
amzn-s3-demo-destination-bucket
-logs
' region='us-west-2
' # Create the logging bucket. aws s3 mb s3://$loggingBucket --region $region aws s3api put-bucket-acl --bucket $loggingBucket --grant-write URI=http://acs.amazonaws.com/groups/s3/LogDelivery --grant-read-acp URI=http://acs.amazonaws.com/groups/s3/LogDelivery # List the buckets in this account. buckets="$(aws s3 ls | awk '{print $3}')" # Put a bucket logging configuration on each bucket. for bucket in $buckets do # This if statement excludes the logging bucket. if [ "$bucket" != "$loggingBucket" ] ; then continue; fi printf '{ "LoggingEnabled": { "TargetBucket": "%s", "TargetPrefix": "%s/" } }' "$loggingBucket" "$bucket" > logging.json aws s3api put-bucket-logging --bucket $bucket --bucket-logging-status file://logging.json echo "$bucket done" done rm logging.json echo "Complete"
Verifying your server access logs setup
After you enable server access logging, complete the following steps:
-
Access the destination bucket and verify that the log files are being delivered. After the access logs are set up, Amazon S3 immediately starts capturing requests and logging them. However, it might take a few hours before the logs are delivered to the destination bucket. For more information, see Bucket logging status changes take effect over time and Best-effort server log delivery.
You can also automatically verify log delivery by using Amazon S3 request metrics and setting up Amazon CloudWatch alarms for these metrics. For more information, see Monitoring metrics with Amazon CloudWatch.
-
Verify that you are able to open and read the contents of the log files.
For server access logging troubleshooting information, see Troubleshoot server access logging.