Sign in to the AWS Management Console and open the Amazon S3 console
In the Buckets list, choose the name of the bucket that you want to enable server access logging for.
Choose Properties.
In the Server access logging section, choose Edit.
Under Server access logging, choose Enable.
Under Destination bucket, specify a bucket and an optional prefix. If you specify a prefix, we recommend including a forward slash (/) after the prefix to make it easier to find your logs
Under Log object key format, choose default
Choose Save changes.
Querying Access Logs for Requests Using Amazon Athena
Step 1: Create a Database
CREATE DATABASE s3_access_logs_db;
Step 2: Create a Table Schema
CREATE EXTERNAL TABLE `s3_access_logs_db.mybucket_logs`(
`bucketowner` STRING,
`bucket_name` STRING,
`requestdatetime` STRING,
`remoteip` STRING,
`requester` STRING,
`requestid` STRING,
`operation` STRING,
`key` STRING,
`request_uri` STRING,
`httpstatus` STRING,
`errorcode` STRING,
`bytessent` BIGINT,
`objectsize` BIGINT,
`totaltime` STRING,
`turnaroundtime` STRING,
`referrer` STRING,
`useragent` STRING,
`versionid` STRING,
`hostid` STRING,
`sigv` STRING,
`ciphersuite` STRING,
`authtype` STRING,
`endpoint` STRING,
`tlsversion` STRING,
`accesspointarn` STRING,
`aclrequired` STRING)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
'input.regex'='([^ ]*) ([^ ]*) \\[(.*?)\\] ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\"[^\"]*\"|-) (-|[0-9]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) (\"[^\"]*\"|-) ([^ ]*)(?: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*))?.*$')
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://amzn-s3-demo-bucket1-logs/prefix/';
Step 3: Query Access Logs
SELECT bucket_name, requester, remoteip, key, errorcode, httpstatus, requestdatetime
FROM s3_access_logs_db.mybucket_logs
WHERE httpstatus NOT IN ('200', '201', '204', '206')
AND (requestdatetime LIKE '%23/Sep/2024%' OR requestdatetime LIKE '%25/Sep/2024%');
This query will retrieve all PUT object requests where the HTTP status indicates a potential issue (i.e., not in the successful codes 200, 201, 204, or 206) on September 23 and 25, 2024.
You can change the date according to your need.
AWS Official Documentation for Reference:
Use Athena to analyze Amazon S3 server access logs | AWS re:Post (repost.aws)
Using Amazon S3 server access logs to identify requests - Amazon Simple Storage Service
