On 08/16/2010 10:18 PM, Hina Tabassum wrote:
Hi,
Read about your work at www.sourgeforge.net
<http://www.sourgeforge.net> regarding Amazon S3 logs. Great work I
must say.
I have a problem & really know nothing about S3 facility. I heard it
first time.
I want to track bandwidth use for each bucket/user. At the end of the
month, we need to generate a report for each bucket and report the
bandwidth used.
So I need to know how to start the work or what I am supposed to start
from. I am able to get a list of buckets but thats it. Except that I
am able to do nothing.
How to download daily logs for a bucket is the place where I am lost.
Pls help.
---
Hina
Hi Hina,
there are two sorts of access logs - one is the log of the S3 storage
itself (generated per-bucket) and one is usage log for CloudFront,
obviously only available if you deliver content from S3 over CloudFront,
i.e. when you use S3 as a storage for web content as oppose to, say, for
backup purposes.
The former (S3 logs) are enabled with:
s3cmd accesslog
--access-logging-target-prefix=s3://your-log-bucket/target-bucket/s3/
s3://target-bucket
That will enable access logging for 's3://target-bucket' and store the
logs to 's3://your-log-bucket/target-bucket/s3/' (obviously you should
create s3://your-log-bucket first).
Similarly for CloudFront logging can be enabled with:
s3cmd cfmodify
--access-logging-target-prefix=s3://your-log-bucket/target-bucket/cf/
cf://THEDISTID
At the end of the month simply download all the logs from
s3://your-log-bucket/ and process to get whatever reports you need.
It is recommended to store the S3 logs in a separate bucket, not in the
one being monitored. Otherwise writing the logfiles into the same bucket
generates access requests and is logged too, which creates kind of a loop.
Hope that helps
Michal
------------------------------------------------------------------------------
This SF.net email is sponsored by
Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
