Comments on: Amazon S3: Reading Server Log Files

By: Vaibhav

Vaibhav — Mon, 03 Oct 2011 06:08:08 +0000

Thanks Brian for sharing.

By: Brian

Brian — Sat, 01 Oct 2011 17:28:04 +0000

I just wrote a python script to download and parse s3 bucket logs. You can run it directly to generate a customizable text table (see the source for how to customize it) or you can include it in scripts of your own and access the log data directly.

As a handle extra feature: It caches logs it has already download and doesn’t try to download them again so you can run it over and over and only fetch the data you haven’t seen yet.

You can get it here:
https://github.com/netguy204/bucketlogs

By: Vaibhav

Vaibhav — Mon, 16 Aug 2010 11:50:30 +0000

Well – did you look at the post that is linked in the beginning of this post? It tells you how you can enable logging on S3.

Also, there are many great services out there which are not very expensive at all which would do it for you.

By: Hina

Hina — Mon, 16 Aug 2010 10:23:38 +0000

I have a problem & really know nothing about S3 facility. I heard it first time.

I want to track bandwidth use for each bucket/user. At the end of the month, we need to generate a report for each bucket and report the bandwidth used.

So I need to know how to start the work or what I am supposed to start from. I am able to get a list of buckets but thats it. Except that I am able to do nothing.

How to download daily logs for a bucket is the place where I am lost. Pls help.