Hi,
I finally spent some time trying to work out why the --add-headers option
didn't seem to work to add the "x-amz-request-payer: requester" header required
to accept charges for access to a requester pays bucket
(http://docs.amazonwebservices.com/AmazonDevPay/latest/DevPayDeveloperGuide/index.html?S3RequesterPays.html).
It seems that any headers specified using the --add-headers option were never
actually used (at least in 'ls' and 'get' and it seems many other commands
too). The following patch works against s3cmd-1.0.0:
--- s3cmd-1.0.0-clean/S3/S3.py 2010-10-24 08:17:22.000000000 -0400
+++ s3cmd-1.0.0-sw/S3/S3.py 2011-03-30 16:44:06.000000000 -0400
@@ -32,6 +32,9 @@
def __init__(self, s3, method_string, resource, headers, params = {}):
self.s3 = s3
self.headers = SortedDict(headers or {}, ignore_case = True)
+ # Add in any extra headers from s3 config object
+ if self.s3.config.extra_headers:
+ self.headers.update(self.s3.config.extra_headers)
self.resource = resource
self.method_string = method_string
self.params = params
and when saved at 'patchfile' you can apply with the following in the unpacked
s3cmd directory:
simeon@homebox s3cmd-1.0.0>cat patchfile | patch -p1
patching file S3/S3.py
With the patch one can then (NOTE YOU'LL BE PAYING A FEW CENTS FOR THIS but the
files do exist and are in requester pays buckets, see:
http://arxiv.org/help/bulk_data_s3):
simeon@homebox s3cmd-1.0.0-sw>./s3cmd
--add-header="x-amz-request-payer:requester" ls
s3://arxiv/pdf/arXiv_pdf_manifest.xml
2011-03-30 20:19 249105 s3://arxiv/pdf/arXiv_pdf_manifest.xml
simeon@homebox s3cmd-1.0.0-sw>./s3cmd
--add-header="x-amz-request-payer:requester" info
s3://arxiv/pdf/arXiv_pdf_manifest.xml
s3://arxiv/pdf/arXiv_pdf_manifest.xml (object):
File size: 249105
Last mod: Wed, 30 Mar 2011 20:19:30 GMT
MIME type:
MD5 sum: 9d1174b740c9eeeebeab6d7ba0773008
ERROR: Access to bucket 'arxiv' was denied
(I assume the error message is because the ACL status is still denied)
simeon@homebox s3cmd-1.0.0-sw>./s3cmd
--add-header="x-amz-request-payer:requester" get
s3://arxiv/pdf/arXiv_pdf_manifest.xml
s3://arxiv/pdf/arXiv_pdf_manifest.xml -> ./arXiv_pdf_manifest.xml [1 of 1]
249105 of 249105 100% in 0s 389.78 kB/s done
whereas without the header (or without the patch) you'll get an denial:
simeon@homebox s3cmd-1.0.0-sw>./s3cmd ls s3://arxiv/pdf/arXiv_pdf_manifest.xml
ERROR: Access to bucket 'arxiv' was denied
Cheers,
Simeon
------------------------------------------------------------------------------
Forrester Wave Report - Recovery time is now measured in hours and minutes
not days. Key insights are discussed in the 2010 Forrester Wave Report as
part of an in-depth evaluation of disaster recovery service providers.
Forrester found the best-in-class provider in terms of services and vision.
Read this report now! http://p.sf.net/sfu/ibm-webcastpromo
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
