All -
I need a little help with the LOCAL_FILE option. I'm getting objects
from this public bucket -
# s3cmd ls --recursive s3://datasets.elasticmapreduce/ngrams/books/ |
awk '{print $4}'
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/1gram/data
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/2gram/data
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/3gram/data
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/4gram/data
I want the LOCAL_PATH to be the object path without the object name, ie
./ngrams/books/20090715/chi-sim-all/1gram/data
./ngrams/books/20090715/chi-sim-all/2gram/data
./ngrams/books/20090715/chi-sim-all/3gram/data
./ngrams/books/20090715/chi-sim-all/4gram/data
ie
s3cmd get
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/1gram/data
./ngrams/books/20090715/chi-sim-all/1gram/data
s3cmd get
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/2gram/data
./ngrams/books/20090715/chi-sim-all/2gram/data
s3cmd get
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/3gram/data
./ngrams/books/20090715/chi-sim-all/3gram/data
s3cmd get
s3://datasets.elasticmapreduce/ngrams/books/20090715/chi-sim-all/4gram/data
./ngrams/books/20090715/chi-sim-all/4gram/data
I'm distributing the gets to different processes so I can't use
s3cmd's recursive get. Is there a way to do this?
Thanks,
Craig
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
