I've not been to do this and it's not for shortage of large files to
throw around!!!
Q: How much data can I store?
The total volume of data and number of objects you can store are
unlimited. Individual Amazon S3 objects can range in size from 1 byte to
5 terabytes. The largest object that can be uploaded in a single PUT is
5 gigabytes. For objects larger than 100 megabytes, customers should
consider using the Multipart Upload
<http://docs.amazonwebservices.com/AmazonS3/latest/dev/UploadingObjects.
html> capability.
Maybe there's a trigger for the Multi-part upload facilitiy.
Matt Rogers
Location: UK
UK Cell/SMS: +447958002382
Indian Cell/SMS: +919884690539
Email: m...@aroxo.com <mailto:m...@aroxo.com>
Skype: mattrogers1975
From: Something Something [mailto:mailinglist...@gmail.com]
Sent: 13 July 2012 15:18
To: s3tools-general@lists.sourceforge.net
Subject: Re: [S3tools-general] Transferring files over 5G to AWS
Hmm... but 'Rob' said... "As of the 1.0 version of s3cmd, you can upload
file larger than 5G. "
Has anyone transferred a file over 5G using s3cmd (version 1.0 & above)?
Please let me know. Thanks.
On Fri, Jul 13, 2012 at 12:41 AM, Matt Rogers <m...@aroxo.com> wrote:
5Gb is the maximum file size for an upload, you'll need to split files
over 5Gb before uploading.
Matt Rogers
Location: UK
UK Cell/SMS: +447958002382 <tel:%2B447958002382>
Indian Cell/SMS: +919884690539 <tel:%2B919884690539>
Email: m...@aroxo.com <mailto:m...@aroxo.com>
Skype: mattrogers1975
From: Something Something [mailto:mailinglist...@gmail.com]
Sent: 13 July 2012 07:46
To: s3tools-general@lists.sourceforge.net
Subject: Re: [S3tools-general] Transferring files over 5G to AWS
It's not working for me for large files :( When I try with a small
file it works, but when I try with a file over 5G, I get this:
------------------------------------------------------------------------
-----------------------------------------------
WARNING: Waiting 15 sec...
./myfileBig.txt -> s3://<bucket_name>/test/myfileBig.txt [1 of 1]
8192 of 21158596877 0% in 1s 6.31 kB/s failed
ERROR: Upload of './myfileBig.txt' failed too many times. Skipping that
file.
------------------------------------------------------------------------
-----------------------------------------------
I am using the following version of s3cmd:
s3cmd version 1.0.0
Please help. Thanks.
On Wed, Jul 11, 2012 at 5:04 PM, Dunie, Rob <robert.du...@pega.com>
wrote:
As of the 1.0 version of s3cmd, you can upload file larger than 5G.
Rob
________________________________
From: Something Something [mailinglist...@gmail.com]
Sent: Wednesday, July 11, 2012 6:58 PM
To: s3tools-general@lists.sourceforge.net
Subject: [S3tools-general] Transferring files over 5G to AWS
Hello,
Is it possible to transfer a file bigger than 5G using s3cmd? Somewhere
I read that I can use command line arguments such as these:
x-amz-storage-class:REDUCED_REDUNDANCY s3_multipart_threshold:1073741824
s3_multipart_min_part_size:104857600"
Is that true? Where can I find more info about this? Tried s3cmd put
--help but that didn't help.
Please help. Thanks.
------------------------------------------------------------------------
------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
------------------------------------------------------------------------
------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
