Greetings,
I have the following issue. I have a directory that I want to upload to
Amazon S3. The whole directory can be uploaded except a file that is >
80 MB
For all files, when I try to upload them, I get a progress number that
happily moves up until the file is uploaded. However, for this file the
upload gets inmediately stuck after uploading about 100 kb, and nothing
happens until I get a timeout on the socket some minutes later
I checked already permissions. I was able to upload a file with 30, 50
mb with no problems on the exact same location. I also trying fiddling
around with some of the settings regarding buffer size with no sucess.
The only think I know is that is is a time out error on the socket.
It seems to me, by looking at the code, that for some reason Amazon
stops receiving more bytes. I wonder if there is something in amazon,
that would interrupt an upload if a file is over a certain threshold?
I have here the result of executing the command using the debug and
verbose flag
This is my command
s3cmd sync -v -d -c .s3cfg --delete-removed /data/pshare/upload/
s3://pshare-dsadev-test/upload/
And here is the interesting extract
INFO: Summary: 3 local files to upload, 0 remote files to delete
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs':
'uid:5002/gname:www/uname:upload/gid:8/mode:33204/mtime:1375104235/atime:1375202191/ctime:1375104743'}
DEBUG: Content-Type set to 'binary/octet-stream'
DEBUG: String
'upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
encoded to
'upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
DEBUG: SignHeaders: 'PUT\n\nbinary/octet-stream\n\nx-amz-date:Tue, 30
Jul 2013 16:39:40
+0000\nx-amz-meta-s3cmd-attrs:uid:5002/gname:www/uname:upload/gid:8/mode:33204/mtime:1375104235/atime:1375202191/ctime:1375104743\n/pshare-dsadev-test/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
DEBUG: CreateRequest:
resource[uri]=/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
DEBUG: Unicodising
'/data/pshare/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
using UTF-8
DEBUG: SignHeaders: 'PUT\n\nbinary/octet-stream\n\nx-amz-date:Tue, 30
Jul 2013 16:39:40
+0000\nx-amz-meta-s3cmd-attrs:uid:5002/gname:www/uname:upload/gid:8/mode:33204/mtime:1375104235/atime:1375202191/ctime:1375104743\n/pshare-dsadev-test/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip'
/data/pshare/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
->
s3://pshare-dsadev-test/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
[1 of 3]
DEBUG: get_hostname(pshare-dsadev-test): pshare-dsadev-test.s3.amazonaws.com
DEBUG: format_uri():
http://pshare-dsadev-test.s3.amazonaws.com/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
98304 of 95955680 0% in 0s 222.97 kB/stimed out
98304 of 95955680 0% in 0s 222.97 kB/s failed
WARNING: Upload failed:
/upload/1619b266-8458-4b38-8a32-e720759a3dad/c4922966-3ae3-4d7b-b7bf-e3825351cd7a/rts.zip
(timed out)
WARNING: Retrying on lower speed (throttle=0.00)
WARNING: Waiting 3 sec...
Mario Alberto Ortegón-Cabrera
--
mario.orte...@dsa-ac.de
+49 (0)2408 9492 - 751
DSA - Daten- und Systemtechnik GmbH
Pascalstraße 28, 52076 Aachen, Germany
http://www.dsa.de
Sitz und Registergericht/Domicile and Court of Registry: Aachen
HRB-Nr./Commercial Register No.: 11960
Geschäftsführung/Board of Management:
Dr. Eckhard Schulz, Dr. Volker Gehring, Dr. Ansgar Schleicher
------------------------------------------------------------------------------
Get your SQL database under version control now!
Version control is standard for application code, but databases havent
caught up. So what steps can you take to put your SQL databases under
version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
