I learned that this just happens from time to time. The suggested solution was to break the file into pieces so that failures don't set you back hours. Now I break my 4GB file into 0.5 GB chunks. I use a script to split the file and send the chunks, retrying on failure. It's been flawless for weeks now.
On Thu, 2010-04-01 at 09:27 -0400, Jeremy Haile wrote: > I'm using s3cmd 0.9.9 and trying to do a put to S3 of several 1GB files, > but I continually get connection reset by peer errors and I'm not sure > what to do. Here are my logs. Any ideas? > > INFO: Compiling list of local files... > INFO: Applying --exclude/--include > INFO: Summary: 18 local files to upload > INFO: Sending file '/var/backups/myfile-s3-M-W-F_000', please wait... > INFO: Sending file '/var/backups/myfile-s3-M-W-F_001', please > wait...IINFO: Sending file '/var/backups/myfile-s3-M-W-F_002', please > wait... > INFO: Sending file '/var/backups/myfile-s3-M-W-F_003', please wait... > WARNING: Upload failed: /backups/M-W-F/myfile-s3-M-W-F_003 ([Errno 104] > Connection reset by peer) > WARNING: Retrying on lower speed (throttle=0.01) > WARNING: Waiting 3 sec... > INFO: Sending file '/var/backups/myfile-s3-M-W-F_003', please wait... > WARNING: Upload failed: /backups/M-W-F/myfile-s3-M-W-F_003 ([Errno 104] > Connection reset by peer) > WARNING: Retrying on lower speed (throttle=0.05) > WARNING: Waiting 6 sec... > INFO: Sending file '/var/backups/myfile-s3-M-W-F_003', please wait... > WARNING: Upload failed: /backups/M-W-F/myfile-s3-M-W-F_003 ([Errno 104] > Connection reset by peer) > WARNING: Retrying on lower speed (throttle=0.25) > WARNING: Waiting 9 sec...WARNING: Upload failed: > /backups/M-W-F/myfile-s3-M-W-F_002 ([Errno 104] Connection reset by > WARNING: Upload failed: /backups/M-W-F/myfile-s3-M-W-F_003 ([INFO: > Sending file '/var/backups/myfile-s3-M-W-F_002', please wait... > WARNING: Upload failed: /backups/M-W-F/myfile-s3-M-W-F_002 ([Errno 104] > Connection reset by WARNING: Upload failed: > /backups/M-W-F/myfile-s3-M-W-F_003 ([Errno 104] Connection reset by peer) > WARNING: Retrying on lower speed (throttle=6.25) > WARNING: Waiting 15 sec... > INFO: Sending file '/var/backups/myfile-s3-M-W-F_003', please wait... > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > S3tools-general mailing list > S3tools-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/s3tools-general -- Rob Scala Buyers Edge, LLC 351 North Frontage Road, Suite A210 New London, CT 06320 Phone: 860-701-0538 x27 Fax: 860-701-0576 ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ S3tools-general mailing list S3tools-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/s3tools-general
