According to Amazon http://docs.amazonwebservices.com/AmazonS3/2006-03-01/gsg/
"Each object can hold up to 5 GB of data." You may have got lucky with the 5. 2 file and it finally failed on the 5.5 Thanks, Fr. Robert Bower Batushka's Web Hosting & Design frrob...@batushkas.com 574.850.7982 http://www.batushkas.com http://www.batushkas.net Alex Efros wrote: > Hi! > > I've large file (~5.5 GB - /home dir backup), and tried several times to > upload it to S3 using s3cmd 0.9.8.4. Each time it took several hours and > finally failed. I've just tried to upgrade to 0.9.9, it also failed but > with another error message (see at bottom of this message). > > All other ~1600 files uploaded without troubles using 'sync' command, > while this file failed both using 'sync' and 'put' commands. > Also I should note this file is largest from all ~1600 files - next > large file size is ~5.2GB and it uploaded normally. > > # time md5sum home.full.20090302_031507.tar.gpg > d3722332b583f2d2e5f53413ee14fe57 home.full.20090302_031507.tar.gpg > > real 2m10.812s > user 0m25.490s > sys 0m8.570s > > # ls -l home.full.20090302_031507.tar.gpg > -rw-r--r-- 1 backup backup 5586801747 2009-03-02 03:57 > home.full.20090302_031507.tar.gpg > > # time s3cmd put home.full.20090302_031507.tar.gpg s3://backup.ourcompany > WARNING: MD5 Sums don't match! > WARNING: MD5 Sums don't match! > WARNING: MD5 Sums don't match! > WARNING: MD5 Sums don't match! > ERROR: Upload of 'home.full.20090302_031507.tar.gpg' failed too many times. > Skipping that file. > > real 303m38.849s > user 7m56.840s > sys 1m8.640s > > # time /etc/cron.daily/backup_to_s3 > Compiling list of local files... > Retrieving list of remote files... > Found 1663 local files, 1632 remote files > Verifying checksums... > Summary: 8 local files to upload, 0 remote files to delete > WARNING: MD5 Sums don't match! > WARNING: MD5 Sums don't match! > WARNING: MD5 Sums don't match! > WARNING: MD5 Sums don't match! > ERROR: home.full.20090302_031507.tar.gpg: upload failed too many times. > Skipping that file. > File 'mysql.incr.20090304_030008.tar.gpg' stored as > s3://backup.ourcompany/mysql.incr.20090304_030008.tar.gpg (80084541 bytes in > 171.0 seconds, 457.26 kB/s) [2 of 8] > File 'home.incr.20090304_030010.tar.gpg' stored as > s3://backup.ourcompany/home.incr.20090304_030010.tar.gpg (14791469 bytes in > 1.9 seconds, 7.47 MB/s) [3 of 8] > File 'mysql.incr.20090304_030010.tar.gpg' stored as > s3://backup.ourcompany/mysql.incr.20090304_030010.tar.gpg (1554 bytes in 0.0 > seconds, 9.43 MB/s) [4 of 8] > File 'svn.incr.20090304_030010.tar.gpg' stored as > s3://backup.ourcompany/svn.incr.20090304_030010.tar.gpg (3553 bytes in 0.0 > seconds, 22.17 MB/s) [5 of 8] > File 'config.incr.20090304_071720.tar.gpg' stored as > s3://backup.ourcompany/config.incr.20090304_071720.tar.gpg (32459 bytes in > 0.3 seconds, 125.75 kB/s) [6 of 8] > File 'home.incr.20090304_071720.tar.gpg' stored as > s3://backup.ourcompany/home.incr.20090304_071720.tar.gpg (2705201 bytes in > 1.0 seconds, 2.60 MB/s) [7 of 8] > File 'mysql.incr.20090304_071720.tar.gpg' stored as > s3://backup.ourcompany/mysql.incr.20090304_071720.tar.gpg (979 bytes in 0.0 > seconds, 6.06 MB/s) [8 of 8] > Done. Uploaded 97619756 bytes in 34733.5 seconds, 2.74 kB/s > > real 599m43.248s > user 10m35.850s > sys 2m7.890s > > > 0.9.9 fail in different way: > > # time s3cmd put home.full.20090302_031507.tar.gpg s3://backup.ourcompany > home.full.20090302_031507.tar.gpg -> s3://backup.ourcompany/ [1 of 1] > 5586801747 of 5586801747 100% in 11194s 487.39 kB/s done > ERROR: S3 error: 400 (MalformedXML): The XML you provided was not well-formed > or did not validate against our published schema > > real 186m34.334s > user 2m27.220s > sys 0m28.430s > > > ------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H _______________________________________________ S3tools-general mailing list S3tools-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/s3tools-general
