Hi!
I've large file (~5.5 GB - /home dir backup), and tried several times to
upload it to S3 using s3cmd 0.9.8.4. Each time it took several hours and
finally failed. I've just tried to upgrade to 0.9.9, it also failed but
with another error message (see at bottom of this message).
All other ~1600 files uploaded without troubles using 'sync' command,
while this file failed both using 'sync' and 'put' commands.
Also I should note this file is largest from all ~1600 files - next
large file size is ~5.2GB and it uploaded normally.
# time md5sum home.full.20090302_031507.tar.gpg
d3722332b583f2d2e5f53413ee14fe57 home.full.20090302_031507.tar.gpg
real 2m10.812s
user 0m25.490s
sys 0m8.570s
# ls -l home.full.20090302_031507.tar.gpg
-rw-r--r-- 1 backup backup 5586801747 2009-03-02 03:57
home.full.20090302_031507.tar.gpg
# time s3cmd put home.full.20090302_031507.tar.gpg s3://backup.ourcompany
WARNING: MD5 Sums don't match!
WARNING: MD5 Sums don't match!
WARNING: MD5 Sums don't match!
WARNING: MD5 Sums don't match!
ERROR: Upload of 'home.full.20090302_031507.tar.gpg' failed too many times.
Skipping that file.
real 303m38.849s
user 7m56.840s
sys 1m8.640s
# time /etc/cron.daily/backup_to_s3
Compiling list of local files...
Retrieving list of remote files...
Found 1663 local files, 1632 remote files
Verifying checksums...
Summary: 8 local files to upload, 0 remote files to delete
WARNING: MD5 Sums don't match!
WARNING: MD5 Sums don't match!
WARNING: MD5 Sums don't match!
WARNING: MD5 Sums don't match!
ERROR: home.full.20090302_031507.tar.gpg: upload failed too many times.
Skipping that file.
File 'mysql.incr.20090304_030008.tar.gpg' stored as
s3://backup.ourcompany/mysql.incr.20090304_030008.tar.gpg (80084541 bytes in
171.0 seconds, 457.26 kB/s) [2 of 8]
File 'home.incr.20090304_030010.tar.gpg' stored as
s3://backup.ourcompany/home.incr.20090304_030010.tar.gpg (14791469 bytes in 1.9
seconds, 7.47 MB/s) [3 of 8]
File 'mysql.incr.20090304_030010.tar.gpg' stored as
s3://backup.ourcompany/mysql.incr.20090304_030010.tar.gpg (1554 bytes in 0.0
seconds, 9.43 MB/s) [4 of 8]
File 'svn.incr.20090304_030010.tar.gpg' stored as
s3://backup.ourcompany/svn.incr.20090304_030010.tar.gpg (3553 bytes in 0.0
seconds, 22.17 MB/s) [5 of 8]
File 'config.incr.20090304_071720.tar.gpg' stored as
s3://backup.ourcompany/config.incr.20090304_071720.tar.gpg (32459 bytes in 0.3
seconds, 125.75 kB/s) [6 of 8]
File 'home.incr.20090304_071720.tar.gpg' stored as
s3://backup.ourcompany/home.incr.20090304_071720.tar.gpg (2705201 bytes in 1.0
seconds, 2.60 MB/s) [7 of 8]
File 'mysql.incr.20090304_071720.tar.gpg' stored as
s3://backup.ourcompany/mysql.incr.20090304_071720.tar.gpg (979 bytes in 0.0
seconds, 6.06 MB/s) [8 of 8]
Done. Uploaded 97619756 bytes in 34733.5 seconds, 2.74 kB/s
real 599m43.248s
user 10m35.850s
sys 2m7.890s
0.9.9 fail in different way:
# time s3cmd put home.full.20090302_031507.tar.gpg s3://backup.ourcompany
home.full.20090302_031507.tar.gpg -> s3://backup.ourcompany/ [1 of 1]
5586801747 of 5586801747 100% in 11194s 487.39 kB/s done
ERROR: S3 error: 400 (MalformedXML): The XML you provided was not well-formed
or did not validate against our published schema
real 186m34.334s
user 2m27.220s
sys 0m28.430s
--
WBR, Alex.
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
S3tools-general mailing list
S3tools-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/s3tools-general
