Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-16 Thread Michal Ludvig
Hi Joe, thanks for that. Looks like it crashed during checksumming and not during the actual upload, that's what I fixed in rc1. I'll have a look at this one too. Michal -- Beautiful is writing same markup. Internet

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-16 Thread Joe Auty
Here is the error message and trace info: Your "cron" job on myserver /usr/bin/s3cmd sync --rr --delete-removed --exclude 'maillogs/maillog' /nm/dorian/ s3://nmbackup/doriandata/ produced the following output: ! An unexpecte

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-16 Thread Michal Ludvig
On 11/16/2010 07:25 AM, Joe Auty wrote: Joe Auty wrote: Michal Ludvig wrote: Hi Joe, this problem has been fixed in s3cmd 1.0.0(-rc1). I hope you're not observing the crashes with this newest s3cmd, are you? See http://s3tools.org/s3cmd-100rc1-released for more info on what has been chang

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-15 Thread Joe Auty
Joe Auty wrote: Michal Ludvig wrote: Hi Joe, this problem has been fixed in s3cmd 1.0.0(-rc1). I hope you're not observing the crashes with this newest s3cmd, are you? See http://s3tools.org/s3cmd-100rc1

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-15 Thread Joe Auty
Michal Ludvig wrote: Hi Joe, this problem has been fixed in s3cmd 1.0.0(-rc1). I hope you're not observing the crashes with this newest s3cmd, are you? See http://s3tools.org/s3cmd-100rc1-released for more info on what has been cha

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-15 Thread Michal Ludvig
Hi Joe, this problem has been fixed in s3cmd 1.0.0(-rc1). I hope you're not observing the crashes with this newest s3cmd, are you? See http://s3tools.org/s3cmd-100rc1-released for more info on what has been changed/fixed. Michal On 11/15/2010

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-15 Thread Joe Auty
Joe Auty wrote: Thanks for this! Another problem I'm running into is that if a file is deleted on the source before it is sent, the sync fails and spews out an error message. This is a problem with data such as email where an email messag

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-15 Thread Joe Auty
Thanks for this! Another problem I'm running into is that if a file is deleted on the source before it is sent, the sync fails and spews out an error message. This is a problem with data such as email where an email message might be deleted in real time, or perhaps

Re: [S3tools-general] Techniques for skipping checksum match on sync

2010-11-13 Thread Michal Ludvig
Hi Joe, currently it isn't possible to skip checksums comparison. If all your log files are in the same directory or have some common name pattern, e.g. *.log, you can exclude them with --exclude \*.log or some other variant of --exclude-from/--rexclude/--rexc

[S3tools-general] Techniques for skipping checksum match on sync

2010-11-12 Thread Joe Auty
Hello, Is there a way to get s3cmd to not fuss over matching checksums on live log files and files that are constantly being written to? I realize that I can exclude them, but there are many of them that are dynamically created, I'd rather not list them individually