Re: --log-file=File output

2014-03-11 Thread Robert DuToit
On Mar 12, 2014, at 12:21 AM, Kevin Korb wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > - -v would be %n%L > You probably don't want --progress in a log file. Ah yes - I completely missed the formatting options - a long scroll down the man page! I’ll experiment with those. Than

Re: --log-file=File output

2014-03-11 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - -v would be %n%L You probably don't want --progress in a log file. On 03/12/2014 12:20 AM, Robert DuToit wrote: > Thanks Kevin, > > On Mar 12, 2014, at 12:10 AM, Kevin Korb > wrote: > > See --log-file-format Most of the info for it is in man > r

Re: --log-file=File output

2014-03-11 Thread Robert DuToit
Thanks Kevin, On Mar 12, 2014, at 12:10 AM, Kevin Korb wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > See --log-file-format Most of the info for it is in man rsyncd.conf > since it is mostly used by servers. I looked in there - a bit daunting. Ideally I would just want the stdou

Re: --log-file=File output

2014-03-11 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 See --log-file-format Most of the info for it is in man rsyncd.conf since it is mostly used by servers. On 03/12/2014 12:02 AM, Robert DuToit wrote: > Hi All, I have a situations where I need output to a file and > normally would just append “&>outpu

--log-file=File output

2014-03-11 Thread Robert DuToit
Hi All, I have a situations where I need output to a file and normally would just append “&>output.log” to the rsync command line but can’t do that in this situation and need to run rsync with just the args via nstask. I tried the internal --log-file=File option which works except it outputs ve

Re: Caching {filePath,mtime64,checksum} values to speed up execution-time

2014-03-11 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 OK, in that case you should try using --ignore-times instead - --checksum. With --ignore-times rsync will redo the delta transfer of all files. This is usually faster than --checksum and won't cause much additional data transfer. Unless --whole-file

Re: Caching {filePath,mtime64,checksum} values to speed up execution-time

2014-03-11 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --checksum should not be used during normal rsync operations. It is for special cases only. Rsync can still have a lot of overhead getting the timestamps via stat() but that can't really be helped. I don't really understand how file mtimes would b

Re: Caching {filePath,mtime64,checksum} values to speed up execution-time

2014-03-11 Thread Doug Robinson
Ah, forgot to mention that we can't preserve time-stamps and matching byte counts are false negatives so we really need to use file checksums as comparisons. Thanks again. Doug -- WANdisco // *Non-Stop Data* t. 925-396-1125 e. doug.robin...@wandisco.com -- Join us in New York and San Franci

Caching {filePath,mtime64,checksum} values to speed up execution-time

2014-03-11 Thread Doug Robinson
Folks: When using rsync to copy huge amounts of data I've found that a significant amount of time is spent computing the checksums. Sometimes hours, ... sometimes days - it depends on the total amount of data checked! And after that sometimes it's only a few files that need to be updated. I've

Re: silent data corruption with rsync

2014-03-11 Thread Leen Besselink
On Tue, Mar 11, 2014 at 11:52:51AM -0500, Karl O. Pinc wrote: > On 03/11/2014 11:02:28 AM, Sig Pam wrote: > > Hi everbody! > > > > I'm currently working in a project which has to copy huge amounts of > > data from one storage to another. For a reason I cannot validate any > > longer, there is a ro

Re: silent data corruption with rsync

2014-03-11 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I have actually witnessed rsync silently corrupting data. But it wasn't rsync's fault. I had a bad RAM DIMM that was corrupting the part of RAM being used as the disk cache. Now I always get ECC RAM. On 03/11/2014 12:52 PM, Karl O. Pinc wrote: > On

Re: silent data corruption with rsync

2014-03-11 Thread Karl O. Pinc
On 03/11/2014 11:02:28 AM, Sig Pam wrote: > Hi everbody! > > I'm currently working in a project which has to copy huge amounts of > data from one storage to another. For a reason I cannot validate any > longer, there is a roumor that "rsync may silently corrupt data". > Personally, I don't believe

[Bug 10495] New: "skipping directory foo" (does not transfer directories by default)

2014-03-11 Thread samba-bugs
https://bugzilla.samba.org/show_bug.cgi?id=10495 Summary: "skipping directory foo" (does not transfer directories by default) Product: rsync Version: 3.1.0 Platform: All OS/Version: All Status: NEW Severit

silent data corruption with rsync

2014-03-11 Thread Sig Pam
Hi everbody! I'm currently working in a project which has to copy huge amounts of data from one storage to another. For a reason I cannot validate any longer, there is a roumor that "rsync may silently corrupt data". Personally, I don't believe that. "They" explain it this way: "rsync does an i