I forgot to paste the line I used for the real testing but it's here: navi:/mnt# rsync --verbose --sparse --append-verify --fuzzy --progress --stats --recursive --times --delete /mnt/miracle/ /mnt/storage/
-Mikko 2009/12/5 kordex - <kor...@gmail.com>: > Hello, > > Yes I ran exclusive testing: > > Checking source (/mnt/miracle): > > navi:/mnt# fsck.jfs -f /dev/sda1 > fsck.jfs version 1.1.12, 24-Aug-2007 > processing started: 12/5/2009 16.11.38 > The current device is: /dev/sda1 > Block size in bytes: 4096 > Filesystem size in blocks: 244190000 > **Phase 0 - Replay Journal Log > **Phase 1 - Check Blocks, Files/Directories, and Directory Entries > **Phase 2 - Count links > **Phase 3 - Duplicate Block Rescan and Directory Connectedness > **Phase 4 - Report Problems > **Phase 5 - Check Connectivity > **Phase 6 - Perform Approved Corrections > **Phase 7 - Rebuild File/Directory Allocation Maps > **Phase 8 - Rebuild Disk Allocation Maps > 976760000 kilobytes total disk space. > 13986 kilobytes in 2316 directories. > 766065198 kilobytes in 29268 user files. > 1532 kilobytes in extended attributes > 212356 kilobytes reserved for system use. > 210494900 kilobytes are available for use. > Filesystem is clean. > navi:/mnt# > > Wiping the target (/mnt/storage) filesystem: > > navi:/mnt# mkfs.jfs /dev/mapper/void-helix > mkfs.jfs version 1.1.12, 24-Aug-2007 > Warning! All data on device /dev/mapper/void-helix will be lost! > > Continue? (Y/N) y > \ > > Format completed successfully. > > 976748544 kilobytes total disk space. > > navi:/mnt# fsck.jfs -f /dev/mapper/void-helix > fsck.jfs version 1.1.12, 24-Aug-2007 > processing started: 12/5/2009 16.25.53 > The current device is: /dev/mapper/void-helix > Block size in bytes: 4096 > Filesystem size in blocks: 244187136 > **Phase 0 - Replay Journal Log > **Phase 1 - Check Blocks, Files/Directories, and Directory Entries > **Phase 2 - Count links > **Phase 3 - Duplicate Block Rescan and Directory Connectedness > **Phase 4 - Report Problems > **Phase 5 - Check Connectivity > **Phase 6 - Perform Approved Corrections > **Phase 7 - Rebuild File/Directory Allocation Maps > **Phase 8 - Rebuild Disk Allocation Maps > 976748544 kilobytes total disk space. > 0 kilobytes in 1 directories. > 0 kilobytes in 0 user files. > 0 kilobytes in extended attributes > 182256 kilobytes reserved for system use. > 976566288 kilobytes are available for use. > Filesystem is clean. > > navi:/mnt# mount /dev/sda1 /mnt/miracle > navi:/mnt# mount /dev/mapper/void-helix /mnt/storage/ > navi:/mnt# mount > /dev/sda1 on /mnt/miracle type jfs (rw) > /dev/mapper/void-helix on /mnt/storage type jfs (rw) > > navi:/mnt# cp "/mnt/miracle/./waffcd/music/Moving Past The > Boundaries/Negative Format - 07 Momentum.flac" /mnt/storage/ > > navi:/mnt# cksum "/mnt/miracle/./waffcd/music/Moving Past The > Boundaries/Negative Format - 07 Momentum.flac" > 2863733908 34132620 /mnt/miracle/./waffcd/music/Moving Past The > Boundaries/Negative Format - 07 Momentum.flac > navi:/mnt# cksum /mnt/storage/Negative\ Format\ -\ 07\ Momentum.flac > 2863733908 34132620 /mnt/storage/Negative Format - 07 Momentum.flac > > navi:/mnt# mkdir /mnt/storage/bar > > navi:/mnt# rsync --verbose --sparse --append-verify --fuzzy --progress > --stats --recursive --times --delete > "/mnt/miracle/./waffcd/music/Moving Past The Boundaries/Negative > Format - 07 Momentum.flac" /mnt/storage/bar > sending incremental file list > Negative Format - 07 Momentum.flac > 34132620 100% 23.79MB/s 0:00:01 (xfer#1, to-check=0/1) > > Number of files: 1 > Number of files transferred: 1 > Total file size: 34132620 bytes > Total transferred file size: 34132620 bytes > Literal data: 34132620 bytes > Matched data: 0 bytes > File list size: 49 > File list generation time: 0.001 seconds > File list transfer time: 0.000 seconds > Total bytes sent: 34136885 > Total bytes received: 31 > > sent 34136885 bytes received 31 bytes 13654766.40 bytes/sec > total size is 34132620 speedup is 1.00 > > navi:/mnt# cksum /mnt/storage/bar/Negative\ Format\ -\ 07\ Momentum.flac > 2863733908 34132620 /mnt/storage/bar/Negative Format - 07 Momentum.flac > > Works fine, eh? Well let's see how the real thing goes: > > Well I don't want to cksum the whole thing again as I have done it two > times already, and my slow controller (SiL 3114) and G4 powerpc cpu > (7447) are bit slowing doing that, so I assume partitial check is ok: > > Hitting ^C. Stops here: > > music/1998 - Cathedron/Sephiroth - 01 - Wolftribes.flac > 32768 0% 78.82kB/s 0:06:10 ^C > rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at > rsync.c(541) [sender=3.0.3] > rsync: writefd_unbuffered failed to write 97 bytes [generator]: Broken pipe > (32) > > Checksuming the files I copied: > > navi:/mnt/storage# find ./ -type f > ../storage.txt > > navi:/mnt/storage# foreach i ( "`cat ../storage.txt`" ) > foreach? cksum "$i" >> ../storage.cksum.txt > foreach? end > > Comparing checksums (the one I canceled might fail ofcourse): > > navi:/mnt/storage# cd .. > > navi:/mnt# wc -l storage.txt > 223 storage.txt > > navi:/mnt# head -n 223 miracle.cksums.txt > bar > > navi:/mnt# diff storage.cksum.txt bar | wc -l > 182 > > ... > > 213,215c213,215 > < 2249859435 41520924 ./music/1992 - How to Destroy Angels/04 - > Tectonic Plates.flac > < 1562355406 26647298 ./music/1992 - How to Destroy Angels/05 - Dismal > Orb.flac > < 4252661771 56535930 ./music/1992 - How to Destroy Angels/06 - How to > Destroy Angels II.flac > --- >> 1965594369 41520924 ./music/1992 - How to Destroy Angels/04 - Tectonic >> Plates.flac >> 3734256630 26647298 ./music/1992 - How to Destroy Angels/05 - Dismal Orb.flac >> 667032915 56535930 ./music/1992 - How to Destroy Angels/06 - How to Destroy >> Angels II.flac > > ... > > Few last lines from diff. > > Let's try the cp: > > navi:/mnt# cp "/mnt/miracle/./music/1992 - How to Destroy Angels/06 - > How to Destroy Angels II.flac" /mnt/storage/ > > navi:/mnt# cksum "/mnt/miracle/./music/1992 - How to Destroy Angels/06 > - How to Destroy Angels II.flac" > 667032915 56535930 /mnt/miracle/./music/1992 - How to Destroy > Angels/06 - How to Destroy Angels II.flac > > navi:/mnt# cksum /mnt/storage/06\ -\ How\ to\ Destroy\ Angels\ II.flac > 667032915 56535930 /mnt/storage/06 - How to Destroy Angels II.flac > > -Mikko > > > 2009/12/5 Matt McCutchen <m...@mattmccutchen.net>: >> On Fri, 2009-12-04 at 18:21 +0200, kordex - wrote: >>> I just rsync'd my 1TB partitions formated with jfs filesystem >>> containing ~720GB data. I saw -3G difference between source and >>> destination which made me suspect possible dataloss. Also I had >>> experienced loss of integrity on my previous copies but I did not find >>> the cause until now. >>> >>> As command line I used following: >>> sh-3.2# rsync --verbose --sparse --append-verify --fuzzy --progress >>> --stats --recursive --times --delete /mnt/miracle /mnt/storage >> >>> As we all know the second field of output from cksum is the size: so >>> the size matches but the content does not. >> >> OK... can you reproduce the problem by copying some of the source files >> to a different place on the destination filesystem, or on a different >> filesystem? What if you use cp instead of rsync? Have you run fsck on >> the source and destination filesystems? >> >>> This maybe due fuzzy option but append-verify should (?) confirm that >>> the already existing data matches the appended one (?). >> >> Yes, when a file transfer is performed. If the destination already has >> a file that passes the quick check with the source file, no file >> transfer is performed. In that case, consider --ignore-times to disable >> the quick check. >> >> -- >> Matt >> >> > -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
