-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/08/13 22:16, xiaolong mou wrote: > Thanks. I suspected hardware issue as well. However I did a local > rsync test at the USB drive side and NAS side with the same files. > If there was a local hardware problem (RAM or USB hard drive driver > etc) it should show up in this test, but everything was fine!
I would think this would hit a bad RAM problem but it wouldn't if the bad RAM was in the other system and there is no guarantee either way. > Could it be network driver somehow corrupting data? Although it is > hard to believe. I have used the NAS attached to the router for > years, never noticed any network issue. Also the rsync checksum > after transfer should catch this case, right? I suppose it is possible. Rsync only checks that the file it assembles in memory+cache matches what the other system was sending. It assumes that the write to disk will also match. > I did the original test (transferring data from NAS to router back > to NAS) using rsync over ssh. This time I did see several warning > messages, saying "xxxx failed verification -- update discarded > (will try again)". The diff was fine, apparently the retry worked. This could still be bad RAM. > I am fairly puzzled. Hopefully someone can help shed some light to > track it down. If it were me, based on my previous experience, I would shut down both systems and run memtest86+ or "Windows Memory Diagnostics" on both systems. Make sure to enable the extended tests. Let them run overnight and see if they identify a problem. > > On Fri, Mar 8, 2013 at 9:09 PM, Kevin Korb <k...@sanitarium.net> > wrote: I have seen this behavior before. Twice. > > Both times the cause was bad RAM on the target system. The bad > RAM was corrupting the files within the disk write cache so that > rsync believed it was writing the correct data but the disk was not > getting the correct data. Ever since that happened to me I have > insisted on ECC RAM and have never had such a problem again. > > It is also possible that a disk or disk controller could be at > fault. > > Rsync does do an in memory checksum before it renames the file > into place but it does not re-read the file back from the disk > (which in reality would also require a dump of the cache). > > On 03/08/13 20:55, xiaolong mou wrote: >>>> Hi, >>>> >>>> I am backing up about 500G of data from a linux-based NAS to >>>> a USB hard drive attached to a router (rt-n16 with tomatousb >>>> firmware). Both NAS and the router have rsync-3.0.9. The >>>> router is running rsync in the daemon mode. To test the set >>>> up, I rsync'd the files to the empty USB hard drive, then >>>> back to the NAS in a new temp dir. Afterwards, I did a local >>>> "diff -r" on the NAS. To my surprise, a few files were >>>> corrupted (only 3 out of 17K files). "cmp -l" shows single >>>> byte difference between the original and rsync'd files. >>>> However, I didn't see any error message on the NAS side or in >>>> the rsyncd logs. How is that possible? Doesn't rsync always >>>> do a checksum verification after copying the files? I have >>>> been using rsync for years for local backup. The feeling of >>>> silent file corruption is scary. Could someone point me to >>>> the right direction? I really want to get to the bottom of >>>> this. Much appreciated. >>>> >>>> Regards, Xiaolong >>>> > >> -- Please use reply-all for most replies to avoid omitting the >> mailing list. To unsubscribe or change options: >> https://lists.samba.org/mailman/listinfo/rsync Before posting, >> read: http://www.catb.org/~esr/faqs/smart-questions.html - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Florida k...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlE6q+AACgkQVKC1jlbQAQdbjwCg2+JqHLJZycDl6vW697T9ezQG 7/0AoKMsyxOtuQnpdEX7vyXnbz10cY1Z =sNuu -----END PGP SIGNATURE----- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
