Rsync link-dest not working for a host with the most recent rsync/cygwin 1.7
It appears something about the about rsync or the 1.7 version of cygwin is preventing the --link-destination function of rsync from working. Files that have not been changed at all should be linked together. On the hosts still running a cygwin 1.5 version of rsync the link-destination seems to work exactly as I expect it to. I am using dirvish (http://www.dirvish.org/) on a Linux box (deb 5.0.3) to backup some of the data on windows boxes in the network. I have the cygwin+rsync setup in daemon mode on the servers. I recently upgraded a couple of the boxes to the latest cygwin 1.7. On the systems with cygwin 1.5 files are occasionally missed because of the path length limitations so I really want to get a cygwin 1.7 setup working. This is the command that is being issued by dirvish to perform the backup. rsync -vrltH --delete -pgo --stats -z -D --numeric-ids --timeout=300 --exclude-from=/srv/dirvish/srv-01/2009101610/exclude --link-dest=/srv/dirvish/srv-01/2009101420/tree 10.2.35.241::data/ /srv/dirvish/srv-01/2009101610/tree Here is the checksums and the output of stat of two files on my Linux box that I should have been linked together. No changes where made to this file on the windows box between the first and second backups. r...@pabil:# sha256sum 2009101420/tree/staff/Supply\ letter\ preschool.doc 2009101610/tree/staff/Supply\ letter\ preschool.doc 35cac3b2f0a54199da06c7ffdd5addf8634fa1d0829c11be8af037cf5572ddd1 2009101420/tree/staff/Supply letter preschool.doc 35cac3b2f0a54199da06c7ffdd5addf8634fa1d0829c11be8af037cf5572ddd1 2009101610/tree/staff/Supply letter preschool.doc r...@pabil:# stat 2009101420/tree/staff/Supply\ letter\ preschool.doc 2009101610/tree/staff/Supply\ letter\ preschool.doc File: `2009101420/tree/staff/Supply letter preschool.doc' Size: 22016 Blocks: 48 IO Block: 4096 regular file Device: fd02h/64770dInode: 12042401Links: 1 Access: (0700/-rwx--) Uid: ( 544/ UNKNOWN) Gid: (0/root) Access: 2009-10-14 20:50:32.0 -0700 Modify: 2005-08-26 13:58:39.0 -0700 Change: 2009-10-14 20:50:32.0 -0700 File: `2009101610/tree/staff/Supply letter preschool.doc' Size: 22016 Blocks: 48 IO Block: 4096 regular file Device: fd02h/64770dInode: 29475538Links: 1 Access: (0700/-rwx--) Uid: ( 544/ UNKNOWN) Gid: (0/root) Access: 2009-10-16 10:51:06.0 -0700 Modify: 2005-08-26 13:58:39.0 -0700 Change: 2009-10-16 10:51:06.0 -0700 I have posted the output of 'cygcheck -s -v -r' here (http://pastebin.com/m68d01c02). I would appreciate some help getting the --list-dest to work properly. If you have some ideas please share. If you need more information to help diagnose or find a solution please ask. Thanks, Chris Francy -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Rsync link-dest not working for a host with the most recent rsync/cygwin 1.7
First things first, I have narrowed it down and learned something that resolves the issue for what I am trying to do. For versions of rsync before 2.5.6 you could not use the link-dest reliably unless you copy the permissions/ownership information. With newer versions of rsync this is not necessary. Since I now have newer versions of rsync on both the backup server and the hosts I am backing up I was able to omit the options to copy the permissions/ownership information and things are being linked as I expect them to. I still find it a bit odd that this worked fine on a cygwin 1.5 setup by not 1.7. > On Oct 19 11:55 Corinna Vinschen wrote: >> On Oct 16 13:51, Chris Francy wrote: >> It appears something about the about rsync or the 1.7 version of >> cygwin is preventing the --link-destination function of rsync from >> working. Files that have not been changed at all should be linked >> together. On the hosts still running a cygwin 1.5 version of rsync >> the link-destination seems to work exactly as I expect it to. >> >> I am using dirvish (http://www.dirvish.org/) on a Linux box (deb >> 5.0.3) to backup some of the data on windows boxes in the network. I >> have the cygwin+rsync setup in daemon mode on the servers. > > I'm not fluent with the --link-dest option. I'm also missing a hint > what is wrong. Are you expecting hardlinks and get copies of the files? Sorry. Yes I am expecting identical files between to be hard-linked together. > If I want to reproduce this locally, I need a testcase which allows > to reproduce this with just a single file which gets rsynced locally, So to demonstrate the link-dest function on my linux box here is a simple example using relative directories including the output on my backup server (debian linux 5.0) (http://pastebin.com/m191da91f). I have a test case (http://pastebin.com/m6797debc) that I ran against both two of my systems to demonstrate the difference in behavior. Here (http://pastebin.com/m77f9f3b6) is my output when I run the command on my backup server to get the stuff off my 1.5 host. Here (http://pastebin.com/m29252e) is my output when I run the command on my backup server to get the stuff off my 1.7 host. In this case the test file in the two destination directories are not hard linked together like in the 1.5 case. I also noticed that in the the uid/gid is set differently compared to the 1.5 system. So it appears something about permissions isn't coming across the same as it was on cygwin 1.5. Since I don't care about the retaining the permissions, and I now have a version of rsync on both sides that allows me to use link-dest without copying permissions I am happy. If someone wants to continue digging into this further I am still available. Thanks, Chris -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Rsync link-dest not working for a host with the most recent rsync/cygwin 1.7
On Tue, Oct 20, 2009 at 10:12 PM, Steven Monai wrote: > Chris Francy wrote: > > I'm in a situation somewhat similar to yours. Up to now I have been > using the current stable Cygwin (v1.5) to test rsync as a possible > backup solution. I have a home-baked rsync/--link-dest script that > ... > limitations, but it would really suck if the --link-dest magic stopped > working in v1.7. I have learned that if I don't copy the permissions/ownership information then the link-dest works fine. I only used those options in the past because older versions of rsync would not do link-dest unless you copied ownership/permissions. > Probably a dumb question, but: Are you 100% certain that > 2009101420/tree/ and 2009101610/tree/ are on the same filesystem? Yup, 100% certain. > Could you post the contents of your rsync daemon's rsyncd.conf ? Are you > using identical rsyncd.conf files in your 1.5 and 1.7 installations? Yup, i am sure they are identical, and the config is very simple. --- rsyncd.conf use chroot = false strict modes = false hosts allow = 10.x.x.x log file = rsyncd.log pid file = rsyncd.pid [data] path = /cygdrive/d read only = true transfer logging = no --- > Is --link-dest failing completely (i.e. no hardlinks created at all), or > only partially (i.e. some unchanged files are hardlinked, while some are > merely copied)? What version of rsync is running on the Debian box? Absolutely nothing was getting linked together. r...@linuxhost:$ rsync --version rsync version 3.0.3 protocol version 30 Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others. Web site: http://rsync.samba.org/ Capabilities: 64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints, socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace, append, ACLs, xattrs, iconv, symtimes Thanks, Chris -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Rsync link-dest not working for a host with the most recent rsync/cygwin 1.7
On Thu, 22 Oct 2009 20:08 PM, Steven Monai wrote: >> Since I don't care about the retaining the >> permissions, and I now have a version of rsync on both sides that >> allows me to use link-dest without copying permissions I am happy. If >> someone wants to continue digging into this further I am still >> available. > > Thanks in advance for indulging me just a little bit further. > > Could you re-run your Cygwin-1.7 test case with --itemize-changes added > to the rsync command-line? That should provide a hint at what Debian's > rsync thinks is different between its existing file copy and Cygwin's > source file. Also, I would like to see what Cygwin's 'stat' command says > about the source file on the Windows box. The output of stat of my testfile on the box with cygwin 1.7 is stat testfile File: `testfile' Size: 1048576 Blocks: 1024 IO Block: 65536 regular file Device: bcdb6416h/3168494614d Inode: 27303072741079683 Links: 1 Access: (0700/-rwx--) Uid: (4294967295/ UNKNOWN) Gid: ( 401/mkpasswd) Access: 2009-10-20 23:37:29.577294700 -0700 Modify: 2009-10-21 00:36:28.0 -0700 Change: 2009-10-20 23:37:30.030425500 -0700 Here (http://pastebin.com/m6c414439) is the output of my rerunning my tests against the server with cygwin 1.7 with the --itemize-changes and -vv rsync options. Chris -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
