[Bug 3253] Feature request - Sync perms/ownership/time etc but with no file transfer

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3253


[EMAIL PROTECTED] changed:

   What|Removed |Added

   Severity|normal  |enhancement
 Status|NEW |RESOLVED
 Resolution||WONTFIX




--- Comment #1 from [EMAIL PROTECTED]  2005-11-16 12:45 MST ---
I think this would be best handled by some other application.  For instance, it
would be simple to write a perl script that would be able to parse the output
of "ls -lR" from one system and apply the permissions/ownership/groups to the
matching files on another system (skipping those that don't exist).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3253] Feature request - Sync perms/ownership/time etc but with no file transfer

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3253





--- Comment #2 from [EMAIL PROTECTED]  2005-11-16 18:51 MST ---
Created an attachment (id=1575)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1575&action=view)
perl script solution

I enjoy coding simple perl scripts like this, so here is a perl script that
will parse the output of "ls -lR dir [dir...]" or "ls -nR dir [dir...]" and
restore the permissions, owner, and group of all the files it finds.  You can
feel free to manually edit ls's output to remove unnecessary directories and/or
files, and then use the result to apply the listed attrs to another host's
files.  For instance, assuming the script "file-attr-restore" is somewhere in
the $PATH of server "remotehost", this sequence of commands would should update
the attrs on remotehost:

ls -lR /etc /usr/local/etc >saved-permissions
cat saved-permissions | ssh remotehost file-attr-restore


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] New: Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271

   Summary: Rsync instances stay in memory when using in daemon mode
   Product: rsync
   Version: 2.6.7
  Platform: x86
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Hello, all

We are using rsync as file transfer tool to make run-time
synchronization of two remote servers. So rsync is started each time a
file is modified to make changes be propagated to the remote peer.

Receiving side.
   The rsync is working in daemon mode (rsync --daemon)

Sending site.
   There is a frequently modified file, which is sended by rsync to
receiving side after each modification.
   Before sending the file our system performs check for existing rsync
process, which are already sending the same file.
   If it is so, this rsync process is terminated by sending to it TERM
signal, and next starting new rsync process.

Problem.
   Sometimes on the receiving side appear child rsync processes which
permanently hang in memory and never finish. I observed that these processes
write to log file string "rsync: read error: Connection reset by peer" and then
hang.
   After I removed logging from rsync (see log.c file) this problem is
not appeared anymore.
   This problem is encountered only when rsync is in the daemon mode.
When we are using rsync in the standalone mode all works fine.

   I assume that the cause of the problem is the usage of string
formatting functions in the signal handlers in the rsync source code.
Many of such formattings functions are using static buffers/variables
and when invoked in signal handler may caused to inpredicable results.

All testing was done on the SuSe 9.0 Linux x86 with rsync 2.6.6 release
and with the latest CVS snapshot.


:/var/log/trace/rsync> ps ax|grep rsync
18998 pts/8S  0:36 strace -f rsync --daemon --no-detach
18999 pts/8S  0:00 rsync --daemon --no-detach
19450 pts/8S  0:00 rsync --daemon --no-detach
19945 pts/8S  0:00 rsync --daemon --no-detach
20049 pts/8S  0:00 rsync --daemon --no-detach
20077 pts/8S  0:00 rsync --daemon --no-detach


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271





--- Comment #1 from [EMAIL PROTECTED]  2005-11-18 06:21 MST ---
Created an attachment (id=1580)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1580&action=view)
log file of strace, log file of rsync and log.c file


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #1 from [EMAIL PROTECTED]  2005-11-22 14:16 MST ---
Any ideas on this?  It's been open 5 weeks and probably got overlooked...  Tnx!


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3277] New: [Feature req] "I couldn't do something you asked" warning

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3277

   Summary: [Feature req] "I couldn't do something you asked"
warning
   Product: rsync
   Version: 2.6.7
  Platform: x86
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I've recently been burned a couple of times by trying to copy filesystems with
--numeric-id and forgetting to do it as root.  The transfer succeeds just fine,
but of course the owner/group etc info can't be set, and this was
-particularly- easy to overlook because the same users (mostly) existed at both
ends, but with different UIDs, so it's easy to miss that something went wrong
because mostly you see what looks like "correct" output from ls & friends, but
is actually wrong because some of the UIDs weren't set, etc.  (This was
actually what I was mostly doing with your recent file-attr-restore script...)

Even though I was running with -v, I don't recall seeing any errors emitted
when this happened.  I'm wondering whether it would be a good idea to create an
option to make this much more obvious, so the cautious among us can set it by
default in aliases & scripts & whatnot and hence be warned when we're about to
make an annoying-to-fix mistake; it would also make it less critical to
carefully scan for these hard-to-notice failures---after all, rsync is supposed
to -save- us from that sort of thing...

Just enabling it with excessive verbosity is probably not the right thing,
since it'd get lost in the noise, and most people probably don't run at high
verbosities if they're not -already- debugging something.  I'm of mixed
opinions about whether the behavior should be a default, since it could be
irritating to see thousands of "couldn't set uid/gid" messages if you're okay
with not running as root.  If the option is enabled, a warning should
definitely be emitted at the end of the run (where it's not buried) that
something went wrong (and probably the exit status should change).  And perhaps
there should be another option that emits a warning for each file?  Or
something like --warn-me=none,end,all or something like that.

There are probably other things besides uid/gid that might benefit from this
(tiemstamps? ACLs?), but I fear either a huge number of warnings if they're all
enabled, or the hair of a warning bitmask ("tell me about UID/GID, but not
about problems setting permissions") or something.

If this looks like the sort of thing that might be better hashed out on the
list, I can forward this there instead; let me know.

Thanks!


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #2 from [EMAIL PROTECTED]  2005-11-22 23:41 MST ---
What version are you using?  You have 2.6.7 selected in the bug report, but
that's still in development.

The copy-on-write optimization wasn't done until v2.6.1 (Apr 2004):

- The generator is now better about not modifying the file list
  during the transfer in order to avoid a copy-on-write memory
  bifurcation (on systems where fork() uses shared memory).
  Previously, rsync's shared memory would slowly become unshared,
  resulting in real memory usage nearly doubling on the receiving
  side by the end of the transfer.  Now, as long as permissions
  are being preserved, the shared memory should remain that way
  for the entire transfer.

You use the -p option, so you meet the "permissions being preserved" condition.

The file_struct data and other chunks of data are allocated out of the free
memory pool.  If lots of those allocated chunks are returned to the pool, pool
management involves memory being modified, so that would require new writes.

Wayne - does rsync free up anything substantial at any time after the fork that
might trigger this?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #3 from [EMAIL PROTECTED]  2005-11-23 00:34 MST ---
I'm actually using 2.6.7.  In fact, I'm using a version from CVS in which Wayne
added --min-size and --max-size, and fixed (and then re-fixed) a bug in hlink.
This isn't currently the very latest CVS, but I believe corresponds to
rsync-HEAD-20051014-2036GMT, plus the hlink stuff.  I could fairly trivially
update to the very latest CVS if it was useful in figuring out what's going on;
my fundamental question was, "Is this a bug or am I misunderstanding
something?" and it's sounding like you believe it's a bug.

I'm reasonably sure that I saw this same behavior in the rsync that ships in
Ubuntu Breezy, which is their (somewhat Debianized and with a broken
--min|max-size) version of 2.6.5.  If it was really necessary, I could
re-verify that I saw this behavior in that version, although it'd take some
work.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #4 from [EMAIL PROTECTED]  2005-11-23 05:19 MST ---
Thanks for confirming that you are using CVS.  No need to update to the very
latest.

Hmm.  You are using --delete, which is done before any file transfers.  Ah.
 Looks like it builds an entire equivalent file list for files on the receiving
side.  So it may also be building a 200MB file list to do deletes.

Try running without --delete and see if that extra memory usage still happens. 
If it goes away, try using --delete-during, which now that I think about it,
was intended to solve this very problem with large numbers of files.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 2607] Rsync logging time incorrectly

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=2607





--- Comment #7 from [EMAIL PROTECTED]  2005-11-23 14:09 MST ---
(In reply to comment #6)

The problem is that glibc's implementation (2.3.5) of strftime() calls tzset(),
which attempts to reload the timezone information.  If the environment variable
TZ has not been set, and the timezone file has not been copied into the chroot
jail, then the first attempt to use strftime() causes the timezone to revert to
UTC for all future time accesses in the child process.  If one adjusts the
config.h so that strftime is not used, the problem disappears.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |ASSIGNED




--- Comment #5 from [EMAIL PROTECTED]  2005-11-23 17:11 MST ---
I have been trying various copy commands to try to duplicate this, but haven't
seen anything wrong.  If -p is left off the memory for the shared file list
will become unshared, but I haven't yet seen another scenario that would cause
that.

> Looks like it builds an entire equivalent file list
> for files on the receiving side.

Older rsync versions did create a duplicate file list when deleting, but that
was changed in recent releases to only need enough memory for a single
directory at a time plus whatever memory is needed for the scanning function to
recurse down to the deepest dir.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271





--- Comment #2 from [EMAIL PROTECTED]  2005-11-23 22:07 MST ---
I have found that adding "#define SHUTDOWN_ALL_SOCKETS" to config.h (at least
on the client side) solves the problem.  Without it, the connection is closed
with an error (connection reset), rather than with an end-of-file.  The
server's receiving code handles the error, and it goes into _exit_cleanup and
gets all the way to the exit() call, but exit() hangs because it tries to flush
a socket that is in an error state.  If the client properly closes its sockets,
the problem does not occur.

Of course, this is only a workaround.  It does not handle the case where the
connection is legitimately reset because of network errors.  It just keeps
rsync from foiling itself.  A more complete solution in the receiving code is
required.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #6 from [EMAIL PROTECTED]  2005-11-23 23:07 MST ---
I tried adding --delete-during (so the full invocation now looks like "rsync
-vrltH --delete -pgo --stats -z -D --numeric-ids -i --delete-during
--exclude-from=/FS/dirvish/HOST/20051124/exclude
--link-dest=/FS/dirvish/HOST/20051123/tree [EMAIL PROTECTED]:/
/FS/dirvish/HOST/20051124/tree" and the behavior didn't change.

But now that I think about it, it's not clear if --delete could be a problem in
the first place, because these are dirvish runs.  That means that I'm using -H
and --link-dest to populate a tree that originally starts out empty, and winds
up containing only a very few files that consume actual disk space (whatever
got created or modified since the dirvish run yesterday), and about 2 million
hardlinks into the previous day's tree.  If rsync is writing to an
otherwise-empty tree, it seems to me that --delete has nothing to do---which
makes me wonder why dirvish even bothers to supply it automatically, frankly,
since dirvish -always- starts from an empty destination tree.  Is there some
reason why it makes sense to supply --delete at all?  (Unfortunately, we can't
ask dirvish's original author why he did this, alas.)  Or does --delete cause
process inflation if there -isn't- much to do instead of if there -is-?

Once this run completes in a couple hours (I'm debugging some other, unrelated
things at the same time in this run), I may just blow its tree away and start
over without --delete in any form (by editing the dirvish script) and see if
that changes its behavior, but I'd be pretty mystified if it did unless my
understanding of --delete, --link-dest, and empty destination trees is just
wrong.

Just in case I'm being completely faked out here and the second process really
is sharing most of its memory, here are the top few lines of "top" running on
the destination host:

top - 00:34:30 up 8 days,  8:25,  2 users,  load average: 3.05, 2.00, 0.96
Tasks:  65 total,   3 running,  62 sleeping,   0 stopped,   0 zombie

Cpu(s):  6.0% us, 39.9% sy,  0.0% ni,  0.0% id, 51.3% wa,  2.0% hi,  0.7% si
Mem:516492k total,   508012k used, 8480k free,76180k buffers
Swap:  1544184k total,12476k used,  1531708k free,68404k cached

  PID USER  PR  NI  VIRT  RES  SHR S %CPU %MEMTIME+  COMMAND
10795 root  18   0  259m 253m  676 D 15.2 50.3   0:47.19 rsync
10865 root  16   0  251m 245m  688 S  0.0 48.7   0:00.08 rsync

What's actually kinda interesting there is that it claims to have 8m free, and
76m of buffers, -and- to have 253+245m of rsync resident, all on a machine with
only 512m total memory (and not including ~30m of other processes).  And yet
I'm pretty sure I -saw- the free memory go from about 200m to about 0 when that
second process started up on, on previous runs.  (On this one, I didn't quite
catch it in the act and am not sure how much free memory there was before the
inflation of the second process.)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #7 from [EMAIL PROTECTED]  2005-11-24 11:27 MST ---
You are right that dervish does not need to use --delete when copying into a
new directory, but it also doesn't hurt anything (since it won't actually do
much of anything).

I read something that made it sound like this might be a recent change in the
Linux kernel, so I added a sleep(1000) to both the generator and the receiver
right after they fork and then ran a test on a system with a linux-2.4 kernel
and a linux-2.6 kernel.  The processes stayed shared on the 2.4 system, but
became unshared right after the fork on the 2.6 kernel, so this appears to be
something that needs to be investigated in Linux itself.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3186] Surprisingly large unshared memory usage

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3186





--- Comment #8 from [EMAIL PROTECTED]  2005-11-24 12:28 MST ---
Yikes.  Well, I'm certainly runnning 2.6 on everything here.

You're certainly in a better position than I am to try to bug-report this to
the kernel developers; is that your next step?  (I'm assuming there wasn't some
API change that makes this "not a bug", but I haven't investigated.)

(Thanks!)

Btw, I suspect that dirvish's use of --delete might have originated in
debugging, or if someone tries to recreate a failed run by redoing it on the
same day (and hence in the same tree) without blowing away the tree first,
since by default the trees are named by dates.  In those cases, --delete would
make sense, and since leaving it in theoretically will do nothing in the usual
case, I'll ignore it.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271





--- Comment #3 from [EMAIL PROTECTED]  2005-11-24 19:30 MST ---
OK, further research has shown that I was not quite right about the cause of
the problem.  In my tests, the network i/o process does die, and it's the
"generator" process that hangs.  I traced it all the way into the exit_cleanup
routine, and it hangs when it calls log_exit().


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271





--- Comment #4 from [EMAIL PROTECTED]  2005-11-26 10:05 MST ---
(In reply to comment #3)
> In my tests, the network i/o process does die, and it's the
> "generator" process that hangs.  I traced it all the way into the exit_cleanup
> routine, and it hangs when it calls log_exit().

The generator process tries to write its error messages over the network
socket, which has gone into an error state.  However, it does not detect that
because select() keeps saying that the descriptor is not available for writing,
so it enters an infinite loop.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271





--- Comment #5 from [EMAIL PROTECTED]  2005-11-26 10:08 MST ---
Created an attachment (id=1594)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1594&action=view)
patch to prevent lingering processes

This causes the writefd_unbuffered routine to check if the socket has died. 
However, it seems to break syslog when chrooted, so let's be careful.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271


[EMAIL PROTECTED] changed:

   What|Removed |Added

Attachment #1594 is|0   |1
   obsolete||




--- Comment #6 from [EMAIL PROTECTED]  2005-11-26 12:23 MST ---
Created an attachment (id=1595)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1595&action=view)
different way of handling a broken socket

Perhaps this is better.  The generator process doesn't use the timeout given in
the config file anyway.  It is designed to block!  The other patch may give up
on the socket prematurely.  This one tries to write, though it may block for a
long time.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271


[EMAIL PROTECTED] changed:

   What|Removed |Added

Attachment #1595 is|0   |1
   obsolete||




--- Comment #7 from [EMAIL PROTECTED]  2005-12-01 19:24 MST ---
Created an attachment (id=1604)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1604&action=view)
even better way of handling a dead socket

This patch doesn't break the contract of writefd_unbuffered because it risks a
blocking write only when no timeout has been specified and there is no
io_error_fd to read.  It avoids the messy attempts to detect a dead socket.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3298] New: --rsh option incorrectly interprets quotes

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3298

   Summary: --rsh option incorrectly interprets quotes
   Product: rsync
   Version: 2.6.6
  Platform: x86
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Using rsync -e 'ssh -i "/some directory with spaces/identity"' causes rsync to
try to execute ssh like:

execve("/usr/bin/ssh", ["ssh", "-i", "\"/some directory with
spaces/identity\"", ...])

Which of course fails, because the quotes were to delimit the argument, not
part of the filename.  This makes it more difficult to use options with spaces
and is inconsistant with the --rsync-path option, which uses sh to interpret
the command.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |ASSIGNED




--- Comment #8 from [EMAIL PROTECTED]  2005-12-02 18:05 MST ---
Thanks for the suggested patches.  I agree that those continue statements after
select() returns -1 look like they should be improved, but I'm not sure why you
allowed the code in the read_timeout() function to fall through after an error:
the bit-field values will be left undefined by the error, so I think we need
more code to figure out what to do.  Perhaps we should change the code to only
continue if count is 0 or if errno is EINTR, EWOULDBLOCK, or EAGAIN (and exit
with an error on all other errno values).

In the writefd_unbuffered() function, what errno is being returned when your
new code is being triggered?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3277] [Feature req] "I couldn't do something you asked" warning

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3277


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |ASSIGNED




--- Comment #1 from [EMAIL PROTECTED]  2005-12-02 18:34 MST ---
I think that this problem is limited to user and group settings when not run as
root. The complication is that -a implies -o and -g, but we don't want errors
to be returned when running as a normal user for a normal copy. One possible
solution is to add some kind of an extra mode where rsync knows that it should
definitely try setting the user and/or group and report a failure if it cannot.

There is a diff in the patches dir named owner-group-mod.diff that chooses to
force the user to specify -oo and/or -gg to tell rsync to trying setting all
users and/or all groups no matter what.  I have debated whether -aog should be
enough to also trigger these new actions or if (as the patch used to be
implemented) -oogg is still required even if -a was also specified.  Any
thoughts?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271





--- Comment #9 from [EMAIL PROTECTED]  2005-12-02 21:23 MST ---
(In reply to comment #8)
> Thanks for the suggested patches.  I agree that those continue statements 
> after
> select() returns -1 look like they should be improved, but I'm not sure why 
> you
> allowed the code in the read_timeout() function to fall through after an 
> error:


I didn't change read_timeout().


> In the writefd_unbuffered() function, what errno is being returned when your
> new code is being triggered?


Good point.  When select() returns an error, we ought not to write.  So, my
code could be put right after check_timeout().  The bug is caused by the fact
that select() doesn't return an error.  It keeps returning 0.  In a perfect
world, select() would detect that the socket is in an error state, but it
doesn't.  It sees a full transmit buffer and says that the socket is not ready.

How about this:

if (count == 0) {
  check_timeout();
  --- my patch ---
}

if (count <= 0) {
   --- leave this as is ---
}


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3271] Rsync instances stay in memory when using in daemon mode

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3271


[EMAIL PROTECTED] changed:

   What|Removed |Added

Attachment #1604 is|0   |1
   obsolete||




--- Comment #10 from [EMAIL PROTECTED]  2005-12-02 22:35 MST ---
Created an attachment (id=1605)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1605&action=view)
avoids select() call when possible

Forget what I said in my last post.  It doesn't fix the bug!!

Perhaps this patch is clearer?  It avoids the select() call entirely when it is
safe to block.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3299] New: rsync: now replaces non-ASCII character with numerical values

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3299

   Summary: rsync: now replaces non-ASCII character with numerical
values
   Product: rsync
   Version: 2.6.6
  Platform: Other
   URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=307242
OS/Version: Linux
Status: NEW
  Severity: major
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Package: rsync
Version: 2.6.6-1
Followup-For: Bug #307242


I've again used two identical Sarge systems, both using UTF-8.

Using rsync (over ssh), syncing (or listing the contents) from one
system to the other, non-ASCII characters get replaced with numerical
values like '\303\245', eg:

  [EMAIL PROTECTED]:~$ rsync system2:~/test_åäö_test
drwxr-xr-x  72 2005/09/25 01:39:30 test_\303\245\303\244\303\266_test

The changelog states:
--
rsync (2.6.5-1) unstable; urgency=low 
   * Now should handle locale-specific characters better in logging output
 (i.e. the correct chars should be displayed, not '?').
--

This statement is obviously not correct. The '?' has just been replaced
with a numerical value instead. (Almost as useless.)

Is this something that's being worked on upstream? Is there a workaround?
All scripts and programs depending upon the output is almost useless since
several month now.

-- System Information:
Debian Release: 3.1
Architecture: i386 (i686)
Kernel: Linux 2.6.8-2-686-smp
Locale: LANG=sv_SE.UTF-8, LC_CTYPE=sv_SE.UTF-8 (charmap=UTF-8)

Versions of packages rsync depends on:
ii  libc6   2.3.2.ds1-22 GNU C Library: Shared libraries an
ii  libpopt01.7-5lib for parsing cmdline parameters

-- no debconf information


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3304] New: on --delay-updates remove .~tmp~ recursively

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3304

   Summary: on --delay-updates remove .~tmp~ recursively
   Product: rsync
   Version: 2.6.6
  Platform: x64
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


On --delay-updates delayed files saved in .~tmp~. On error (disconnect, server
or client killed) Delayed files keeped in .~tmp~. (May be better remove it on
disconnect, but this is another bug and don't concerned client kill or poweroff
client station.) On next rsync they are rewriten and .~tmp~ removed. But if one
of delayed files removed on server side before next rsync, it (in .~tmp~) and
.~tmp~ don't removed after next rsync.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3305] New: -t copies always too many files to mounted windows volume -c copies no files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3305

   Summary: -t copies always too many files to mounted windows
volume -c copies no files
   Product: rsync
   Version: 2.6.7
  Platform: Other
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


rsync -t does not work on a local rsync to a mounted windows directory.
rsync copies the same files (about 50%) always and everytime.
Maybe the strange Creation time mismatch (20 sec in the future on windows) is
responsible. But that is a guess.

=== FSTAB:
//swat/Intranet /mnt/backup smbfs auto,credentials=/etc/fstab.credentials 0 0
Windows server OS: probably windows 2000

=== Try1:
swale # rsync -rt --stats /srv/www/htdocs /mnt/backup/www

Number of files: 3223
Number of files transferred: 1337
Total file size: 27807498 bytes
Total transferred file size: 15169982 bytes
Literal data: 15169982 bytes
Matched data: 0 bytes
File list size: 69523
Total bytes sent: 15299405
Total bytes received: 26760

sent 15299405 bytes  received 26760 bytes  519531.02 bytes/sec
total size is 27807498  speedup is 1.81

== Try2:
swale # rsync -rc --stats /srv/www/htdocs /mnt/backup/www

Number of files: 3223
Number of files transferred: 0
Total file size: 27807498 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 115875
Total bytes sent: 115887
Total bytes received: 20

sent 115887 bytes  received 20 bytes  3679.59 bytes/sec
total size is 27807498  speedup is 239.91

=== Try3:
swale # rsync -rt --stats /srv/www/htdocs /mnt/backup/www

Number of files: 3223
Number of files transferred: 1337
Total file size: 27807498 bytes
Total transferred file size: 15169982 bytes
Literal data: 15169982 bytes
Matched data: 0 bytes
File list size: 69523
Total bytes sent: 15299405
Total bytes received: 26760


== strange result for testing time sync problems * see creation time in the
future ===
swale # date ; touch foo.junk ; date
Wed Dec  7 10:05:17 CET 2005
Wed Dec  7 10:05:17 CET 2005

Windows explorer about foo.junk:
Creation: Wed Dec  7 10:05:35 CET 2005
Modified: Wed Dec  7 10:05:16 CET 2005
Access: Wed Dec  7 10:05:16 CET 2005


===
swale # rsync --version
rsync  version 2.6.3  protocol version 28
Copyright (C) 1996-2004 by Andrew Tridgell and others

Capabilities: 64-bit files, socketpairs, hard links, acls, symlinks,
batchfiles, 
  inplace, IPv6, 64-bit system inums, 64-bit internal inums, SLP

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3305] -t copies always too many files to mounted windows volume -c copies no files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3305





--- Comment #1 from [EMAIL PROTECTED]  2005-12-07 05:10 MST ---
The comment about always copying 50% of the files to a mounted Windows
directory pretty much identifies the problem.

See the --modify-window option for the solution.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3277] [Feature req] "I couldn't do something you asked" warning

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3277


[EMAIL PROTECTED] changed:

   What|Removed |Added

 CC||foner-rsync-
   ||[EMAIL PROTECTED]




--- Comment #2 from [EMAIL PROTECTED]  2005-12-08 16:16 MST ---
I think what I'd like to see is a single option called --strict or something
(surely there's a better name) which I can either set all the time (e.g., by
calling rsync via an alias that sets it), or a corresponding environment
variable I can set in my init.  (And, of course, either/both would have to take
--nostrict so I can turn it off on a case-by-case basis if necessary.)  If I
have to remember to set a separate option, I'll probably forget, and that's
exactly what I'm trying to avoid---I typically wind up typing (or scripting,
even worse) some rsync command and then realizing days or weeks later that I
-meant- to get owners/groups set but didn't happen to run the command as root,
by which point it's way too late. And this seems conceptually simpler to me
than the peculiar other ways of specifying owner/group you mention below--and
extensible to other cases where the user may want to know that rsync wasn't
quite able to fulfill the request (e.g., ACL's or timestamps maybe).

I'm not sure whether it should complain on every file or just issue a warning
at the end, though.  The former might produce a lot of output, but OTOH, it'd
sure be hard to miss, and you'd only get it if you said --strict in the first
place...


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3277] [Feature req] "I couldn't do something you asked" warning

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3277





--- Comment #3 from [EMAIL PROTECTED]  2005-12-09 16:19 MST ---
Instead of treating -o and -g specially, I think it is time rsync had separate
archive options for "running as a normal user for a normal copy", i.e. using
default destination security settings, and preserving all security settings.

Since there are tons of existing backup scripts that rely on -a/--archive
preserving all security settings, I think -a should continue to behave as it
does now, except that inability to set user or group always causes an error
message.  A new option can modify -a so that it does not imply -pog.  I
recommend calling this option -U/--def-security; -U, mnemonic for "use umask",
is probably the best letter that is not already taken.

For good measure, one could also add an option -s/--security that preserves all
security settings (-pog).  That way, people can start to use "rsync -as ..." in
their backup scripts, and it might eventually be practical to change the
behavior of unqualified -a to something like the tar-esque "-rltD, plus -s if
run as root".

Once the two modes are available, I think rsync should take all options
seriously (i.e. -ogD) whether or not it is root.  I also think it should issue
a warning about every failure (i.e. every file) because all three of these
options can succeed on some files and fail on others and it's important that
the user gets the message.  A concise output format that coalesces
preservations that failed for the same reason might be good:
rsync: "/foo/badfile": failed to preserve time, owner, group: Operation not
permitted (1)
rsync: "/foo/badfile": failed to preserve ACLs: Operation not supported (2)

I might try to make some of these changes in my custom rsync.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3326] New: rsync with local --files-from and remote source

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3326

   Summary: rsync with local --files-from and remote source
   Product: rsync
   Version: 2.6.6
  Platform: Other
OS/Version: AIX
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I'm trying to copy files from a remote source to the local system, but using
a local list of files. This fails because it seems that rsync is insisting the
file list must also be remote.

# ./rsync -n  --files-from=/tmp/files remotehost:/ /tmp/

rsync: on remote machine: --files-from=-: unknown option
rsync error: syntax or usage error (code 1) at main.c(994)
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(434)

The reason I want to do this is so that I can control file distributions from a
central "golden host" (which can ssh or rsh everywhere).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3326] rsync with local --files-from and remote source

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3326


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||INVALID




--- Comment #1 from [EMAIL PROTECTED]  2005-12-14 12:17 MST ---
This error:

rsync: on remote machine: --files-from=-: unknown option

is telling you that the remote rsync doesn't know about the --files-from
option, so it needs to be upgraded for you to be able to pull files from it
using --files-from.  (This is because the sender needs to be told that it is
getting a file list in addition to the command-line args, and when you're
pulling files, the sender is the remote system.)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3305] -t copies always too many files to mounted windows volume -c copies no files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3305


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||INVALID




--- Comment #2 from [EMAIL PROTECTED]  2005-12-14 12:27 MST ---
As John indicated, using --modify-window=1 should avoid this deficiency in the
Windows filesystem.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3298] --rsh option incorrectly interprets quotes

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3298


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||FIXED




--- Comment #1 from [EMAIL PROTECTED]  2005-12-14 15:04 MST ---
I added single- and double-quote parsing to rsync's handling of the
remote-shell command.  Rsync doesn't use the shell to parse this string (since
that could cause problems for the option-forwarding code).  I decided to not
include backslash parsing in order to (1) keep it simple and (2) avoid causing
problems for cygwin users.  I also decided to keep the quirk that tabs are not
considered arg-breaking characters.  This should give us very good
backward-compatibility.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 2802] rsync uses extraneous libresolv.dylib

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=2802


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||FIXED




--- Comment #1 from [EMAIL PROTECTED]  2005-12-14 15:54 MST ---
I changed the AC_CHECK_LIB() macro to AC_SEARCH_LIBS() so that libresolv will
not get included if it is not needed.  (This got rid of the need for libresolv
on Linux, and I'll be checking the build farm to make sure that this doesn't
cause a problem on other OSes.)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3334] New: some time rsync is exiting with received SIGUSR1 or SIGINT (code 20) at rsync.c(229).

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3334

   Summary: some time rsync is exiting with received SIGUSR1 or
SIGINT (code 20) at rsync.c(229).
   Product: rsync
   Version: 2.6.6
  Platform: All
OS/Version: Linux
Status: NEW
  Severity: major
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


+++ This bug was initially created as a clone of Bug #2760 +++

Rsync dies consistently after quite some time.  My filelist is huge.
Here's the error with 2.6.6 on SLES9 x86_64:
received SIGUSR1 or SIGINT (code 20) at rsync.c(163) 

The las guy was using version 2.5.7:
We are running the rsync over the Network, and it is getting killed some 
times. The error is as below.

received SIGUSR1 or SIGINT (code 20) at rsync.c(229)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3175] it would be nice if --link-dest matched up devices and symlinks too

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3175


[EMAIL PROTECTED] changed:

   What|Removed |Added

   Severity|normal  |enhancement
 Status|ASSIGNED|RESOLVED
 Resolution||FIXED




--- Comment #8 from [EMAIL PROTECTED]  2005-12-15 16:51 MST ---
The latest CVS version now has support for hard-linking devices and symlinks
via the --link-dest option (and falls back to making duplicates if the OS
doesn't allow that).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3334] some time rsync is exiting with received SIGUSR1 or SIGINT (code 20) at rsync.c(229).

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3334


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||WORKSFORME




--- Comment #1 from [EMAIL PROTECTED]  2005-12-15 17:12 MST ---
I noticed recently that this message is inaccurate:  it also gets output if
rsync gets killed by a SIGTERM or SIGHUP.  I'd imagine that your OS is killing
rsync because it is consuming too much memory (check your system logs to know
for sure).  I'd suggest dividing the file list up into multiple hierarchies and
trying again.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 2947] stdout with [-v] -H --link-dest and slink/sock/fifo/regf

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=2947


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution||FIXED




--- Comment #3 from [EMAIL PROTECTED]  2005-12-15 21:26 MST ---
I have checked in fixes for several of the items that you mentioned:

* Rsync now keeps quiet about an item in a hard-link cluster if it was properly
hard-linked in the --link-dest dir.

* Rsync now treats fifos/devices/symlinks the same as files when it is running
with --link-dest/--compare-dest/--copy-dest (I changed my mind on this point). 
This means that if those items are already up-to-date in a --FOO-dest dir, that
rsync won't mention it being in need of an update (and will hard-link it to the
prior version in the case of --link-dest and not create a new version at all in
the case of --compare-dest).

As mentioned in my first response, the delete issues you cited are not bugs:
rsync only mentions deletions from the actual destination directory, not extra
items that might be found in one or more of the adjunct --FOO-dest dirs.  Yes,
this does mean that using --delete along with --link-dest is usually useless
(because --link-dest works best into an empty hierarchy, and there's nothing to
delete in an empty hierarchy).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3358] New: rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358

   Summary: rsync chokes on large files
   Product: rsync
   Version: 2.6.6
  Platform: PPC
OS/Version: Mac OS X
Status: NEW
  Severity: major
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I try to rsync a 25-50 GB AES128 encrypted disk image called 'test' between two
Mac OS X-machines. This is with rsync 2.6.6 (is there a 2.6.7? The front page
just says 2.6.6)


% rsync -av --progress --stats --rsh=ssh /test 2nd-machine:/test
Warning: No xauth data; using fake authentication data for X11 forwarding.
tcsh: TERM: Undefined variable.
building file list ... 
1 file to consider
test
rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown" [sender]:
Broken pipe (32)
rsync: write failed on "/test": No space left on device (28)
rsync error: error in file IO (code 11) at
/SourceCache/rsync/rsync-20/rsync/receiver.c(312)
rsync: connection unexpectedly closed (92 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at
/SourceCache/rsync/rsync-20/rsync/io.c(359)
rsync: connection unexpectedly closed (1240188 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(434)


The receiving machine has space left (2+ GB). Before I upgraded to 2.6.6 I had
2.6.2 on the sending machine and 2.6.3 on the receiving machine. With that
combination I got another error message:

% rsync -av --progress --stats --rsh=ssh test 2nd-machine:/test
Warning: No xauth data; using fake authentication data for X11
forwarding.
tcsh: TERM: Undefined variable.
building file list ... 
1 file to consider
test
rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown":
Broken pipe
rsync error: error in rsync protocol data stream (code 12) at
/SourceCache/rsync/rsync-14/rsync/io.c(836)



The files _should_ be identical, I first transfered them with sftp without
problems but they will change in the future and then I want to use rsync to
keep them identical. This was just a test to verify my plan - a test that
didn't seem to workout that well.

I don't know if this matter but here are some more information about my setup:

Powerbook G3 with 10.3.9
Powerbook G4 with 10.4.3

Wireless 802.11G-network between router and G4, wired network between G3 and
router. The router is a Linksys WRT54GS.

Both the older versions and the most recent versions works very well when I
work with smaller filer (for example, I synchronized 40 GB with mp3:a without
problems)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

[Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #1 from [EMAIL PROTECTED]  2005-12-28 11:21 MST ---
The pertinent error is this:

rsync: write failed on "/test": No space left on device (28)

That is an error from your OS that indicates that there was no room to write
out the destination file.  Keep in mind that when rsync updates a file, it
creates a new version of the file (unless --inplace was specifed), so your
destination directory needs to have enough free space available to hold the
largest updated file.

As for why the file is updating, if the modified time and size don't match,
rsync will update the file (efficiently).  You can use the --checksum option to
avoid this unneeded update at the expense of a lot of extra disk I/O to compute
each file's checksum before figuring out if a transfer is needed.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #2 from [EMAIL PROTECTED]  2005-12-29 13:47 MST ---
Intereseting, didn't knwo that rsync worked that way - I thought the default
behaviour was to only replace the parts of the file that had changed. Anyway,
this  motivates a follow-up question:

If I understand it correctly if you file 1 on computer A and file 2 on computer
B and some minor changes has been made to 1 and you want to sync these changes
to B rsync basically make a copy of 2 and works with that. If 1/2 are big, like
in my example when they where 25-50 GB, the copy operation from 2.0 to 2.1
generates a lot of disk acticity.

In my case when I rsync between two laptops all this disk activity is a little
unfortunate since laptop drives are so slow. Now to my question, is there a way
to reduce disk activity? Does the --inplace switch work around this?

Thanks.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #3 from [EMAIL PROTECTED]  2005-12-29 13:48 MST ---
Btw, I am just trying your suggestions. First I will try the inplace switch and
secondly I will test syncing with twice the amount of space required for the
file available.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #4 from [EMAIL PROTECTED]  2005-12-29 13:54 MST ---
Sorry for spamming, but I just realised what you meant when you wrote:

You can use the --checksum option to avoid this unneeded update at the expense
of a lot of extra disk I/O to compute each file's checksum before figuring out
if a transfer is needed.


If rsyns is _not_ checksumming files, why does rsyns remain in this state:

building file list ... 
1 file to consider


for maybe 30 minutes when it transfers my big file?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3362] New: Add option to normalize Unicode filenames

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3362

   Summary: Add option to normalize Unicode filenames
   Product: rsync
   Version: 2.6.7
  Platform: Other
OS/Version: All
Status: NEW
  Severity: enhancement
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


This is somewhere between a bug and an enhancement. When moving files with
Unicode names between different OSes, it would make life a lot easier if there
was an option to normalize filenames to a particular unicode form on the
destination.

For example, rsyncing from OS X to Linux names the files using the OS X
standard, Normalization form D. Linux prefers Normalization form C, which means
sharing this folder via Samba results in a number of folders that can't be
accessed. Likewise when running rsync from OS X to a Windows box running
cygwin. I think the best solution to this would be something like
"--destination-charset=UTF-8-NFC", to rename the files from the source charset
to the destionation charset.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3363] New: rsync gets stuck on first file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3363

   Summary: rsync gets stuck on first file
   Product: rsync
   Version: 2.6.6
  Platform: x86
OS/Version: Windows XP
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I'm trying to use rsync to copy several pictures from my work computer
(xxx.xxx.xxx.edu) to my home computer.  I'm using cygwin.  This is what I run
on my home computer:

  $ rsync -avu --progress [EMAIL PROTECTED]:"My\ Pictures/" .

It successfully creates the first few directories, but when it starts on the
first file, it seems to get stuck.  The progress bar stays at all zeros, and it
doesn't seem to be using any CPU.

  [EMAIL PROTECTED]'s password:
  receiving file list ...
  1986 files to consider
  ./
  2004/
  2004/2004_10_03_Hoosiers/
  2004/2004_10_03_Hoosiers/IMG_1096.jpg
   0   0%0.00kB/s0:00:00

I don't think I'm doing anything unusual or fancy... maybe there's something
about my syntax that is wrong and I'm just not seeing it.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 2938] Escape Colons in Filenames

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=2938


[EMAIL PROTECTED] changed:

   What|Removed |Added

 CC||[EMAIL PROTECTED]




--- Comment #3 from [EMAIL PROTECTED]  2006-01-02 03:03 MST ---
Cygwin doesn't "translate" filenames in its default configuration. In order to
use names that are non Windows-compatible the managed mounts are to be used:
search "man mount" for "-o managed" option. Warning: managed mode CAN'T be used
on directories whose files were not ALL created using managed mode itself.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3363] rsync gets stuck on first file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3363


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||LATER




--- Comment #1 from [EMAIL PROTECTED]  2006-01-02 09:16 MST ---
The problem you're encountering is that pipes in cygwin lose data, and rsync
hangs waiting for that missing data to arrive.  Once cygwin gets their pipe
code fixed, rsync will work fine on cygwin.  In the meantime, you can avoid
this problem by avoiding a remote-shell -- i.e. switch over to running a daemon
and using daemon-mode's :: syntax (that may or may not be possible for you).

I'm resolving this as "LATER" because there is no other appropriate resolution
that indicates that the bug lies not in rsync, but in the OS.  Later, we can
verify that it is fixed (once this is fixed in cygwin).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||INVALID




--- Comment #5 from [EMAIL PROTECTED]  2006-01-02 09:49 MST ---
(In reply to comment #4)
> If rsyns is _not_ checksumming files, why does rsyns remain in this state:
> [...]
> for maybe 30 minutes when it transfers my big file?

Because it is transferring the file.  Yes, this involves file-transfer
checksumming, but I was talking about pre-transfer checksum generation (and its
use in determining which files get transferred) which is what --checksum
enables.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #6 from [EMAIL PROTECTED]  2006-01-02 10:21 MST ---
This is weird, there is no network activity during this building file list
phase. However, as soon as it is finished, rsync saturates my network.

I thought rsync worked, if the file's size and modification date doesn't match,
by creating a binary tree and then checksumming the parts between every node,
recursively to the root of the tree, and then only transferring the parts where
the checksum didn't match.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #7 from [EMAIL PROTECTED]  2006-01-02 11:02 MST ---
(In reply to comment #6)
> This is weird, there is no network activity during this building file list
> phase. However, as soon as it is finished, rsync saturates my network.

What is weird about that?  As soon as rsync outputs the "1 file to consider"
message, the file-list-building stage is over, and rsync then starts to
transfer the file if it is in need of an update.  (If --checksum was specified,
the receiving rsync would first be busily checksumming the file to decide if
the file was actually changed before (possibly) starting the transfer.)

> I thought rsync worked, if the file's size and modification date doesn't
> match, by creating a binary tree and then checksumming the parts between
> every node, recursively to the root of the tree, and then only transferring
> the parts where the checksum didn't match.

There are no b-trees involved -- rsync immediately starts to send checksum info
from the receiving side to the sender, who then diffs the remote checksums with
the sending-side file and sends instructions to the receiver on how to recreate
the file using as much of the local data as possible (this new file is built in
a separate temp-file unless the --inplace option was specified).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3358] rsync chokes on large files

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3358





--- Comment #8 from [EMAIL PROTECTED]  2006-01-02 11:42 MST ---
> What is weird about that?

You wrote in a previous comment when I asked why rsync is considering a file
for 30 minutes if it is not checksumming it: 

> Because it is transferring the file. 

To which I replied that there is no noticable network activity when rsync is in
this state. However, when it is finished with the 'consideration phase' the
network is saturated.

I think it is weird that transferring a 25 GB file doesn't generate any network
activity when rsync is in the 'consideration phase' but transferring the same
file when rsync is in another phase saturates the network.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3371] New: no way to limit disk IO bandwidth

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3371

   Summary: no way to limit disk IO bandwidth
   Product: rsync
   Version: 2.6.6
  Platform: x86
   URL: https://bugzilla.samba.org/show_bug.cgi?id=2800
OS/Version: Linux
Status: NEW
  Severity: enhancement
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Hi,

My problem is the opposite of bug #2800.
I have a large filesystem that I want to keep in sync,
but I dont want to thrash the disk each time I start rsync.
I dont mind if the checksum stage takes a long time,
I just want to be gentle with disk IO.
Can a --disk-bwlimit option be added?

Thanks,

-Truxton


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3382] New: hang in read() in exclude.test of testsuite

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3382

   Summary: hang in read() in exclude.test of testsuite
   Product: rsync
   Version: 2.6.6
  Platform: Alpha
OS/Version: OSF/1
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I just built rsync 2.6.6 on my Tru64 5.1B box, using the vendor C compiler. 
The build went fine and most of the testsuite passes (I'm running as myself, so
a couple of the tests skip since they require root privs), but the exclude.test
hangs.

Using truss, I see that rsync is stuck in a tight read loop, calling read on fd
0 and getting back 0 bytes.

I attached to rsync with a debugger and stopped execution, and got this:

(ladebug) attach 198075 rsync
Reading symbolic information ...done
Attached to process id 198075  

Interrupt (for process)

Stopping process localhost:198075 (rsync).
stopped at [ __read(...) 0x3ff800d67d8] 

Information:  An  type was presented during execution of the previous
command.  For complete type information on this symbol, recompilation of the
program will be necessary.  Consult the compiler man pages for details on
producing full symbol table information using the '-g' (and '-gall' for cxx)
flags.

(ladebug) where
>0  0x3ff800d67d8 in __read(...) in /usr/shlib/libc.so
#1  0x3ff80177b20 in __read_nc(...) in /usr/shlib/libc.so
#2  0x3ff800da6d4 in __filbuf(...) in /usr/shlib/libc.so
#3  0x120023574 in parse_filter_file(listp=0x14010, fname=Info: no
allocation applies for symbol fname at the current PC
, mflags=1024, xflags=1) "exclude.c":999
#4  0x120023140 in parse_rule(listp=0x14010, pattern=0x14000e3e7="",
mflags=0, xflags=0) "exclude.c":938
#5  0x120031140 in parse_arguments(argc=0x11fffbfd8, argv=0x11fffbfd0,
frommain=1) "options.c":749
#6  0x12002a878 in main(argc=9, argv=0x11fffc018) "main.c":1104
#7  0x1200187b8 in __start(...) in rsync

(ladebug) up
>1  0x3ff80177b20 in __read_nc(...) in /usr/shlib/libc.so
(ladebug) up
>2  0x3ff800da6d4 in __filbuf(...) in /usr/shlib/libc.so
(ladebug) up
>3  0x120023574 in parse_filter_file(listp=0x14010, fname=Info: no 
>allocation applies for symbol fname at the current PC
, mflags=1024, xflags=1) "exclude.c":999
999 if ((ch = getc(fp)) == EOF) {
(ladebug) list $curline - 10 : 20
989 exit_cleanup(RERR_FILEIO);
990 }
991 return;
992 }
993 dirbuf[dirbuf_len] = '\0';
994 
995 while (1) {
996 char *s = line;
997 int ch, overflow = 0;
998 while (1) {
>   999 if ((ch = getc(fp)) == EOF) {
   1000 if (ferror(fp) && errno == EINTR)
   1001 continue;
   1002 break;
   1003 }
   1004 if (word_split && isspace(ch))
   1005 break;
   1006 if (eol_nulls? !ch : (ch == '\n' || ch ==
'\r'))
   1007 break;
   1008 if (s < eob)
(ladebug) n
stopped at [void parse_filter_file(struct filter_list_struct*, const char*,
unsigned int, int):1000 0x1200235c4]
   1000 if (ferror(fp) && errno == EINTR)
(ladebug) print fp
0x3ffc0080050
(ladebug) print errno
4
(ladebug) print line
"- /mid/for/foo/extra"



I checked errno.h, and "4" is indeed EINTR.  I think that the problem may be
that clearerr() should be called before the continue -- otherwise there's
nothing that would ever clear out the error condition after it has happened
once.

I'm attaching a simple patch that fixes the problem.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3382] hang in read() in exclude.test of testsuite

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3382





--- Comment #1 from [EMAIL PROTECTED]  2006-01-06 14:46 MST ---
Created an attachment (id=1652)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1652&action=view)
clearerr() before trying getc() again, otherwise may loop indefinitely


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3382] hang in read() in exclude.test of testsuite

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3382


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||FIXED




--- Comment #2 from [EMAIL PROTECTED]  2006-01-06 15:20 MST ---
Thanks for the fix!  I've just checked this into CVS for the upcoming 2.6.7.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3392] New: fuzzy misbehaving if source is a file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3392

   Summary: fuzzy misbehaving if source is a file
   Product: rsync
   Version: 2.6.6
  Platform: All
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I run rsync 2.6.6 on both the server and the client and perform a download
rsync --fuzzy --other-options... rsync://some/url /local/directory
Both the remote and the local URLs are absolute paths.

If the remote URL is a directory and I perform recursive synchronization
(e.g. "-a") then the --fuzzy option works perfectly just as I expect it,
and it co-operates nicely with the --compare-dest or --copy-dest option.

However, if the remote URL is a single plain file, then --fuzzy misbehaves.
No matter what the remote or local path is, no matter if I specify a
--compare-dest or --copy-dest option or not, no matter what its value is,
these are all completely ignored, and the local file with the most similar name
is searched in the current directory of the rsync process. This is seen from
strace's output (only "." is opened as a directory), seen from "skipping
directory xyz" messages that mention the subdirectories of the current dir,
and seen from the fact that if I place the similar file here then rsync is
much faster (i.e. it finds it here).

Instead, rsync should look for similar filenames under the target directory
(the directory component of the local path given in the last argument), or
under the directories given by --compare-dest or --copy-dest.

(Also, I think that generally if only absolute paths are given to rsync, it
should do nothing with the current directory, it should be irrelevant what it
is.)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3392] fuzzy misbehaving if source is a file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3392





--- Comment #1 from [EMAIL PROTECTED]  2006-01-10 17:22 MST ---
This behavior is a consequence of the strange logic in get_local_name in
main.c.  If the destination path is given as a file, then rsync uses a "local
name" and accesses the destination file by its full path rather than first
changing to the containing directory of the destination file.

When I was writing my custom rsync, I found that the behavior of get_local_name
fouled up default ACL observance, so I rewrote and heavily commented
get_local_name.  The upshot is that my rsync changes to the containing
directory when receiving no matter what.  Please consider making the same
change in the official rsync.  This change may help the situation with --fuzzy
to some degree, but issues remain, such as whether the basename of the source
path or the destination path is used to search for a fuzzy basis file.

My custom rsync is available here:
http://mysite.verizon.net/hashproduct/myrsync/


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3392] fuzzy misbehaving if source is a file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3392





--- Comment #2 from [EMAIL PROTECTED]  2006-01-10 17:22 MST ---
Created an attachment (id=1661)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1661&action=view)
main.c from my custom rsync, showing rewritten get_local_name


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3392] fuzzy misbehaving if source is a file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3392


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||FIXED




--- Comment #3 from [EMAIL PROTECTED]  2006-01-15 00:27 MST ---
As Matt noted, the fuzzy option was expecting the current directory to be the
parent directory of the destination file, and this wasn't true for a single
file being copied to a new name.

I have checked in a fix that makes rsync always push_dir() into the destination
file's parent directory.  (Thanks for the attachment, Matt -- I used some of
your comments and the general logic from get_local_name(), though I rewrote
it.)

One other nice side-effect is that rsync gives a better error now if you copy a
single file to a /totally/bogus/path/name.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3392] fuzzy misbehaving if source is a file

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3392





--- Comment #4 from [EMAIL PROTECTED]  2006-01-15 08:19 MST ---
Nice.  It occurs to me that maybe the first call to do_stat should be changed
to link_stat(dest_path, &st, keep_dirlinks) in order to obey --keep-dirlinks
when finding the top-level target directory.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3304] on --delay-updates remove .~tmp~ recursively

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3304


[EMAIL PROTECTED] changed:

   What|Removed |Added

 CC||[EMAIL PROTECTED]




--- Comment #1 from [EMAIL PROTECTED]  2006-01-16 07:47 MST ---
and here an easy reproducible testcase:

mirror:/tmp/rsynctest# mkdir src dest
mirror:/tmp/rsynctest# echo foo > src/abc
mirror:/tmp/rsynctest# rsync -av --delay-updates --delete src/ dest/
building file list ... done
./
abc

sent 113 bytes  received 48 bytes  322.00 bytes/sec
total size is 4  speedup is 0.02
mirror:/tmp/rsynctest# mkdir dest/.~tmp~
mirror:/tmp/rsynctest# rsync -av --delay-updates --delete src/ dest/
building file list ... done
./

sent 63 bytes  received 26 bytes  178.00 bytes/sec
total size is 4  speedup is 0.04
mirror:/tmp/rsynctest# ls -la dest/
total 16
drwxr-xr-x  3 root root 4096 Jan 16 15:45 .
drwxr-xr-x  4 root root 4096 Jan 16 15:45 ..
drwxr-xr-x  2 root root 4096 Jan 16 15:46 .~tmp~
-rw-r--r--  1 root root4 Jan 16 15:45 abc
mirror:/tmp/rsynctest#


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3415] New: -R and --delete cause --delete to be ignored

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3415

   Summary: -R and --delete cause --delete to be ignored
   Product: rsync
   Version: 2.6.6
  Platform: Other
OS/Version: Windows 2000
Status: NEW
  Severity: major
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


this works:
rsync --delete --progress --force -va /dir1 rsync://[EMAIL PROTECTED]/backup
while this one does not delete files on the destination:
rsync --delete --progress --force -vaR /dir1 rsync://[EMAIL PROTECTED]/backup


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 2790] Add support for converting filenames into different encodings

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=2790





--- Comment #3 from [EMAIL PROTECTED]  2006-01-16 20:51 MST ---
I've been working on a filename-conversion solution that uses the iconv()
function.  After putting a bunch of thought into various designs, I think I
have a good solution that I have coded up as a patch for the latest CVS version
(also available in the latest "nightly" tar file).  You can grab the patch
here:

http://opencoder.net/iconv.diff

This is a fairly early version, so if anyone would like to help with the
testing, please be sure to be fairly cautious with it at first.

The patch doesn't have any changes to the configure script only because I
already checked those into CVS (since they were pretty small -- they just check
for things like iconv_open(), iconv.h, etc.).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3299] rsync: now replaces non-ASCII character with numerical values

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3299


[EMAIL PROTECTED] changed:

   What|Removed |Added

   Severity|major   |normal
 Status|NEW |ASSIGNED




--- Comment #1 from [EMAIL PROTECTED]  2006-01-16 21:04 MST ---
The changelog statement you cited is correct for locales that don't use
multibyte encodings (of which UTF-8 is not one).  For instance, rsync outputs
all the extended characters from ISO-8859-1 without any mangling.

I've been considering how best to add multibyte support to rsync, and I think
that I can leverage the way iconv() works to have it tell me if characters are
valid in the current locale.  A patch that does this (along with adding
filename conversion support) is here:

http://opencoder.net/iconv.diff

This is still a young patch, so be careful if you decide to give it a try.

The patch applies to the latest CVS source.  See the diff for build and usage
instructions.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3415] -R and --delete cause --delete to be ignored

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3415





--- Comment #1 from [EMAIL PROTECTED]  2006-01-17 12:20 MST ---
I can't duplicate this.  Is the version of rsync that the daemon is running
old?  Are the command-lines you cited modified at all from what failed?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3304] on --delay-updates remove .~tmp~ recursively

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3304


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |ASSIGNED




--- Comment #2 from [EMAIL PROTECTED]  2006-01-17 12:38 MST ---
It might not be safe for rsync to delete a non-empty .~tmp~ dir if multiple
rsync commands are running at the same time (hopefully updating different
files).  One could argue that two simultaneously running rsyncs should really
be using different partial-dir settings to avoid other potential failures (such
as an empty directory being removed by one rsync right before the other can
open a temp file in it).  I'll consider allowing this, but it might be better
to make this an option, or to only enable it if a custom --partial-dir name is
chosen.

As for your example, rsync does not remove the empty .~tmp~ dir because it
didn't use it during that run.  If rsync had updated a file in the destination
directory, it would have removed the dir (as long as it was empty).  This
behavior is certainly debatable, and might change in the future.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3415] -R and --delete cause --delete to be ignored

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3415





--- Comment #2 from [EMAIL PROTECTED]  2006-01-17 13:12 MST ---
here are some more details:
client:
[EMAIL PROTECTED]:/usr/local/sbin/ > rsync --version
rsync  version 2.6.6  protocol version 29
Copyright (C) 1996-2005 by Andrew Tridgell and others

Capabilities: 64-bit files, no socketpairs, hard links, symlinks, batchfiles,
  inplace, IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
[EMAIL PROTECTED]:/usr/local/sbin/ > uname -a
Linux peta 2.4.22-xfs #1 Sun Jun 12 21:17:17 PDT 2005 armv5b unknown unknown
GNU/Linux
[EMAIL PROTECTED]:/usr/local/sbin/ >



server:
terra:/home/backup-data/test# rsync --version
rsync  version 2.6.4  protocol version 29
Copyright (C) 1996-2005 by Andrew Tridgell and others

Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
  inplace, IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.
terra:/home/backup-data/test# uname -a
Linux terra 2.6.8-2-386 #1 Thu May 19 17:40:50 JST 2005 i686 GNU/Linux
terra:/home/backup-data/test#

script on client:

#!/bin/sh
# dz 2003 07 23
export RSYNC_PASSWORD=topsecrect
BACKUPTARGET="rsync://[EMAIL PROTECTED]/backup-data/test"
BACKUPCMD="rsync --delete -avR --progress"
$BACKUPCMD /public/data/test1/  $BACKUPTARGET

---
It was in the eary morning when I posted the bug, so set up a test again today,
but again the same problem:
1) rsync of some arbitrary data => all files are transfered to the server
2) remove some date on the client
3) rsync again => no files are transfered/deleted


I googled around and found some discussions about it.
The server is a x86-machine running debian3.1 (sarge)
The client is an arm NSLU2 running unslung4 


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3304] on --delay-updates remove .~tmp~ recursively

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3304





--- Comment #3 from [EMAIL PROTECTED]  2006-01-17 14:01 MST ---
Perfectly understandable. As for now a hint in the in the manpage (or at
runtime, when 'stale' .~tmp~ directories are encountered) should suffice, since
the current behaviour can lead to unexpected (and unexplainable) disk-space
"loss".

At least my (and probably others too) assumption of rsync was, that a "rsync
--delete src/ dest/" ensured that the two trees are identical, and
--delay-updates silently breaks this assumption.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3304] on --delay-updates remove .~tmp~ recursively

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3304





--- Comment #4 from [EMAIL PROTECTED]  2006-01-17 14:56 MST ---
Perhaps the special treatment of the .~tmp~ directory could be implemented by
placing a receiving-end protect filter on any files inside it.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] New: dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422

   Summary: dry run fails when encountering dangling symbolic link
   Product: rsync
   Version: 2.6.4
  Platform: All
OS/Version: Linux
Status: NEW
  Severity: normal
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Hi,

I use rsync for backup and once in a while somebody replaces a symbolic link
with the contents of the file. In this case, I find that dry-run will
report errors while rsync without -n flag works fine.
A test case can be generated like this:

mkdir old
cd old
mkdir data
ln -s data link
cd ..
mkdir new
mkdir new/link
rsync -n -arlpogt --delete new/ old/

The rsync output will be:
building file list ... done
deleting data/
deleting link
link/

Now when we delete data before, rsync fails:
rmdir old/data
rsync -n -arlpogt --delete new/ old/
building file list ... done
deleting link
rsync: opendir "/data1/ppe/tmp/old/link" failed: No such file or directory (2)
./
link/

Rsync fails with return code 23, however, if you remove the '-n' flag,
rsync is able to sync and does not report errors. I think this is an error,
because dry-run should just report that the symbolic link is updated, i.e.:

building file list ... done
./
link/

and return no error code. For me this is also a problem, because I build
scripts around rsync and I use dry-run for testing what will be done.
The script makes than a backup of the files which would be deleted. But
when dry-run fails, the script does not know whether its save to continue.
Best, Peter.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3415] -R and --delete cause --delete to be ignored

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3415


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||FIXED
Version|2.6.6   |2.6.4




--- Comment #3 from [EMAIL PROTECTED]  2006-01-18 16:52 MST ---
(In reply to comment #2)
> client: [...] rsync  version 2.6.6  protocol version 29
> server: [...] rsync  version 2.6.4  protocol version 29
> [...] $BACKUPCMD /public/data/test1/  $BACKUPTARGET

There are some very significant details there:  the server is 2.6.4, and 2.6.4
has a bug in it when someone uses --relative with a source path that ends in a
slash (there's a bug-fix mentioned in the OLDNEWS file under the 2.6.5 release
that mentiones this fix).  The simplest work-around is to drop the trailing
slash at the end of "/public/data/test1/".


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422





--- Comment #1 from [EMAIL PROTECTED]  2006-01-18 19:47 MST ---
I reproduced this bug with my custom rsync 2.6.6.matt.7.  According to verbose
output, delete_in_dir("link") seems to be getting called erroneously on the
receiver.  I'm trying to find the cause.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422





--- Comment #2 from [EMAIL PROTECTED]  2006-01-18 20:38 MST ---
Created an attachment (id=1702)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1702&action=view)
Makes receiving rsync process deletions only in directories

I found the problem.  The receiving rsync performs deletions by scanning the
file list from the sender for directories.  For each directory the sender sent,
the receiver looks for something at the same path in the destination.  If the
receiver finds something, it assumes that something is a corresponding
directory and processes deletions inside it; if the receiver finds nothing, it
just moves on.

That assumption is OK most of the time.  If one runs Peter's second test case
without --dry-run, rsync deletes the symlink "link" on the grounds that it is
about to be replaced by a directory; when it goes to process deletions in the
directory "link" that the sender named, it finds nothing there on the
destination.  But with --dry-run, "link" gets fake-deleted from the receiver. 
delete_in_dir("link") is called, sees something by the name of "link",
incorrectly assumes it is a directory, and tries to process deletions.

That no error is produced when the symlink is valid is immaterial.  When rsync
is completely in link-aware mode (--links and not --keep-dirlinks), it should
treat symlinks simply as files of a special kind that contain strings; it
should never issue a system call that follows a symlink.

I wrote a very simple patch that makes delete_in_dir try to delete only in
directories.  That fixes the problem, but I don't know if it's the right way to
fix the problem.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422





--- Comment #3 from [EMAIL PROTECTED]  2006-01-19 00:46 MST ---
(In reply to comment #2)

This sounds like a good solution. Thanks, I didn't expect a response so
quickly! Shall I test this out with the latest rsync? 
You already did some testing, but perhaps my test might catch some
side-effects.
I have about 3 Terrabytes of data to 'rsync' over night. For my test, I would
take the latest official source rpm and upgrade it to 2.6.6 plus your patch.
Or, is there a test suite for rsync?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422





--- Comment #4 from [EMAIL PROTECTED]  2006-01-19 06:57 MST ---
Yes, please do take a recent source RPM or source tarball and try the patch. 
If you use rpmbuild, you can get it to apply the patch automatically after
unpacking the source tarball that comes with the SRPM.  Put the patch in
SOURCES/ of your RPM-building arena, and name it in rsync.spec like this:
Patch0: delete-only-in-directories.diff
Then insert after the %setup line:
%patch0 -p0
There's a testsuite that you can run with "make test" in a built rsync source
tree, either one you make manually or the one rpmbuild makes in BUILD/.  That
will check that the patch doesn't break other features.  But this might be
putting the carriage before the horse, as I'm still hoping Wayne will weigh in
as to whether my patch is the right way to fix the problem.

I think a "dry-run" test case should be added that tests the dry run feature
comprehensively on -vv verbosity and makes sure it produces the same output as
a real run in a number of borderline cases, such as the one in this bug.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422





--- Comment #5 from [EMAIL PROTECTED]  2006-01-19 07:46 MST ---
(In reply to comment #4)

OK, I will do that (have done rpm's before), but it will take some time, as I'm
rather busy at the moment (~2 weeks). This way, Wayne also has some more time
to respond. This bug certainly is not a critical one, but should be fixed. 
Best, Peter.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||FIXED




--- Comment #6 from [EMAIL PROTECTED]  2006-01-19 12:03 MST ---
As Matt discovered, the bug is simply that when rsync is in a dry-run scenario,
it may not have deleted an in-the-way file/symlink/device by the time the
delete code decides to check if a sender-side directory exists on the receiving
side.

Thanks for the patch!  It's the right fix, and I've checked it into CVS for the
upcoming 2.6.7.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3422] dry run fails when encountering dangling symbolic link

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3422





--- Comment #7 from [EMAIL PROTECTED]  2006-01-19 12:43 MST ---
(In reply to comment #6)
Great, so I do not need to do the testing. Thanks to all
for the effort. 


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3304] on --delay-updates remove .~tmp~ recursively

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3304





--- Comment #5 from [EMAIL PROTECTED]  2006-01-19 13:15 MST ---
Surprise: according to the man page of 2.6.6, a receiving-end protect filter is
indeed placed on the partial dir, which is the same as the delay-updates dir. 
Never mind comment #4.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] New: Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430

   Summary: Error with ACL-patch and -x on mountpoint
   Product: rsync
   Version: 2.6.6
  Platform: Other
OS/Version: FreeBSD
Status: NEW
  Severity: minor
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Hi,

using rsync-2.6.6 with ACL patch on two freebsd systems (with dirvish) and the
options -x and --acls, we get the following error:

send_acl: sys_acl_get_file(mayerr/test, SMB_ACL_TYPE_ACCESS): Operation not
supported

and so on for 3 more cycles.

Rsync is right about the fact that this SMB share mount point doesn't support
acls, but I still wonder why it even tries to access it with the -x option.

As far as I can tell the backups still work, but having those errors all over
the logs isn't nice.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #1 from [EMAIL PROTECTED]  2006-01-20 10:06 MST ---
The issue is that, when rsync detects that a directory is on another
filesystem, it sends the directory itself but skips the contents.  See
send_directory in flist.c: it reads all the directory entries into the file
list, sends them all (possibly including ones on other filesystems), and then
calls send_if_directory on each one to recurse into those that is in fact on
the same filesystem.

I'm inclined to fix this by moving the test for being on the same filesystem
into the readdir loop of send_directory, but there seem to be a lot of
subtleties to rsync's recursion, so I don't know if this will work.  Thoughts?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |ASSIGNED




--- Comment #2 from [EMAIL PROTECTED]  2006-01-20 10:54 MST ---
(In reply to comment #1)
> I'm inclined to fix this by moving the test for being on the same
> filesystem into the readdir loop of send_directory

No, we copy the mount-point directory on purpose because we want it to be there
should the remote system need to mount their own filesystem at that point.

The right fix is to eliminate this ACL error altogether: rsync should never
complain about a source item not having ACL info; it should just copy the item
without ACL info.  (This is one of the things that must be fixed before the ACL
patch can make it into the released version of rsync.)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3362] Add option to normalize Unicode filenames

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3362


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||DUPLICATE




--- Comment #1 from [EMAIL PROTECTED]  2006-01-20 11:02 MST ---
See bug #2790 comment #3 for a link to a patch that provides charset
conversion.  I'd like feedback on how well this works.

*** This bug has been marked as a duplicate of 2790 ***


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 2790] Add support for converting filenames into different encodings

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=2790


[EMAIL PROTECTED] changed:

   What|Removed |Added

 CC||[EMAIL PROTECTED]




--- Comment #4 from [EMAIL PROTECTED]  2006-01-20 11:02 MST ---
*** Bug 3362 has been marked as a duplicate of this bug. ***


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #3 from [EMAIL PROTECTED]  2006-01-20 11:57 MST ---
(In reply to comment #2)
> No, we copy the mount-point directory on purpose because we want it to be 
> there
> should the remote system need to mount their own filesystem at that point.

True, in some cases it might be useful to copy the mount point, but the empty
folder on the destination might confuse someone about whether the filesystem
was excluded or empty or what.  I don't think it makes sense to copy the mount
point unless we know the true attributes of the mount point itself as opposed
to those overlaid on it by the mounted filesystem.

For example, on my computer, I set 000 permissions on directories intended to
be used as mount points, and then the permissions are overlaid with those of
the root of the mounted filesystem.  That way, it's easy to see whether the
filesystem is mounted, and mistakes like backing up the system to
/media/external-disk when the disk isn't mounted are avoided.  If rsync were to
copy those mount points, I would expect to see their underlying 000 permissions
on the destination, not the permissions of the top of the mounted filesystem.

> The right fix is to eliminate this ACL error altogether: rsync should never
> complain about a source item not having ACL info; it should just copy the item
> without ACL info.  (This is one of the things that must be fixed before the 
> ACL
> patch can make it into the released version of rsync.)

Yes, that should be done no matter how rsync treats mount points.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #4 from [EMAIL PROTECTED]  2006-01-20 13:02 MST ---
I know of no way to find the attributes of the underlying mount-point directory
when the mount is present.  And leaving out the mount-point dir is not an
improvement in my book.  I also wouldn't want to tweak the permissions of a
mount-point dir to an arbitrary value because the destination filesystem might
have a real directory there, and we wouldn't want to adversely affect its
permissions.

We could make this user-selectable by letting the user repeat the -x option to
choose to eliminate the mount-point dirs from the transfer.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #5 from [EMAIL PROTECTED]  2006-01-20 13:43 MST ---
(In reply to comment #4)
> We could make this user-selectable by letting the user repeat the -x option to
> choose to eliminate the mount-point dirs from the transfer.

In fact that was just what I was going to suggest.  When in doubt, add an
option!  :)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #6 from [EMAIL PROTECTED]  2006-01-20 15:03 MST ---
Created an attachment (id=1704)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1704&action=view)
Makes -x -x exclude mount points themselves

This patch should do it.  Very casual testing (rsync -n --exclude=/*/*/** /
test/ with varying amounts of -x) suggests that it works.

I noticed that, when --copy-links or --copy-unsafe-links is enabled, rsync will
follow cross-filesystem symlinks to nondirectories but not to directories. 
This behavior seems strange, but I didn't change it.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #7 from [EMAIL PROTECTED]  2006-01-21 09:16 MST ---
AFAIK, other tools that have a don't-traverse-file-systems option (find, du,
..) in the unix world ignore the mount point (as it logically belongs to the
fs you don't want to traverse to), that's why I was surprised rsync didn't. But
I guess I could live with a non-default option, too.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3430] Error with ACL-patch and -x on mountpoint

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3430





--- Comment #8 from [EMAIL PROTECTED]  2006-01-21 13:35 MST ---
Created an attachment (id=1705)
 --> (https://bugzilla.samba.org/attachment.cgi?id=1705&action=view)
Suggested changes to man page for --one-file-system

Wayne seems to have expanded the documentation of -x in the man page to take
into account -x -x and other issues, but I would like to suggest some further
changes that I hope will make the man page clearer.  At the very least, the old
section about -x should be deleted.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3432] New: rsync -azv --cvs-exclude forgets "LocalSettings.php"

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3432

   Summary: rsync -azv --cvs-exclude forgets "LocalSettings.php"
   Product: rsync
   Version: 2.6.6
  Platform: x86
   URL: http://pto.linux.dk/albackup.tgz
OS/Version: Linux
Status: NEW
  Severity: major
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


Download http://pto.linux.dk/albackup.tgz
# unpack with; 
$ tar xvzf albackup.tgz
# Use rsync 2.6.6 (or earlier versions); 
$ rsync -azv --cvs-exclude mediawiki-1.5.3 newdir
# Note that newdir/mediawiki-1.5.3/ does *not* contain LocalSettings.php
# This is an error if I read the default_cvsignore array of
rsync-2.6.6/exclude.c correctly.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3432] rsync -azv --cvs-exclude forgets "LocalSettings.php"

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3432





--- Comment #1 from [EMAIL PROTECTED]  2006-01-21 14:26 MST ---
--cvs-ignore ignores both that laundry list in exclude.c and any additional
files specified in per-directory .cvsignore files.  I downloaded the package
you linked and found that the MediaWiki people listed LocalSettings.php in
.cvsignore, so rsync's behavior is correct.  A "sufficiently empowered user"
should resolve this bug invalid--why does Bugzilla let me click on things I'm
not allowed to actually do?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3432] rsync -azv --cvs-exclude forgets "LocalSettings.php"

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3432


[EMAIL PROTECTED] changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution||INVALID




--- Comment #2 from [EMAIL PROTECTED]  2006-01-21 14:33 MST ---
Matt has the solution - the mediawiki guys has entered the missing files into
.cvsignore, which rsync correctly parses and thus ignores LocalSettings.php.
Nasty problem, but this is not an error in rsync!


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3444] New: Cygwin files trashed when they only differ in the case of the name

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3444

   Summary: Cygwin files trashed when they only differ in the case
of the name
   Product: rsync
   Version: 2.6.6
  Platform: x86
OS/Version: Windows XP
Status: NEW
  Severity: critical
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


When cygwin is involved on either end of the transmission, when the only
difference between a file on the local machine and a file on the remote machine
is the case of their file name and that the --delete option is provided, the
destination file is deleted and the source file is not copied.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3444] Cygwin files trashed when they only differ in the case of the name

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3444





--- Comment #1 from [EMAIL PROTECTED]  2006-01-25 16:38 MST ---
It's pretty clear that the trouble is the case insensitivity of the filesystem.
 In fact, I reproduced this behavior on a vfat partition mounted on Linux:
mkdir test1 test2
touch test1/Foo
touch --reference=test1/Foo test2/foo
rsync -avvvii --delete-after test1/ test2/

Note that statting foo, Foo, or FOO will find the destination file, but readdir
gives its name as foo.  When rsync decides to copy test1/Foo, it stats
test2/Foo and finds a matching file: it has nothing to do.  Then it scans
test2/ for things to delete and finds an extraneous file test2/foo, which it
deletes.  So --delete-after can cause data loss.

When --delete-before or --delete-during is used, rsync deletes the destination
file as unmatched and then copies the source file all over again; this is
inefficient, but it does not cause data loss and it brings case on the
destination into line with case on the source.

One way to fix this data loss problem is to detect the case insensitivity of
the destination filesystem and compare files case insensitively with the
entries of the file list when deciding whether to delete them.  But that's
really ugly.  Having rsync remember the i-node numbers of the files it creates
and skip deleting those files might work, but it's still a hack.  For the
moment, avoid --delete-after on filesystems that accept inexact filenames,
including case insensitive ones and ones that perform
translation/canonicalization of character encodings (Mac OS X?).


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3461] New: rsync is not atomic when temp-dir is on different device

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3461

   Summary: rsync is not atomic when temp-dir is on different device
   Product: rsync
   Version: 2.6.7
  Platform: Other
OS/Version: Linux
Status: NEW
  Severity: enhancement
  Priority: P3
 Component: core
AssignedTo: [EMAIL PROTECTED]
ReportedBy: [EMAIL PROTECTED]
 QAContact: [EMAIL PROTECTED]


I do have a problem when rsync'ing files when I specify --temp-dir on a
different device than the destination.

An example:

# pwd
/disk1
# rsync --temp-dir /disk2 remote::file.dat file.dat

When the transfer finishes, the temporary file on /disk2 is copied directly to
/disk1/file.dat resulting in /disk1/file.dat being truncated and gradually
filled.  When temp-dir is on the same device, the new file is created with a
temporary name and then renamed, so that at no point /disk1/file.dat is
garbled.

The problem could be alleviated by letting reboust_rename() in util.c
optionally copy the file to a temporary name on the target device, and then
rename it. 

This is actually quite a problem for us, we want the temporary file on a
different device for performance.  When it's on the same device, rsync creates
both heavy read and heavy write traffic to the same disk (which is otherwise
quite idle), resulting in poor performance (much poorer than what the extra
copy incurs.)


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3461] rsync is not atomic when temp-dir is on different device

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3461





--- Comment #1 from [EMAIL PROTECTED]  2006-01-28 14:19 MST ---
I don't see how using a --temp-dir on a different device could make the
transfer faster, if indeed it does.  At some point, all of the data of file.dat
must go from memory to disk 1.  Using a --temp-dir on a different device, with
or without your proposed atomicity fix, does not avoid this bottleneck; after
the additional round trip to and from disk 2, the data must be written to disk
1 as before.

This bug brings up an interesting point.  I wasn't aware that robust_rename
copied files between filesystems when necessary, and I don't think it should. 
Copying data into an existing destination file has the same potentially
undesirable characteristics as --inplace, not least of which is non-atomicity,
but the user is never warned.

If the temp dir is on a different filesystem from the eventual location of a
destination file, what should rsync do?  Ignoring the temp dir and using a
temporary file next to the destination could be hazardous because some users
use a secure temp dir to avoid races in an insecure destination directory. 
Using two temporary files, one in the temp dir and one next to the destination,
is strictly worse than using a single file next to the destination, unless the
disk performance is anomalous like yours.

The current behavior of copying into the existing destination file should only
be used if the user is aware of the consequences.  To specify that she is, she
could give --inplace in addition to --temp-dir=XXX; the sole effect of
--inplace would be to allow this behavior.  If --temp-dir is given without
--inplace and this situation arises, I think rsync should produce an error,
skip the file, and say "some files could not be transferred".


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

DO NOT REPLY [Bug 3461] rsync is not atomic when temp-dir is on different device

[bugzilla-daemon]

https://bugzilla.samba.org/show_bug.cgi?id=3461





--- Comment #2 from [EMAIL PROTECTED]  2006-01-28 16:45 MST ---
I was a bit unclear, it's not that it's just a different device, it's a
different physical disk.

For a dataset of about 45GB we see an rsync time of about 2 hrs. When we use a
temp-dir on another disk it drops to about 30 minutes. 

We have measured it and traced the performance problem down to the following:
When the temporary file is on the same disk as the target, rsync must read the
old file and the network, it also must write the temporary file, to the same
disk as it reads from. It turns out that both the read and write speed of the
disk drops substantially compared to separate disks. This is probably because
the disk head needs to move between reads and writes, it doesn't move
infinitely fast.  

With separate disks, there's a single-stream read from one disk and a
single-stream write to the other disk.  When copying afterwards, it's still
single-streams to each disk, and we get close to 40MB/s.  When we do
intertwined reads and writes to the same disk, we get far less, in some cases
only 5MB/s.  Even with the extra copy, the dual-disk solution runs faster, the
only problem being non-atomicity.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the QA contact for the bug, or are watching the QA contact.
-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html