Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-19 Thread Craig Barratt
Alberto Accomazzi writes: > What I'm referring to are those options that a client passes to the > server which influence file selection, checksum and block generation. I > haven't looked at the rsync source code in quite a while, but off the > top of my head here are the issues to look at when

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-19 Thread Alberto Accomazzi
Chris Shoemaker wrote: There's no doubt that caching the file list on the server side would indeed be a very useful feature for all those who use rsyncd as a distribution method. We all know how difficult it can be to reliably rsync a large directory tree because of the memory and I/O costs in

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Chris Shoemaker
On Tue, May 18, 2004 at 11:11:51AM -0400, Alberto Accomazzi wrote: > > Wayne Davison wrote: > > > >I'm wondering if batch mode should be removed from the main rsync > >release and relegated to a parallel project? It seems to me that a > >better feature for the mainstream utility would be somet

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Donovan Baarda
On Wed, 2004-05-19 at 06:10, Chris Shoemaker wrote: > On Tue, May 18, 2004 at 10:06:52AM -0400, Alberto Accomazzi wrote: > > Chris Shoemaker wrote: [...] > > that the feature is useless, but just caution people that they need to > > understand the assumptions that this use of rsync is based upon.

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Chris Shoemaker
On Tue, May 18, 2004 at 10:06:52AM -0400, Alberto Accomazzi wrote: > Chris Shoemaker wrote: > > > Indeed, what you describe seems to have been the design motivation. > > I > >can share what my desired application is: I want to create a mirror of a > >public server onto my local machine w

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Chris Shoemaker
On Mon, May 17, 2004 at 09:42:18PM -0700, Jos Backus wrote: > On Mon, May 17, 2004 at 05:18:10PM -0400, Chris Shoemaker wrote: > > BTW, there is a work-around. If you don't mind duplicating the mirror > > twice, one solution is to do a regular (no --write-batch) rsync update of one > > copy of

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Chris Shoemaker
On Mon, May 17, 2004 at 08:10:57PM -0700, Wayne Davison wrote: > On Mon, May 17, 2004 at 05:18:10PM -0400, Chris Shoemaker wrote: > > The "knowledge" or "memory" of that exact state is more likely to > > reside with the receiver (who just left that state) than with the > > sender (who may never hav

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Alberto Accomazzi
Wayne Davison wrote: The "knowledge" or "memory" of that exact state is more likely to reside with the receiver (who just left that state) than with the sender (who may never have been in that state). Therefore it is more likely to be useful to the receiver than to sender. This is only true if y

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-18 Thread Alberto Accomazzi
Chris Shoemaker wrote: Indeed, what you describe seems to have been the design motivation. I can share what my desired application is: I want to create a mirror of a public server onto my local machine which physically disconnected from the Internet, and keep it current. So, I intend to first rs

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-17 Thread Jos Backus
On Mon, May 17, 2004 at 05:18:10PM -0400, Chris Shoemaker wrote: > BTW, there is a work-around. If you don't mind duplicating the mirror > twice, one solution is to do a regular (no --write-batch) rsync update of one > copy of the mirror, and then do the --write-batch during a local to local

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-17 Thread Wayne Davison
On Mon, May 17, 2004 at 05:18:10PM -0400, Chris Shoemaker wrote: > The "knowledge" or "memory" of that exact state is more likely to > reside with the receiver (who just left that state) than with the > sender (who may never have been in that state). Therefore it is more > likely to be useful to t

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-17 Thread Chris Shoemaker
On Mon, May 17, 2004 at 10:15:23AM -0400, Alberto Accomazzi wrote: > > Chris, > > to put things in the right prespective, you should read (if you haven't > done so already) the original paper describing the design behind batch > mode. The design and implementation of this functionality goes ba

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-17 Thread Alberto Accomazzi
Chris, to put things in the right prespective, you should read (if you haven't done so already) the original paper describing the design behind batch mode. The design and implementation of this functionality goes back to a project called the Internet2 Distributed Storage Infrastructure (I2-DSI

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-16 Thread Chris Shoemaker
On Sat, May 15, 2004 at 02:20:50PM -0700, Wayne Davison wrote: > On Sat, May 15, 2004 at 11:53:05AM -0400, Chris Shoemaker wrote: > > Not only does the client pass the "--write-batch=pfx" argument to the > > server, but it's actually the server calling all the batch write > > routines. > > This is

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-15 Thread Wayne Davison
On Sat, May 15, 2004 at 11:53:05AM -0400, Chris Shoemaker wrote: > Not only does the client pass the "--write-batch=pfx" argument to the > server, but it's actually the server calling all the batch write > routines. This isn't client/server differentiated, the current code appears to have been wri

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-15 Thread Chris Shoemaker
On Sun, May 09, 2004 at 08:50:07AM -0700, Wayne Davison wrote: > On Fri, May 07, 2004 at 06:54:32PM -0400, Chris Shoemaker wrote: > > /* if (!write_batch) */ > >send_exclude_list(f_out); > > > > at main.c:641. > > > > This seems to work better, because things get further. > > I looked at the

Re: batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-09 Thread Wayne Davison
On Fri, May 07, 2004 at 06:54:32PM -0400, Chris Shoemaker wrote: > /* if (!write_batch) */ >send_exclude_list(f_out); > > at main.c:641. > > This seems to work better, because things get further. I looked at the receiving side, and I think that code should probably be "if (!read_batch)" inst

batch-mode fixes [was: [PATCH] fix read-batch SEGFAULT]

2004-05-07 Thread Chris Shoemaker
On Fri, May 07, 2004 at 02:29:12AM -0400, Chris Shoemaker wrote: > > Status update: I've just finished tracing through the client behavior > > related to the apparent hang. All seems to go well with client/server > > negotiation, until the client tries to receive the file list (calling > > recv_fi