Re: Race in svn_atomic_namespace__create

2012-11-06 Thread Stefan Sperling
On Tue, Nov 06, 2012 at 11:47:51AM +0100, Stefan Fuhrmann wrote: > I'd like to see all that solved and SHM being used > for membuffer - which has been designed with that > goal in mind. It's the robustness part that makes it > so much harder to do than I thought back then. Right. I see. Thanks for

Re: Race in svn_atomic_namespace__create

2012-11-06 Thread Stefan Fuhrmann
On Mon, Nov 5, 2012 at 3:16 PM, Stefan Sperling wrote: > On Mon, Nov 05, 2012 at 02:54:07PM +0100, Stefan Fuhrmann wrote: > > On Sun, Nov 4, 2012 at 10:40 AM, Stefan Sperling wrote: > > > I just came across something that reminded me of this thread. > > > It seems PostgreSQL is doing something q

Re: Race in svn_atomic_namespace__create

2012-11-05 Thread Stefan Sperling
On Mon, Nov 05, 2012 at 02:54:07PM +0100, Stefan Fuhrmann wrote: > On Sun, Nov 4, 2012 at 10:40 AM, Stefan Sperling wrote: > > I just came across something that reminded me of this thread. > > It seems PostgreSQL is doing something quite similar to what we > > want to do here: > > > > When the fi

Re: Race in svn_atomic_namespace__create

2012-11-05 Thread Stefan Fuhrmann
On Sun, Nov 4, 2012 at 10:40 AM, Stefan Sperling wrote: > On Wed, Oct 31, 2012 at 03:24:10PM +0100, Stefan Fuhrmann wrote: > > On Wed, Oct 31, 2012 at 2:54 PM, Philip Martin > > wrote: > > > > > Philip Martin writes: > > > > > > > Stefan Fuhrmann writes: > > > > > > > >> Excellent analysis, Phi

Re: Race in svn_atomic_namespace__create

2012-11-04 Thread Stefan Sperling
On Wed, Oct 31, 2012 at 03:24:10PM +0100, Stefan Fuhrmann wrote: > On Wed, Oct 31, 2012 at 2:54 PM, Philip Martin > wrote: > > > Philip Martin writes: > > > > > Stefan Fuhrmann writes: > > > > > >> Excellent analysis, Philip! With r1404112, we use "plain" > > >> APR mmap code with almost no codi

Re: Race in svn_atomic_namespace__create

2012-10-31 Thread Stefan Fuhrmann
On Wed, Oct 31, 2012 at 2:54 PM, Philip Martin wrote: > Philip Martin writes: > > > Stefan Fuhrmann writes: > > > >> Excellent analysis, Philip! With r1404112, we use "plain" > >> APR mmap code with almost no coding overhead. > >> The only downside is that we now have a temporary > >> file sitti

Re: Race in svn_atomic_namespace__create

2012-10-31 Thread Philip Martin
Philip Martin writes: > Stefan Fuhrmann writes: > >> Excellent analysis, Philip! With r1404112, we use "plain" >> APR mmap code with almost no coding overhead. >> The only downside is that we now have a temporary >> file sitting in the db folder. > > Error handling needs attention: > > $ svnadmi

Re: Race in svn_atomic_namespace__create

2012-10-31 Thread Philip Martin
Stefan Fuhrmann writes: > Excellent analysis, Philip! With r1404112, we use "plain" > APR mmap code with almost no coding overhead. > The only downside is that we now have a temporary > file sitting in the db folder. Error handling needs attention: $ svnadmin create repo $ svnadmin dump repo >

Re: Race in svn_atomic_namespace__create

2012-10-31 Thread Stefan Fuhrmann
On Tue, Oct 30, 2012 at 7:18 PM, Stefan Sperling wrote: > On Tue, Oct 30, 2012 at 05:58:20PM +, Philip Martin wrote: > > Another approach would be to create the shared memory created from some > > other, long-lived, process. The user would have to run this process to > > enable caching. To

Re: Race in svn_atomic_namespace__create

2012-10-31 Thread Stefan Fuhrmann
On Tue, Oct 30, 2012 at 6:58 PM, Philip Martin wrote: > Philip Martin writes: > > > That's a serious problem, the two processes are not longer using the > > same shared memory segment to keep in sync. Changes made by one process > > won't be visible to another. > > I don't see how to fix this wi

Re: Race in svn_atomic_namespace__create

2012-10-30 Thread Stefan Sperling
On Tue, Oct 30, 2012 at 05:58:20PM +, Philip Martin wrote: > Another approach would be to create the shared memory created from some > other, long-lived, process. The user would have to run this process to > enable caching. To handle a large number of repositories this would > probably have t

Re: Race in svn_atomic_namespace__create

2012-10-30 Thread Philip Martin
Philip Martin writes: > That's a serious problem, the two processes are not longer using the > same shared memory segment to keep in sync. Changes made by one process > won't be visible to another. I don't see how to fix this with the current APR code. The process that creates the named shared

Re: Race in svn_atomic_namespace__create

2012-10-30 Thread Philip Martin
Stefan Fuhrmann writes: > Maybe, there should be a regression test that > tries concurrent initialization. There are two objects here: the named file and the shared memory segment. The usual sequence for a single process is: create file, create segment, delete segment, delete file. There are t

Re: Race in svn_atomic_namespace__create

2012-10-30 Thread Stefan Fuhrmann
On Tue, Oct 30, 2012 at 12:11 AM, Philip Martin wrote: > Stefan Fuhrmann writes: > > > On Mon, Oct 29, 2012 at 10:46 PM, Philip Martin > > wrote: > > > >> Philip Martin writes: > >> > >> > Philip Martin writes: > >> > > >> >> I can't see any order in which we can do attach/create that doesn't >

Re: Race in svn_atomic_namespace__create

2012-10-29 Thread Philip Martin
Philip Martin writes: > Stefan Fuhrmann writes: > >> The creation code is protected by a repo-global lock/unlock pair. >> So, in theory, there should be no race condition. > > Which lock and where? Does this lock out other processes? I see it's db/rev-prop-atomicsMutex >> It's on my TODO list

Re: Race in svn_atomic_namespace__create

2012-10-29 Thread Philip Martin
Stefan Fuhrmann writes: > On Mon, Oct 29, 2012 at 10:46 PM, Philip Martin > wrote: > >> Philip Martin writes: >> >> > Philip Martin writes: >> > >> >> I can't see any order in which we can do attach/create that doesn't have >> >> a similar race. I think the best solution is a short loop trying

Re: Race in svn_atomic_namespace__create

2012-10-29 Thread Stefan Fuhrmann
On Mon, Oct 29, 2012 at 10:46 PM, Philip Martin wrote: > Philip Martin writes: > > > Philip Martin writes: > > > >> I can't see any order in which we can do attach/create that doesn't have > >> a similar race. I think the best solution is a short loop trying > >> attach-create a few times befor

Re: Race in svn_atomic_namespace__create

2012-10-29 Thread Philip Martin
Philip Martin writes: > Philip Martin writes: > >> I can't see any order in which we can do attach/create that doesn't have >> a similar race. I think the best solution is a short loop trying >> attach-create a few times before giving up. > > I've committed a loop in r1403463. That doesn't fix

Re: Race in svn_atomic_namespace__create

2012-10-29 Thread Philip Martin
Philip Martin writes: > I can't see any order in which we can do attach/create that doesn't have > a similar race. I think the best solution is a short loop trying > attach-create a few times before giving up. I've committed a loop in r1403463. That doesn't fix the race but it is now very unli