Re: [HACKERS] FlexLocks

2011-12-02 Thread Kevin Grittner
"Kevin Grittner" wrote: > Robert Haas wrote: >> Kevin Grittner wrote: >>> The extraWaits code still looks like black magic to me >> [explanation of the extraWaits behavior] > > Thanks. I'll spend some time reviewing this part. There is some > rearrangement of related code, and this should

Re: [HACKERS] FlexLocks

2011-12-01 Thread Robert Haas
On Wed, Nov 30, 2011 at 7:01 PM, Kevin Grittner wrote: > "Kevin Grittner" wrote: > >> OK.  There are a few things I found in this pass which missed in the >> last.  One contrib module was missed, I found another typo in a >> comment, and I think we can reduce the include files a bit.  Rather >> t

Re: [HACKERS] FlexLocks

2011-11-30 Thread Kevin Grittner
"Kevin Grittner" wrote: > OK. There are a few things I found in this pass which missed in the > last. One contrib module was missed, I found another typo in a > comment, and I think we can reduce the include files a bit. Rather > than describe it, I'm attaching a patch file over the top of yo

Re: [HACKERS] FlexLocks

2011-11-30 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> Why is it OK to drop these lines from the else condition in >> ProcArrayEndTransaction()?: >> >>/* must be cleared with xid/xmin: */ >>proc->vacuumFlags &= ~PROC_VACUUM_STATE_MASK; > > It's probably not. Oops. OK. I see that's bac

Re: [HACKERS] FlexLocks

2011-11-23 Thread Kevin Grittner
"Kevin Grittner" wrote: > Robert Haas wrote: >> Updated patches attached. > > I have to admit I don't have my head around the extraWaits issue, > so I can't personally vouch for that code, although I have no > reason to doubt it, either. Everything else was something that I at > least *think*

Re: [HACKERS] FlexLocks

2011-11-22 Thread Kevin Grittner
Robert Haas wrote: > Updated patches attached. I've gotten through several days of performance tests for this pair of related patches, with results posted on a separate thread. I'll link those in to the CF application shortly. To summarize the other (fairly long) thread on benchmarks, it see

Re: [HACKERS] FlexLocks

2011-11-21 Thread Bruce Momjian
Robert Haas wrote: > On Wed, Nov 16, 2011 at 12:25 PM, Kevin Grittner > wrote: > >> We could alternatively change one or the other of them to be a > >> struct with one member, but I think the cure might be worse than > >> the disease. ?By my count, we are talking about saving perhaps as > >> many

Re: [HACKERS] FlexLocks

2011-11-18 Thread Pavan Deolasee
On Fri, Nov 18, 2011 at 10:29 PM, Robert Haas wrote: > > So the upside and downside of this approach is that it modifies the > existing LWLock implementation rather than allowing multiple lock > implementations to exist side-by-side.  That means every LWLock in the > system has access to this fun

Re: [HACKERS] FlexLocks

2011-11-18 Thread Robert Haas
On Fri, Nov 18, 2011 at 6:26 AM, Pavan Deolasee wrote: > My apologies for hijacking the thread, but the work seems quite > related, so I thought I should post here instead of starting a new > thread. > > Here is a WIP patch based on the idea of having a shared Q. A process > trying to access the s

Re: [HACKERS] FlexLocks

2011-11-18 Thread Pavan Deolasee
On Thu, Nov 17, 2011 at 10:19 AM, Pavan Deolasee wrote: > On Thu, Nov 17, 2011 at 10:01 AM, Robert Haas wrote: > >> >> I am not convinced that that's a better API.  I mean, consider >> something like this: >> >>    /* >>     * OK, let's do it.  First let other backends know I'm in ANALYZE. >>    

Re: [HACKERS] FlexLocks

2011-11-16 Thread Pavan Deolasee
On Thu, Nov 17, 2011 at 10:01 AM, Robert Haas wrote: > > I am not convinced that that's a better API.  I mean, consider > something like this: > >    /* >     * OK, let's do it.  First let other backends know I'm in ANALYZE. >     */ >    LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE); >    MyProc->v

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 11:16 PM, Pavan Deolasee wrote: > On Tue, Nov 15, 2011 at 7:20 PM, Robert Haas wrote: >> The lower layer I called "FlexLocks", >> and it's designed to allow a variety of locking implementations to be >> built on top of it and reuse as much of the basic infrastructure as I

Re: [HACKERS] FlexLocks

2011-11-16 Thread Pavan Deolasee
On Tue, Nov 15, 2011 at 7:20 PM, Robert Haas wrote: > The lower layer I called "FlexLocks", > and it's designed to allow a variety of locking implementations to be > built on top of it and reuse as much of the basic infrastructure as I > could figure out how to make reusable without hurting perfor

Re: [HACKERS] FlexLocks

2011-11-16 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> Is there any way to typedef our way out of it [?] > Well, if we just say: > > typedef FlexLockId LWLockId; > > ...that's about equivalent to the #define from the compiler's > point of view. Bummer -- I was hoping there was some equivalent to "

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 11:17 AM, Kevin Grittner wrote: > Robert Haas wrote: > >> Now maybe there is some better way to do this, but at the moment, >> I'm not seeing it.  If we call them all LWLocks, but only some of >> them support LWLockAcquire(), then that's going to be pretty >> weird. > > Is

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 11:14 AM, Greg Stark wrote: > On Wed, Nov 16, 2011 at 3:41 PM, Robert Haas wrote: >>  Now, you're always going to use >> LWLockAcquire() and LWLockRelease() to acquire and release an LWLock, >> but a FlexLockId isn't guaranteed to be an LWLockId - any given value >> might

Re: [HACKERS] FlexLocks

2011-11-16 Thread Kevin Grittner
Robert Haas wrote: > Now maybe there is some better way to do this, but at the moment, > I'm not seeing it. If we call them all LWLocks, but only some of > them support LWLockAcquire(), then that's going to be pretty > weird. Is there any way to typedef our way out of it, such that a LWLock

Re: [HACKERS] FlexLocks

2011-11-16 Thread Greg Stark
On Wed, Nov 16, 2011 at 3:41 PM, Robert Haas wrote: >  Now, you're always going to use > LWLockAcquire() and LWLockRelease() to acquire and release an LWLock, > but a FlexLockId isn't guaranteed to be an LWLockId - any given value > might also refer to a FlexLock of some other type.  If we let eve

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 10:51 AM, Tom Lane wrote: > Robert Haas writes: >> Well, it would certainly be easy enough to add those macros, and I'm >> not necessarily opposed to it, but I fear it could end up being a bit >> confusing in the long run.  If we adopt this infrastructure, then I >> expect

Re: [HACKERS] FlexLocks

2011-11-16 Thread Tom Lane
Robert Haas writes: > Well, it would certainly be easy enough to add those macros, and I'm > not necessarily opposed to it, but I fear it could end up being a bit > confusing in the long run. If we adopt this infrastructure, then I > expect knowledge of different types of FlexLocks to gradually >

Re: [HACKERS] FlexLocks

2011-11-16 Thread Tom Lane
"Kevin Grittner" writes: > Simon Riggs wrote: >> I just don't see the reason to do a global search and replace on >> the lwlock name > I was going to review further before commenting on that, but since > it has now come up -- it seems odd that a source file which uses > only LW locks needs to c

Re: [HACKERS] FlexLocks

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 10:26 AM, Kevin Grittner wrote: > For example, if these two macros were defined, predicate.c wouldn't > have needed any modifications, and I suspect that is true of many > other files (although possibly needing a few other macros): > > #define LWLockId FlexLockId > #define

Re: [HACKERS] FlexLocks

2011-11-16 Thread Kevin Grittner
Simon Riggs wrote: > I just don't see the reason to do a global search and replace on > the lwlock name I was going to review further before commenting on that, but since it has now come up -- it seems odd that a source file which uses only LW locks needs to change so much for the FlexLock imp

Re: [HACKERS] FlexLocks

2011-11-16 Thread Simon Riggs
On Tue, Nov 15, 2011 at 8:33 PM, Alvaro Herrera wrote: >> > I'm not really enthused by the idea of completely rewriting lwlocks >> > for this. Seems like specialised code is likely to be best, as well as >> > having less collateral damage. >> >> Well, the problem that I have with that is that we'

Re: [HACKERS] FlexLocks

2011-11-15 Thread Dan Ports
On Tue, Nov 15, 2011 at 10:55:49AM -0600, Kevin Grittner wrote: > And I would be > surprised if some creative thinking didn't yield a far better FL > scheme for SSI than we can manage with existing LW locks. One place I could see it being useful is for SerializableFinishedListLock, which protects

Re: [HACKERS] FlexLocks

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 3:47 PM, Kevin Grittner wrote: > Alvaro Herrera wrote: > >> As Kevin says nearby it's likely that we could find some way to >> rewrite the SLRU (clog and such) locking protocol using these new >> things too. > > Yeah, I really meant all SLRU, not just clog.  And having see

Re: [HACKERS] FlexLocks

2011-11-15 Thread Kevin Grittner
Alvaro Herrera wrote: > As Kevin says nearby it's likely that we could find some way to > rewrite the SLRU (clog and such) locking protocol using these new > things too. Yeah, I really meant all SLRU, not just clog. And having seen what Robert has done here, I'm kinda glad I haven't gotten ar

Re: [HACKERS] FlexLocks

2011-11-15 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar nov 15 17:16:31 -0300 2011: > On Tue, Nov 15, 2011 at 1:40 PM, Simon Riggs wrote: > > Which is the same locking avoidance technique we already use for sync > > rep and for the new group commit patch. > > Yep... > > > I've been saying for some time that

Re: [HACKERS] FlexLocks

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 1:40 PM, Simon Riggs wrote: > Which is the same locking avoidance technique we already use for sync > rep and for the new group commit patch. Yep... > I've been saying for some time that we should use the same technique > for ProcArray and clog also, so we only need to qu

Re: [HACKERS] FlexLocks

2011-11-15 Thread Simon Riggs
On Tue, Nov 15, 2011 at 1:50 PM, Robert Haas wrote: > It basically > works like a regular LWLock, except that it has a special operation to > optimize ProcArrayEndTransaction().  In the uncontended case, instead > of acquiring and releasing the lock, it just grabs the lock, observes > that there

Re: [HACKERS] FlexLocks

2011-11-15 Thread Kevin Grittner
Robert Haas wrote: > I'm not necessarily saying that any of these particular > things are what we want to do, just throwing out the idea that we > may want a variety of lock types that are similar to lightweight > locks but with subtly different behavior, yet with common > infrastructure for err