On Wed, Dec 21, 2011 at 4:17 PM, Simon Riggs <si...@2ndquadrant.com> wrote: > Partitioning will give us more buffers and more LWlocks, to spread the > contention when we access the buffers. I use that word because its > what we call the technique already used in the buffer manager and lock > manager. If you wish to call this "less than fully-associative" I > really don't mind, as long as we're discussing the same overall > concept, so we can then focus on an implementation of that concept, > which no doubt has many ways of doing it. > > More buffers per lock does reduce the lock contention somewhat, but > not by much. So for me, it seems essential that we have more LWlocks > to solve the problem, which is where partitioning comes in. > > My perspective is that there is clog contention in many places, not > just in the ones you identified.
Well, that's possible. The locking in slru.c is pretty screwy and could probably benefit from better locking granularity. One point worth noting is that the control lock for each SLRU protects all the SLRU buffer mappings and the contents of all the buffers; in the main buffer manager, those responsibilities are split across BufFreelistLock, 16 buffer manager partition locks, one content lock per buffer, and the buffer header spinlocks. (The SLRU per-buffer locks are the equivalent of the I/O-in-progresss locks, not the content locks.) So splitting up CLOG into multiple SLRUs might not be the only way of improving the lock granularity; the current situation is almost comical. But on the flip side, I feel like your discussion of the problems is a bit hand-wavy. I think we need some real test cases that we can look at and measure, not just an informal description of what we think is happening. I'm sure, for example, that repeatedly reading different CLOG pages costs something - but I'm not sure that it's enough to have a material impact on performance. And if it doesn't, then we'd be better off leaving it alone and working on things that do. And if it does, then we need a way to assess how successful any given approach is in addressing that problem, so we can decide which of various proposed approaches is best. > * We allocate a new clog page every 32k xids. At the rates you have > now measured, we will do this every 1-2 seconds. And a new pg_subtrans page quite a bit more frequently than that. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
