Re: [HACKERS] Lock partitions

2006-10-18 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Tom Lane wrote: Hmm, what sort of errors are we talking about? ERROR: too many LWLocks taken That really shouldn't happen ... are you sure you did a full recompile after changing NUM_LOCK_PARTITIONS? Actually ... wait a moment. The d

Re: [HACKERS] Lock partitions

2006-10-18 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Hmm, what sort of errors are we talking about? > ERROR: too many LWLocks taken That really shouldn't happen ... are you sure you did a full recompile after changing NUM_LOCK_PARTITIONS? Actually ... wait a moment. The default value of

Re: [HACKERS] Lock partitions

2006-10-18 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: The number of transaction errors increased when I increased the NUM_LOCK_PARTITIONS, which I think is the reason it failed to run when I set it to 16. Hmm, what sort of errors are we talking about? I wonder if you've exposed a bug. Chang

Re: [HACKERS] Lock partitions

2006-10-18 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > The number of transaction errors increased when I increased the > NUM_LOCK_PARTITIONS, which I think is the reason it failed to run when I > set it to 16. Hmm, what sort of errors are we talking about? I wonder if you've exposed a bug. Changing NUM_LOCK_

Re: [HACKERS] Lock partitions

2006-10-18 Thread Mark Wong
Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: I see this in the CVS commits for 8.2. Did we determine the proper number of lock partitions? Should it be based on the number of buffers or concurrent sessions allowed? No. NUM_LOCK_PARTITIONS needs to be a compile-time constant for

Re: [HACKERS] Lock partitions

2006-09-20 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: I did a gross test and my kit appears broken between the 8.0 and 8.1 releases. I'll try to narrow down the exact date. I've narrowed it down between cvs pulls from Dec 14, 2005 and Dec 15, 2005. Does the attached diff appear to be a plau

Re: [HACKERS] Lock partitions

2006-09-20 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: >> I did a gross test and my kit appears broken between the 8.0 and 8.1 >> releases. I'll try to narrow down the exact date. > I've narrowed it down between cvs pulls from Dec 14, 2005 and Dec 15, > 2005. Does the attached diff appear to be a plausible cau

Re: [HACKERS] Lock partitions

2006-09-20 Thread Mark Wong
Mark Wong wrote: Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Curious, I'm still seeing the same behavior. Maybe I'll take another snapshot from CVS. Hm, maybe I need to try a bit harder here. Does the "not registered" error happen immediately/reliably for you, or do you need to ru

Re: [HACKERS] Lock partitions

2006-09-19 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Curious, I'm still seeing the same behavior. Maybe I'll take another snapshot from CVS. Hm, maybe I need to try a bit harder here. Does the "not registered" error happen immediately/reliably for you, or do you need to run the test awhile?

Re: [HACKERS] Lock partitions

2006-09-14 Thread markw
> Mark Wong <[EMAIL PROTECTED]> writes: >> Curious, I'm still seeing the same behavior. Maybe I'll take another >> snapshot from CVS. > > Hm, maybe I need to try a bit harder here. Does the "not registered" > error happen immediately/reliably for you, or do you need to run the > test awhile? It

Re: [HACKERS] Lock partitions

2006-09-14 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Curious, I'm still seeing the same behavior. Maybe I'll take another > snapshot from CVS. Hm, maybe I need to try a bit harder here. Does the "not registered" error happen immediately/reliably for you, or do you need to run the test awhile? > As for the

Re: [HACKERS] Lock partitions

2006-09-14 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Tom Lane wrote: With that change, I didn't see run_workload report any errors, but maybe I don't know where to look. The error is captured in dbt2/scripts/output/*/client/error.log, where * is the run directory. Hm ... here's what I see

Re: [HACKERS] Lock partitions

2006-09-14 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> With that change, I didn't see run_workload report any errors, but maybe >> I don't know where to look. > The error is captured in dbt2/scripts/output/*/client/error.log, where * > is the run directory. Hm ... here's what I see in there:

Re: [HACKERS] Lock partitions

2006-09-14 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Oops! 'autoreconf --install' is what I run to generate all that stuff. Ah, better. I see at least part of the problem: CREATE OR REPLACE FUNCTION stock_level (INTEGER, INTEGER, INTEGER) RETURNS INTEGER AS '/home/tgl/dbt2/storedproc/pgs

Re: [HACKERS] Lock partitions

2006-09-14 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Oops! 'autoreconf --install' is what I run to generate all that stuff. Ah, better. I see at least part of the problem: CREATE OR REPLACE FUNCTION stock_level (INTEGER, INTEGER, INTEGER) RETURNS INTEGER AS '/home/tgl/dbt2/storedproc/pgsql/c/../../../sto

Re: [HACKERS] Lock partitions

2006-09-14 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: But perhaps something much easier, using subversion: mkdir /mnt/dbt2 # for pgdata svn co https://svn.sourceforge.net/svnroot/osdldbt/trunk/dbt2 dbt2 cd dbt2 ./configure --with-postgresql= configure is not in the svn checkout. I guessed

Re: [HACKERS] Lock partitions

2006-09-14 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > But perhaps something much easier, using subversion: > mkdir /mnt/dbt2 # for pgdata > svn co https://svn.sourceforge.net/svnroot/osdldbt/trunk/dbt2 dbt2 > cd dbt2 > ./configure --with-postgresql= configure is not in the svn checkout. I guessed that I neede

Re: [HACKERS] Lock partitions

2006-09-14 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Tom Lane wrote: This is a server-side failure --- could we see how order_status() is defined? What PG version are you testing exactly? I took pgsqsl snapshot from cvs on Sept 11. Due to the length of the file that order_status() is in a

Re: [HACKERS] Lock partitions

2006-09-14 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> This is a server-side failure --- could we see how order_status() >> is defined? What PG version are you testing exactly? > I took pgsqsl snapshot from cvs on Sept 11. Due to the length of the > file that order_status() is in and of ord

Re: [HACKERS] Lock partitions

2006-09-14 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Sorry for the delay but looks like there's some data coming in. It also looks like my kit is starting to be a little dated. My stored libpq calls are failing. I'm getting this message: ERROR: record type has not been registered This

Re: [HACKERS] Lock partitions

2006-09-14 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Sorry for the delay but looks like there's some data coming in. It also > looks like my kit is starting to be a little dated. My stored libpq > calls are failing. I'm getting this message: > ERROR: record type has not been registered This is a server-

Re: [HACKERS] Lock partitions

2006-09-14 Thread Mark Wong
Tom Lane wrote: Mark Wong <[EMAIL PROTECTED]> writes: Tom Lane wrote: It would be nice to see some results from the OSDL tests with, say, 4, 8, and 16 lock partitions before we forget about the point though. Anybody know whether OSDL is in a position to run tests for us? Yeah, I can run some

Re: [HACKERS] Lock partitions

2006-09-14 Thread Strong, David
: Strong, David Cc: PostgreSQL-development Subject: Re: [HACKERS] Lock partitions "Strong, David" <[EMAIL PROTECTED]> writes: > We have some results for you. We left the buffer partition locks at 128 > as this did not seem to be a concern and we're still using 25 backend > proc

Re: [HACKERS] Lock partitions

2006-09-13 Thread Tom Lane
Jim Nasby <[EMAIL PROTECTED]> writes: > Isn't having more lock partitions than buffer partitions pointless? AFAIK they're pretty orthogonal. It's true though that a typical transaction doesn't hold all that many locks, which is why I don't see a need for a large number of lock partitions.

Re: [HACKERS] Lock partitions

2006-09-13 Thread Jim Nasby
On Sep 13, 2006, at 2:46 PM, Strong, David wrote: We have some results for you. We left the buffer partition locks at 128 as this did not seem to be a concern and we're still using 25 backend processes. We ran tests for 4, 8 and 16 lock partitions. Isn't having more lock partitions than buffe

Re: [HACKERS] Lock partitions

2006-09-13 Thread Tom Lane
"Strong, David" <[EMAIL PROTECTED]> writes: > We have some results for you. We left the buffer partition locks at 128 > as this did not seem to be a concern and we're still using 25 backend > processes. We ran tests for 4, 8 and 16 lock partitions. > For 4 lock partitions, it took 620 seconds to

Re: [HACKERS] Lock partitions

2006-09-13 Thread Strong, David
ssage- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Strong, David Sent: Wednesday, September 13, 2006 10:52 AM To: PostgreSQL-development Subject: Re: [HACKERS] Lock partitions Simon, In the 16/16 (16 buffer partitions/16 lock partitions) test, the WALInsertLock lock had 146

Re: [HACKERS] Lock partitions

2006-09-13 Thread Strong, David
2:22 AM To: Tom Lane Cc: Strong, David; PostgreSQL-development Subject: Re: [HACKERS] Lock partitions On Tue, 2006-09-12 at 12:40 -0400, Tom Lane wrote: > "Strong, David" <[EMAIL PROTECTED]> writes: > > When using 16 buffer and 16 lock partitions, we see that BufMapping &g

Re: [HACKERS] Lock partitions

2006-09-13 Thread Simon Riggs
On Tue, 2006-09-12 at 12:40 -0400, Tom Lane wrote: > "Strong, David" <[EMAIL PROTECTED]> writes: > > When using 16 buffer and 16 lock partitions, we see that BufMapping > > takes 809 seconds to acquire locks and 174 seconds to release locks. The > > LockMgr takes 362 seconds to acquire locks and 26

Re: [HACKERS] Lock partitions

2006-09-12 Thread Strong, David
ve not tried to remove the SInvalLock lock to see where time flows to next, but we might. David -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 12, 2006 9:40 AM To: Strong, David Cc: PostgreSQL-development Subject: Re: [HACKERS] Lock partitions "St

Re: [HACKERS] Lock partitions

2006-09-12 Thread Tom Lane
"Strong, David" <[EMAIL PROTECTED]> writes: > When using 16 buffer and 16 lock partitions, we see that BufMapping > takes 809 seconds to acquire locks and 174 seconds to release locks. The > LockMgr takes 362 seconds to acquire locks and 26 seconds to release > locks. > When using 128 buffer and 1

Re: [HACKERS] Lock partitions

2006-09-12 Thread Strong, David
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Simon Riggs Sent: Tuesday, September 12, 2006 1:37 AM To: Tom Lane Cc: Mark Wong; Bruce Momjian; PostgreSQL-development Subject: Re: [HACKERS] Lock partitions On Mon, 2006-09-11 at 11:29 -0400, Tom Lane wrote: > Mark Wong <[EMAI

Re: [HACKERS] Lock partitions

2006-09-12 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Mon, 2006-09-11 at 11:29 -0400, Tom Lane wrote: >> Great, thanks. The thing to twiddle is LOG2_NUM_LOCK_PARTITIONS in >> src/include/storage/lwlock.h. You need a full backend recompile >> after changing it, but you shouldn't need to initdb, if that hel

Re: [HACKERS] Lock partitions

2006-09-12 Thread Simon Riggs
On Mon, 2006-09-11 at 11:29 -0400, Tom Lane wrote: > Mark Wong <[EMAIL PROTECTED]> writes: > > Tom Lane wrote: > >> It would be nice to see some results from the OSDL tests with, say, 4, > >> 8, and 16 lock partitions before we forget about the point though. > >> Anybody know whether OSDL is in a p

Re: [HACKERS] Lock partitions

2006-09-11 Thread Mark Wong
Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: I see this in the CVS commits for 8.2. Did we determine the proper number of lock partitions? Should it be based on the number of buffers or concurrent sessions allowed? No. NUM_LOCK_PARTITIONS needs to be a compile-time constant for

Re: [HACKERS] Lock partitions

2006-09-11 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> It would be nice to see some results from the OSDL tests with, say, 4, >> 8, and 16 lock partitions before we forget about the point though. >> Anybody know whether OSDL is in a position to run tests for us? > Yeah, I can run some dbt2 tes

Re: [HACKERS] Lock partitions

2006-09-10 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > I see this in the CVS commits for 8.2. Did we determine the proper > number of lock partitions? Should it be based on the number of buffers > or concurrent sessions allowed? No. NUM_LOCK_PARTITIONS needs to be a compile-time constant for a number of r

[HACKERS] Lock partitions

2006-09-10 Thread Bruce Momjian
I see this in the CVS commits for 8.2. Did we determine the proper number of lock partitions? Should it be based on the number of buffers or concurrent sessions allowed? Divide the lock manager's shared state into 'partitions', so as to reduce contention for the former single LockMgrLock. Per