Re: [HACKERS] mosbench revisited

2011-08-15 Thread Robert Haas
On Mon, Aug 15, 2011 at 6:22 PM, Greg Stark wrote: > On Wed, Aug 3, 2011 at 7:21 PM, Robert Haas wrote: >>  I'm kind of interested by the >> result, actually, as I had feared that the spinlock protecting >> ProcArrayLock was going to be a bigger problem sooner. > > I think this depends on how man

Re: [HACKERS] mosbench revisited

2011-08-15 Thread Greg Stark
On Wed, Aug 3, 2011 at 7:21 PM, Robert Haas wrote: >  I'm kind of interested by the > result, actually, as I had feared that the spinlock protecting > ProcArrayLock was going to be a bigger problem sooner. I think this depends on how many connections you have. If you try to scale up your benchmar

Re: [HACKERS] mosbench revisited

2011-08-10 Thread Robert Haas
On Wed, Aug 10, 2011 at 9:46 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mié ago 10 21:27:08 -0400 2011: >> 2011/8/10 Dimitri Fontaine : >> > Robert Haas writes: >> >> However, it doesn't really do anything to solve this problem.  The >> >> problem here is not that the size

Re: [HACKERS] mosbench revisited

2011-08-10 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié ago 10 21:27:08 -0400 2011: > 2011/8/10 Dimitri Fontaine : > > Robert Haas writes: > >> However, it doesn't really do anything to solve this problem. The > >> problem here is not that the size of the relation is changing too > >> frequently - indeed, it'

Re: [HACKERS] mosbench revisited

2011-08-10 Thread Robert Haas
2011/8/10 Dimitri Fontaine : > Robert Haas writes: >> However, it doesn't really do anything to solve this problem. The >> problem here is not that the size of the relation is changing too >> frequently - indeed, it's not changing at all in this test case. The >> problem is rather that testing w

Re: [HACKERS] mosbench revisited

2011-08-10 Thread Dimitri Fontaine
Robert Haas writes: > However, it doesn't really do anything to solve this problem. The > problem here is not that the size of the relation is changing too > frequently - indeed, it's not changing at all in this test case. The > problem is rather that testing whether or not the size has in fact

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Jesper Krogh
On 2011-08-08 15:29, Robert Haas wrote: On Sat, Aug 6, 2011 at 2:16 PM, Dimitri Fontaine wrote: Robert Haas writes: It would be nice if the Linux guys would fix this problem for us, but I'm not sure whether they will. For those who may be curious, the problem is in generic_file_llseek() in f

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Robert Haas
On Sat, Aug 6, 2011 at 3:30 PM, Tom Lane wrote: > Jeff Janes writes: >> My experiments have shown that the freelist proper is not >> substantially faster than the freelist clocksweep--and that is even >> under the assumption that putting things back into the freelist is >> absolutely free. > > Th

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Robert Haas
On Sat, Aug 6, 2011 at 2:16 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> It would be nice if the Linux guys would fix this problem for us, but >> I'm not sure whether they will.  For those who may be curious, the >> problem is in generic_file_llseek() in fs/read-write.c.  On a platform >>

Re: [HACKERS] mosbench revisited

2011-08-08 Thread Robert Haas
On Sat, Aug 6, 2011 at 1:43 PM, Jeff Janes wrote: > The approach is to move the important things from a LWLock to a > spinlock, and to not do any locking for increments to clock-hand > increment and numBufferAllocs. > That means that some buffers might occasionally get inspected twice > and some m

Re: [HACKERS] mosbench revisited

2011-08-06 Thread Tom Lane
Jeff Janes writes: > My experiments have shown that the freelist proper is not > substantially faster than the freelist clocksweep--and that is even > under the assumption that putting things back into the freelist is > absolutely free. The freelist isn't there to make buffer allocation faster, t

Re: [HACKERS] mosbench revisited

2011-08-06 Thread Dimitri Fontaine
Robert Haas writes: > It would be nice if the Linux guys would fix this problem for us, but > I'm not sure whether they will. For those who may be curious, the > problem is in generic_file_llseek() in fs/read-write.c. On a platform > with 8-byte atomic reads, it seems like it ought to be very po

Re: [HACKERS] mosbench revisited

2011-08-06 Thread Jeff Janes
On Wed, Aug 3, 2011 at 3:21 PM, Jim Nasby wrote: > On Aug 3, 2011, at 1:21 PM, Robert Haas wrote: >> 1. "We configure PostgreSQL to use a 2 Gbyte application-level cache >> because PostgreSQL protects its free-list with a single lock and thus >> scales poorly with smaller caches."  This is a compl

Re: [HACKERS] mosbench revisited

2011-08-06 Thread Jeff Janes
On Wed, Aug 3, 2011 at 11:21 AM, Robert Haas wrote: > About nine months ago, we had a discussion of some benchmarking that > was done by the mosbench folks at MIT: > > http://archives.postgresql.org/pgsql-hackers/2010-10/msg00160.php > > Although the authors used PostgreSQL as a test harness for d

Re: [HACKERS] mosbench revisited

2011-08-04 Thread Robert Haas
On Thu, Aug 4, 2011 at 5:09 PM, Aidan Van Dyk wrote: > On Wed, Aug 3, 2011 at 5:04 PM, Robert Haas wrote: > >>      And hoping that the Linux guys decide to do something about it. >>  This isn't really our bug - lseek is quite cheap in the uncontended >> case. > > Has anyone tried this on a recen

Re: [HACKERS] mosbench revisited

2011-08-04 Thread Aidan Van Dyk
On Wed, Aug 3, 2011 at 5:04 PM, Robert Haas wrote: >  And hoping that the Linux guys decide to do something about it. >  This isn't really our bug - lseek is quite cheap in the uncontended > case. Has anyone tried this on a recent kernel (i.e. 2.6.39 or later), where they've finally remove t

Re: [HACKERS] mosbench revisited

2011-08-04 Thread Robert Haas
On Wed, Aug 3, 2011 at 9:16 PM, Robert Haas wrote: > Spinlocks seem to have a very ugly "tipping point". And on that note, here are oprofile results from "pgbench -n -T 300 -S -c 64 -j 64 -M prepared" on the latest master branch, compiled with "-O2 -fno-omit-frame-pointer". shared_buffers=8GB, 6

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Robert Haas
On Wed, Aug 3, 2011 at 6:21 PM, Jim Nasby wrote: > On Aug 3, 2011, at 1:21 PM, Robert Haas wrote: >> 1. "We configure PostgreSQL to use a 2 Gbyte application-level cache >> because PostgreSQL protects its free-list with a single lock and thus >> scales poorly with smaller caches."  This is a compl

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Robert Haas
On Wed, Aug 3, 2011 at 5:35 PM, Tom Lane wrote: > That still seems utterly astonishing to me.  We're touching each of > those files once per query cycle; a cycle that contains two message > sends, who knows how many internal spinlock/lwlock/heavyweightlock > acquisitions inside Postgres (some of w

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Jim Nasby
On Aug 3, 2011, at 1:21 PM, Robert Haas wrote: > 1. "We configure PostgreSQL to use a 2 Gbyte application-level cache > because PostgreSQL protects its free-list with a single lock and thus > scales poorly with smaller caches." This is a complaint about > BufFreeList lock which, in fact, I've seen

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 3, 2011 at 4:38 PM, Tom Lane wrote: >> ... We could possibly accept stale values for the >> planner estimates, but I think heapam's number had better be accurate. > I think the exact requirement is that, if the relation turns out to be > larger than the size we

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Robert Haas
On Wed, Aug 3, 2011 at 4:38 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Aug 3, 2011 at 3:38 PM, Tom Lane wrote: >>> (If the query ended up being a seqscan, I'd expect a second >>> lseek(SEEK_END) when the executor starts up, but I gather from the other >>> complaints that the mosbench p

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Tom Lane
Robert Haas writes: > On Wed, Aug 3, 2011 at 3:38 PM, Tom Lane wrote: >> (If the query ended up being a seqscan, I'd expect a second >> lseek(SEEK_END) when the executor starts up, but I gather from the other >> complaints that the mosbench people were only testing simple indexscan >> queries.)

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Robert Haas
On Wed, Aug 3, 2011 at 3:38 PM, Tom Lane wrote: > Robert Haas writes: >> On a straight pgbench -S test, you get four system calls per query: >> recvfrom(), lseek(), lseek(), sendto().  Adding -M prepared eliminates >> the two lseeks. > > [ scratches head... ]  Two? Yep. > Is that one for the ta

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Tom Lane
Robert Haas writes: > On a straight pgbench -S test, you get four system calls per query: > recvfrom(), lseek(), lseek(), sendto(). Adding -M prepared eliminates > the two lseeks. [ scratches head... ] Two? Is that one for the table and one for its lone index, or are we being redundant there?

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Robert Haas
On Wed, Aug 3, 2011 at 2:49 PM, Tom Lane wrote: > Martijn van Oosterhout writes: >> On Wed, Aug 03, 2011 at 02:21:25PM -0400, Robert Haas wrote: >>> It would be nice if the Linux guys would fix this problem for us, but >>> I'm not sure whether they will.  For those who may be curious, the >>> pro

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Tom Lane
Martijn van Oosterhout writes: > On Wed, Aug 03, 2011 at 02:21:25PM -0400, Robert Haas wrote: >> It would be nice if the Linux guys would fix this problem for us, but >> I'm not sure whether they will. For those who may be curious, the >> problem is in generic_file_llseek() in fs/read-write.c. O

Re: [HACKERS] mosbench revisited

2011-08-03 Thread Martijn van Oosterhout
On Wed, Aug 03, 2011 at 02:21:25PM -0400, Robert Haas wrote: > It would be nice if the Linux guys would fix this problem for us, but > I'm not sure whether they will. For those who may be curious, the > problem is in generic_file_llseek() in fs/read-write.c. On a platform > with 8-byte atomic rea