Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Robert Haas
On Thu, Oct 7, 2010 at 1:21 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> perhaps it would be possible by, say, increasing the number of >> lock partitions by 8x.  It would be nice to segregate these issues >> though, because using pread/pwrite is probably a lot less work >> than rewriting

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Kevin Grittner
Stephen Frost wrote: > Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: >> Robert Haas wrote: >>> perhaps it would be possible by, say, increasing the number of >>> lock partitions by 8x. >> changing this 4 to a 7?: >> >> #define LOG2_NUM_LOCK_PARTITIONS 4 > I'm pretty sure we were t

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Stephen Frost
* Kevin Grittner (kevin.gritt...@wicourts.gov) wrote: > Robert Haas wrote: > > perhaps it would be possible by, say, increasing the number of > > lock partitions by 8x. It would be nice to segregate these issues > > though, because using pread/pwrite is probably a lot less work > > than rewriting

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Kevin Grittner
Robert Haas wrote: > perhaps it would be possible by, say, increasing the number of > lock partitions by 8x. It would be nice to segregate these issues > though, because using pread/pwrite is probably a lot less work > than rewriting our lock manager. You mean easier than changing this 4 to a

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Ivan Voras
On 10/07/10 02:39, Robert Haas wrote: On Wed, Oct 6, 2010 at 6:31 PM, Ivan Voras wrote: On 10/04/10 20:49, Josh Berkus wrote: The other major bottleneck they ran into was a kernel one: reading from the heap file requires a couple lseek operations, and Linux acquires a mutex on the inode to do

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Robert Haas
On Wed, Oct 6, 2010 at 10:07 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> It's good to be you. > > They're HP BL465 G7's w/ 2x 12-core AMD processors and 48G of RAM. > Unfortunately, they currently only have local storage, but it seems > unlikely that would be an issu

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-07 Thread Ivan Voras
On 7 October 2010 03:25, Tom Lane wrote: > Ivan Voras writes: >> On 10/04/10 20:49, Josh Berkus wrote: The other major bottleneck they ran into was a kernel one: reading from the heap file requires a couple lseek operations, and Linux acquires a mutex on the inode to do that. > >>

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > It's good to be you. They're HP BL465 G7's w/ 2x 12-core AMD processors and 48G of RAM. Unfortunately, they currently only have local storage, but it seems unlikely that would be an issue for this. > I don't suppose you could try to replicate the lse

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Robert Haas
On Wed, Oct 6, 2010 at 9:30 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> Hey, I didn't know about those.  That sounds like it might be worth >> investigating, though I confess I lack a 48-core machine on which to >> measure the alleged benefit. > > I've got a couple 2

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Hey, I didn't know about those. That sounds like it might be worth > investigating, though I confess I lack a 48-core machine on which to > measure the alleged benefit. I've got a couple 24-core systems, if it'd be sufficiently useful to test with..

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Tom Lane
Ivan Voras writes: > On 10/04/10 20:49, Josh Berkus wrote: >>> The other major bottleneck they ran into was a kernel one: reading from >>> the heap file requires a couple lseek operations, and Linux acquires a >>> mutex on the inode to do that. > Hmmm... lseek? As in "lseek() then read() or write

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Robert Haas
On Wed, Oct 6, 2010 at 6:31 PM, Ivan Voras wrote: > On 10/04/10 20:49, Josh Berkus wrote: > >>> The other major bottleneck they ran into was a kernel one: reading from >>> the heap file requires a couple lseek operations, and Linux acquires a >>> mutex on the inode to do that. The proper place to

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Jon Nelson
On Wed, Oct 6, 2010 at 5:31 PM, Ivan Voras wrote: > On 10/04/10 20:49, Josh Berkus wrote: > >>> The other major bottleneck they ran into was a kernel one: reading from >>> the heap file requires a couple lseek operations, and Linux acquires a >>> mutex on the inode to do that. The proper place to

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-06 Thread Ivan Voras
On 10/04/10 20:49, Josh Berkus wrote: >> The other major bottleneck they ran into was a kernel one: reading from >> the heap file requires a couple lseek operations, and Linux acquires a >> mutex on the inode to do that. The proper place to fix this is >> certainly in the kernel but it may be poss

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-05 Thread Dan Ports
I wasn't involved in this work but I do know a bit about it. Sadly, the work on Postgres performance was cut down to under a page, complete with the amazing offhand mention of "rewriting PostgreSQL's lock manager". Here are a few more details... The benchmarks in this paper are all about stressing

Re: [PERFORM] [HACKERS] MIT benchmarks pgsql multicore (up to 48)performance

2010-10-04 Thread Josh Berkus
Dan, (btw, OpenSQL Confererence is going to be at MIT in 2 weeks. Think anyone from the MOSBENCH team could attend? http://www.opensqlcamp.org/Main_Page) > The big takeaway for -hackers, I think, is that lock manager > performance is going to be an issue for large multicore systems, and > the un