Re: [HACKERS] Straightforward changes for increased SMP scalability

2008-03-24 Thread Bruce Momjian
Add to TODO: * SMP scalability improvements http://archives.postgresql.org/pgsql-hackers/2007-07/msg00439.php --- Simon Riggs wrote: > David Strong presented some excellent results of his SMP scalability > testing at Ot

Re: [HACKERS] Straightforward changes for increased SMP scalability

2008-03-24 Thread Bruce Momjian
Added to TODO: * SMP scalability improvements http://archives.postgresql.org/pgsql-hackers/2007-07/msg00439.php --- Simon Riggs wrote: > David Strong presented some excellent results of his SMP scalability > testing at

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: > David Strong presented some excellent results of his SMP scalability > testing at Ottawa in May.

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-17 Thread Zeugswetter Andreas ADI SD
> The NUM_BUFFER_PARTITIONS patch is fairly simple. We've > noticed gains with NUM_BUFFER_PARTITIONS set between 256 and > 2048, but little to no gain after 2048, although this might > depend on the benchmark and platform being used. We've Might this also be a padding issue, because 2048 part

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Andrew Sullivan
On Mon, Jul 16, 2007 at 01:23:46PM +0100, Simon Riggs wrote: > Both of these changes are simple enough to consider for 8.3 I'm in favour of scalability, of course, but are they really simple enough to put in for 8.3? I was under the impression that there was a push on to get the thing shipped, an

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Strong, David
>> I'm happy to run some benchmarks to show the improvements with various >> NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this >> is going to be useful. I can run 16 (base), 32, 64, 128 etc. type >> increments, but I'm more concerned about the number of cores to use. Do >> you

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Tom Lane
"Strong, David" <[EMAIL PROTECTED]> writes: > I'm happy to run some benchmarks to show the improvements with various > NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this > is going to be useful. I can run 16 (base), 32, 64, 128 etc. type > increments, but I'm more concerned abou

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Strong, David
07 9:10 AM To: Simon Riggs Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Straightforward changes for increased SMP scalability "Simon Riggs" <[EMAIL PROTECTED]> writes: > 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). Do you have any evidence to back up

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Tom Lane
"Simon Riggs" <[EMAIL PROTECTED]> writes: > 2. Increase NUM_BUFFER_PARTITIONS from 16 to 256 (or higher). Do you have any evidence to back up such a large increase? This change is not free; at the very least it will break contrib/pg_buffercache, which wants to lock all the partitions at once. lwl

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Joshua D. Drake
Andrew Dunstan wrote: Joshua D. Drake wrote: Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: If I have 16 cores, things are still really loud but I start to not be able to tell the difference. The percentage of improvement is much lower. E.g, 16 cores works and Postgr

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Andrew Dunstan
Joshua D. Drake wrote: Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Wait

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Strong, David
>Simon Riggs wrote: > >> Proposals >> >> 1. For the first result, I suggest that we introduce some padding into >> the shmem structure XLogCtlData to alleviate false sharing that may >> exist between holders of WALInsertLock, WALWriteLock and info_lck. The >> cost of this will be at most about 200

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Joshua D. Drake
Gregory Stark wrote: "Joshua D. Drake" <[EMAIL PROTECTED]> writes: +1 on the idea (I can speak to the technical side). What I can say is that it is pretty much known that after 8 cores we slow down. Although 8.2 is better than any other release in this regard. Wait, what benchmarks have you s

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Gregory Stark
"Joshua D. Drake" <[EMAIL PROTECTED]> writes: > +1 on the idea (I can speak to the technical side). What I can say is that it > is pretty much known that after 8 cores we slow down. Although 8.2 is better > than any other release in this regard. Wait, what benchmarks have you seen where we slow

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Joshua D. Drake
Simon Riggs wrote: Proposals 1. For the first result, I suggest that we introduce some padding into the shmem structure XLogCtlData to alleviate false sharing that may exist between holders of WALInsertLock, WALWriteLock and info_lck. The cost of this will be at most about 200 bytes of shmem, w

[HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Simon Riggs
David Strong presented some excellent results of his SMP scalability testing at Ottawa in May. http://www.pgcon.org/2007/schedule/events/16.en.html There are some easy things we can do to take advantage of those results, especially the ones that were hardware independent. The hardware independent