Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Strong, David
>> I'm happy to run some benchmarks to show the improvements with various >> NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this >> is going to be useful. I can run 16 (base), 32, 64, 128 etc. type >> increments, but I'm more concerned about the number of cores to use. Do >> you

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Strong, David
Tom, I'm happy to run some benchmarks to show the improvements with various NUM_BUFFER_PARTITIONS settings. However, I want to make sure that this is going to be useful. I can run 16 (base), 32, 64, 128 etc. type increments, but I'm more concerned about the number of cores to use. Do you have a su

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-07-16 Thread Strong, David
>Simon Riggs wrote: > >> Proposals >> >> 1. For the first result, I suggest that we introduce some padding into >> the shmem structure XLogCtlData to alleviate false sharing that may >> exist between holders of WALInsertLock, WALWriteLock and info_lck. The >> cost of this will be at most about 200

Re: [HACKERS] Prepared Statement Question

2006-10-09 Thread Strong, David
Tom, Thanks for the advice. Yes, we were looking at the possibility of saving the palloc(s) (malloc in some cases) on the statement. David From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Mon 10/9/2006 9:08 AM To: Strong, David Cc: pgsql-hackers

[HACKERS] Prepared Statement Question

2006-10-09 Thread Strong, David
We have a question regarding prepared statements. The following code is located in src/backend/tcop/postgres.c: /* Get the parameter format codes */ numPFormats = pq_getmsgint(input_message, 2); if (numPFormats > 0) { int i; pformats = (int16 *) palloc(numPForma

[HACKERS] Another aspect of set_ps_display ()

2006-10-04 Thread Strong, David
We were just analyzing some more OProfile and ltrace data against Postgres 8.2Beta1 and we noticed a number of calls as follows: strlen("postgres: tpc tpc 192.168.1.200("...)= 58 memset(0xb6b2, '\000', 2344) = 0xb6b2 We have tracked this down to the following code in

Re: [HACKERS] Faster StrNCpy

2006-10-02 Thread Strong, David
Tom, Yes, the clients are using the V3 protocol and prepared statements. David From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Mon 10/2/2006 2:09 PM To: Strong, David Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Faster StrNCpy "Strong,

Re: [HACKERS] Faster StrNCpy

2006-10-02 Thread Strong, David
Mark, Thanks for attaching the C code for your test. I ran a few tests on a 3Ghz Intel Xeon Paxville (dual core) system. I hope the formatting of this table survives: Method Size N=1024*1024 N=1 MEMCPY 63 6964927 us 582494 us MEMCPY 32 7102497 us 582467 us MEMCPY

[HACKERS] Testing strlcpy ()

2006-09-29 Thread Strong, David
We've completed some tests comparing Postgres 8.2beta1 (beta1) and Postgres 8.2beta1 with strlcpy () (strlcpy). First and foremost, the patch seems to be stable - we have not run into any issues with it. After a database reload, there is an 11% difference between strlcpy and beta1 - strlcpy

Re: [HACKERS] Faster StrNCpy

2006-09-28 Thread Strong, David
code that don't use memcpy () currently? David -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 27, 2006 4:27 PM To: Strong, David Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Faster StrNCpy On Wed, Sep 27, 2006 at 07:08:05AM

Re: [HACKERS] Faster StrNCpy

2006-09-27 Thread Strong, David
We sometimes see TupleDescInitEntry () taking high CPU times via OProfile. This does include, amongst a lot of other code, a call to namestrcpy () which in turn calls StrNCpy (). Perhaps this is not a good candidate right now as a name string is only 64 bytes. David __

Re: [HACKERS] Faster StrNCpy

2006-09-27 Thread Strong, David
Tom, Let us know when you've added strlcpy () and we'll be happy to run some tests on the new code. David From: [EMAIL PROTECTED] on behalf of Tom Lane Sent: Tue 9/26/2006 7:25 PM To: josh@agliodbs.com Cc: pgsql-hackers@postgresql.org; Neil Conway; Martijn va

[HACKERS] Questions/observations about set_ps_display ()

2006-09-26 Thread Strong, David
We're looking for some advice and/or comments. During performance and scalability testing with 8.1.4 and also 8.2Beta1, OProfile reported that the strncpy () C library call was taking a large amount of CPU time while we were running one of our benchmarks. We traced a partial benchmark run using l

Re: [HACKERS] Lock partitions

2006-09-14 Thread Strong, David
also add timing for semaphore waits within LWLockAcquire, if that would be a useful statistic. Let me know if there are any other tests or metrics that would be useful. David -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 13, 2006 1:36 PM To

Re: [HACKERS] Lock partitions

2006-09-13 Thread Strong, David
ssage- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Strong, David Sent: Wednesday, September 13, 2006 10:52 AM To: PostgreSQL-development Subject: Re: [HACKERS] Lock partitions Simon, In the 16/16 (16 buffer partitions/16 lock partitions) test, the WALInsertLock lock had 146

Re: [HACKERS] Lock partitions

2006-09-13 Thread Strong, David
2:22 AM To: Tom Lane Cc: Strong, David; PostgreSQL-development Subject: Re: [HACKERS] Lock partitions On Tue, 2006-09-12 at 12:40 -0400, Tom Lane wrote: > "Strong, David" <[EMAIL PROTECTED]> writes: > > When using 16 buffer and 16 lock partitions, we see that BufMapping &g

Re: [HACKERS] Lock partitions

2006-09-12 Thread Strong, David
ve not tried to remove the SInvalLock lock to see where time flows to next, but we might. David -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 12, 2006 9:40 AM To: Strong, David Cc: PostgreSQL-development Subject: Re: [HACKERS] Lock partitions "St

Re: [HACKERS] Lock partitions

2006-09-12 Thread Strong, David
We can pass on what we've seen when running tests here with different BufMapping and LockMgr partition sizes. We use a TPC-C inspired benchmark. Currently it is configured to run 25 backend processes. The test runs for 16 minutes as this is the minimum amount of time we can run and obtain useful i

[HACKERS] Looking at Postgres 8.2

2006-09-07 Thread Strong, David
As part of our ongoing research into Postgres performance and scalability, we recently downloaded version 8.2 from CVS and we wanted to pass on some observations. When comparing 8.2 against 8.1.4, we see that there is roughly a 20% increase in throughput. We credit most of this improvement to the