Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Stefan Kaltenbrunner
On 07/24/2011 05:55 PM, Tom Lane wrote: > Stefan Kaltenbrunner writes: >> interesting - iirc we actually had some reports about current libpq >> behaviour causing scaling issues on some OSes - see >> http://archives.postgresql.org/pgsql-hackers/2009-06/msg00748.php and >> some related threads. Iir

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Tom Lane
Stefan Kaltenbrunner writes: > interesting - iirc we actually had some reports about current libpq > behaviour causing scaling issues on some OSes - see > http://archives.postgresql.org/pgsql-hackers/2009-06/msg00748.php and > some related threads. Iirc the final patch for that was never applied >

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Tom Lane
Jeff Janes writes: > How was this profile generated? I get a similar profile using > --enable-profiling and gprof, but I find it not believable. The > complete absence of any calls to libpq is not credible. I don't know > about your profiler, but with gprof they should be listed in the call > g

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-24 Thread Stefan Kaltenbrunner
On 07/24/2011 03:50 AM, Jeff Janes wrote: > On Mon, Jun 13, 2011 at 7:03 AM, Stefan Kaltenbrunner > wrote: >> On 06/13/2011 01:55 PM, Stefan Kaltenbrunner wrote: >> >> [...] >> >>> all those tests are done with pgbench running on the same box - which >>> has a noticable impact on the results becau

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-07-23 Thread Jeff Janes
On Mon, Jun 13, 2011 at 7:03 AM, Stefan Kaltenbrunner wrote: > On 06/13/2011 01:55 PM, Stefan Kaltenbrunner wrote: > > [...] > >> all those tests are done with pgbench running on the same box - which >> has a noticable impact on the results because pgbench is using ~1 core >> per 8 cores of the ba

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-14 Thread Stefan Kaltenbrunner
On 06/14/2011 02:27 AM, Jeff Janes wrote: > On Mon, Jun 13, 2011 at 7:03 AM, Stefan Kaltenbrunner > wrote: > ... >> >> >> so it seems that sysbench is actually significantly less overhead than >> pgbench and the lower throughput at the higher conncurency seems to be >> cause by sysbench being able

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-14 Thread Jeff Janes
On Mon, Jun 13, 2011 at 9:09 PM, Alvaro Herrera wrote: > I noticed that pgbench's doCustom (the function highest in the profile > posted) returns doing nothing if the connection is supposed to be > "sleeping"; seems an open door for busy waiting.  I didn't check the > rest of the code to see if t

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-13 Thread Itagaki Takahiro
On Tue, Jun 14, 2011 at 13:09, Alvaro Herrera wrote: > I noticed that pgbench's doCustom (the function highest in the profile > posted) returns doing nothing if the connection is supposed to be > "sleeping"; seems an open door for busy waiting. pgbench uses select() with/without timeout in the ca

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-13 Thread Alvaro Herrera
Excerpts from Jeff Janes's message of lun jun 13 20:27:15 -0400 2011: > On Mon, Jun 13, 2011 at 7:03 AM, Stefan Kaltenbrunner > wrote: > ... > > > > > > so it seems that sysbench is actually significantly less overhead than > > pgbench and the lower throughput at the higher conncurency seems to be

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-13 Thread Greg Smith
On 06/13/2011 08:27 PM, Jeff Janes wrote: pgbench sends each query (per connection) and waits for the reply before sending another. Do we know whether sysbench does that, or if it just stuffs the kernel's IPC buffer full of queries without synchronously waiting for individual replies? sysb

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-13 Thread Itagaki Takahiro
On Tue, Jun 14, 2011 at 09:27, Jeff Janes wrote: > pgbench sends each query (per connection) and waits for the reply > before sending another. We can use -j option to run pgbench in multiple threads to avoid request starvation. What setting did you use, Stefan? >> for those curious - the profile

Re: pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-13 Thread Jeff Janes
On Mon, Jun 13, 2011 at 7:03 AM, Stefan Kaltenbrunner wrote: ... > > > so it seems that sysbench is actually significantly less overhead than > pgbench and the lower throughput at the higher conncurency seems to be > cause by sysbench being able to stress the backend even more than > pgbench can.

pgbench cpu overhead (was Re: [HACKERS] lazy vxid locks, v1)

2011-06-13 Thread Stefan Kaltenbrunner
On 06/13/2011 01:55 PM, Stefan Kaltenbrunner wrote: [...] > all those tests are done with pgbench running on the same box - which > has a noticable impact on the results because pgbench is using ~1 core > per 8 cores of the backend tested in cpu resoures - though I don't think > it causes any cha