Re: [GENERAL] insert performance riddle

2005-08-12 Thread Ed L.
On Friday August 12 2005 6:29 pm, Michael Fuhr wrote: > I meant profiling of DBD::Pg, as Greg Sabino Mullane > suggested. Here's his message in case you missed it: No, I didn't miss that, and will do that as a next step. Thanks, Michael. Ed ---(end of broadcast)---

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Michael Fuhr
On Fri, Aug 12, 2005 at 06:20:27PM -0600, Ed L. wrote: > On Friday August 12 2005 6:11 pm, Michael Fuhr wrote: > > Has anything changed on the system since the results were > > consistently slow? New hardware, new versions of anything, > > reboot, etc.? Did you do any profiling when DBD::Pg was > >

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Ed L.
On Friday August 12 2005 6:11 pm, Michael Fuhr wrote: > Has anything changed on the system since the results were > consistently slow?  New hardware, new versions of anything, > reboot, etc.?  Did you do any profiling when DBD::Pg was > consistently slow to see where the bottleneck was? All good q

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Michael Fuhr
On Fri, Aug 12, 2005 at 05:37:22PM -0600, Ed L. wrote: > On Friday August 12 2005 5:27 pm, Michael Fuhr wrote: > > How consistent were the results before? I got the impression > > that you saw consistently bad performance with DBD::Pg when > > other methods performed well. > > Results were very c

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Ed L.
On Friday August 12 2005 5:27 pm, Michael Fuhr wrote: > On Fri, Aug 12, 2005 at 05:20:49PM -0600, Ed L. wrote: > > Well, just as I thought I had this one pinned, my test > > results have become wildly inconsistent, eroding all > > confidence in my prior conclusions about DBI slowness, etc. > > I'v

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Michael Fuhr
On Fri, Aug 12, 2005 at 05:20:49PM -0600, Ed L. wrote: > Well, just as I thought I had this one pinned, my test results > have become wildly inconsistent, eroding all confidence in my > prior conclusions about DBI slowness, etc. I've seen at least > 1000+ QPS performance via DBI/DBD::Pg, Pg, an

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Ed L.
On Thursday August 11 2005 6:20 pm, Michael Fuhr wrote: > On Thu, Aug 11, 2005 at 03:29:29PM -0600, Ed L. wrote: > > Michael, you nailed it again. My libpq test C program > > delivered between 2400 QPS and 5000 QPS vs ~10 QPS for > > DBI/DBD::Pg on this box. > > > > It remains unclear to me why th

Re: [GENERAL] insert performance riddle

2005-08-12 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > It remains unclear to me why the same DBI/DBD::Pg client code > would deliver performance 2-3 orders of magnitude better on > other roughly comparable or inferior boxes. You need to see exactly what is going on to make things so slow. You can try

Re: [GENERAL] insert performance riddle

2005-08-11 Thread Michael Fuhr
On Thu, Aug 11, 2005 at 03:29:29PM -0600, Ed L. wrote: > Michael, you nailed it again. My libpq test C program delivered > between 2400 QPS and 5000 QPS vs ~10 QPS for DBI/DBD::Pg on this > box. > > It remains unclear to me why the same DBI/DBD::Pg client code > would deliver performance 2-3 o

Re: [GENERAL] insert performance riddle

2005-08-11 Thread Steve Wormley
> > Michael, you nailed it again. My libpq test C program delivered > between 2400 QPS and 5000 QPS vs ~10 QPS for DBI/DBD::Pg on this > box. > > It remains unclear to me why the same DBI/DBD::Pg client code > would deliver performance 2-3 orders of magnitude better on > other roughly comparabl

Re: [GENERAL] insert performance riddle

2005-08-11 Thread Ed L.
On Thursday August 11 2005 1:37 pm, Michael Fuhr wrote: > Have you done any client-side tests that eliminate Perl?  I'd > suggest writing a little C program so you can measure libpq's > performance without the extra layers of Perl and DBI/DBD::Pg. >  Test both local (Unix socket) and network (IPv4

Re: [GENERAL] insert performance riddle

2005-08-11 Thread Michael Fuhr
On Thu, Aug 11, 2005 at 12:59:32PM -0600, Ed L. wrote: > > Michael, you seem to have nailed it. The local inserts (via > > Unix domain sockets?) that were running at 6 QPS ran at 6800 > > to 41000 QPS in a PL/pgSQL function. > > Here's another part of the riddle. The query durations for the > i

Re: [GENERAL] insert performance riddle

2005-08-11 Thread Ed L.
On Thursday August 11 2005 12:36 pm, Ed L. wrote: > On Wednesday August 10 2005 6:03 pm, Michael Fuhr wrote: > > On Wed, Aug 10, 2005 at 05:02:46PM -0600, Ed L. wrote: > > > I have two identical servers giving abysmal INSERT > > > performance in pgsql 7.3.4, 7.4.8, and 8.1devel under no > > > load

Re: [GENERAL] insert performance riddle

2005-08-11 Thread Ed L.
On Wednesday August 10 2005 6:03 pm, Michael Fuhr wrote: > On Wed, Aug 10, 2005 at 05:02:46PM -0600, Ed L. wrote: > > I have two identical servers giving abysmal INSERT > > performance in pgsql 7.3.4, 7.4.8, and 8.1devel under no > > load or I/O contention at all (no dumps, no vacuums, no > > apps,

Re: [GENERAL] insert performance riddle

2005-08-10 Thread Michael Fuhr
On Wed, Aug 10, 2005 at 05:02:46PM -0600, Ed L. wrote: > I have two identical servers giving abysmal INSERT performance in > pgsql 7.3.4, 7.4.8, and 8.1devel under no load or I/O contention > at all (no dumps, no vacuums, no apps, etc). Any suggested > investigations appreciated... > > Metric:

[GENERAL] insert performance riddle

2005-08-10 Thread Ed L.
I have two identical servers giving abysmal INSERT performance in pgsql 7.3.4, 7.4.8, and 8.1devel under no load or I/O contention at all (no dumps, no vacuums, no apps, etc). Any suggested investigations appreciated... Metric: I'm measuring average insert speed on the following table with