Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-19 Thread Robert Haas
On Mon, Nov 17, 2014 at 4:27 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Nov 13, 2014 at 7:34 PM, Tom Lane wrote: >>> One thing that occurs to me is that if the generic plan estimate comes >>> out much cheaper than the custom one, maybe we should assume that the >>> generic's cost estim

Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-18 Thread Alban Hertroys
On 17 November 2014 22:27, Tom Lane wrote: > Another idea that occurred to me is to run a planning cycle in which the > actual parameter values are made available to the planner, but as > estimates not hard constants (this facility already exists, it's just not > being used by plancache.c). This

Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-17 Thread Sam Saffron
One interesting option would be kicking in an extra more expensive planning cycle after the Nth run of the query, in general a lot of these planned queries run 1000s of times, if you add some extra cost to run 100 it may not be prohibitive cost wise. On Tue, Nov 18, 2014 at 8:27 AM, Tom Lane wrot

Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-17 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 13, 2014 at 7:34 PM, Tom Lane wrote: >> One thing that occurs to me is that if the generic plan estimate comes >> out much cheaper than the custom one, maybe we should assume that the >> generic's cost estimate is bogus. Right offhand I can't think of a reason >

Re: [HACKERS] [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-17 Thread Robert Haas
On Thu, Nov 13, 2014 at 7:34 PM, Tom Lane wrote: > One thing that occurs to me is that if the generic plan estimate comes > out much cheaper than the custom one, maybe we should assume that the > generic's cost estimate is bogus. Right offhand I can't think of a reason > for a custom plan to look

Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Sam Saffron
Thank you so much! So to recap the general way to reproduce this issue is: create table products(id int primary key, type varchar); insert into products select generate_series(1,1), 'aaa'; insert into products select generate_series(10001,2), 'bbb'; create index idx on products(type); pre

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
David Johnston writes: > ​While "planner hints" comes to mind...on the SQL side can we extend the > "PREPARE" command with two additional keywords?​ > ​PREPARE > name [ ( data_type [, ...] ) ] [ > [NO] GENERIC > ​] ​ > ​AS statement Don't really see the point. The OP's problem is t

Re: [HACKERS] Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread David Johnston
On Thu, Nov 13, 2014 at 5:47 PM, Tom Lane wrote: > David G Johnston writes: > > Tom Lane-2 wrote > >> In the meantime, I assume that your real data contains a small > percentage > >> of values other than these two? If so, maybe cranking up the statistics > >> target would help. If the planner

Re: [GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Tom Lane
Sam Saffron writes: > I have hit a rather odd issue with prepared queries on both pg 9.3 and 9.4 > beta. > I have this table (copy at http://samsaffron.com/testing.db.gz) with a > very odd performance profile: Interesting case. The issue seems to be that your statistics look like this: select

[GENERAL] Performance issue with libpq prepared queries on 9.3 and 9.4

2014-11-13 Thread Sam Saffron
I have hit a rather odd issue with prepared queries on both pg 9.3 and 9.4 beta. I have this table (copy at http://samsaffron.com/testing.db.gz) with a very odd performance profile: When I run the following prepared query it is running significantly slower than the raw counterpart: ``` select *