Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-18 Thread Eduardo Morras
On Wed, 16 Jan 2013 23:42:23 +0100 "T. E. Lawrence" wrote: > > On 15.01.2013, at 17:32, Jeff Janes wrote: > > T.E., Fortunately in point releases from August 2012 (9.0.9, 9.1.5, > > etc.), the default server log settings will log both the cancel and > > the command triggering the cancel. So if

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-16 Thread T. E. Lawrence
On 15.01.2013, at 17:32, Jeff Janes wrote: > T.E., Fortunately in point releases from August 2012 (9.0.9, 9.1.5, > etc.), the default server log settings will log both the cancel and > the command triggering the cancel. So if you are running an up to > date server, you can just look in the logs

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-16 Thread T. E. Lawrence
On 15.01.2013, at 16:36, Tom Lane wrote: > "T. E. Lawrence" > > So, apparently, we need to interrupt the heavy imports on some reasonable >> intervals and do manual VACUUM ANALYZE? > > Data import as such, no matter how "heavy", shouldn't be a problem. > The question is what are you doing tha

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-15 Thread Jeff Janes
On Tue, Jan 15, 2013 at 7:36 AM, Tom Lane wrote: > "T. E. Lawrence" > writes: >> On 15.01.2013, at 05:45, Jeff Janes wrote: Is the autovacuum 100% reliable in relation to VACUUM ANALYZE? > >>> No. For example, if you constantly do things that need an access exclusive >>> lock, then auto

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-15 Thread Tom Lane
"T. E. Lawrence" writes: > On 15.01.2013, at 05:45, Jeff Janes wrote: >>> Is the autovacuum 100% reliable in relation to VACUUM ANALYZE? >> No. For example, if you constantly do things that need an access exclusive >> lock, then autovac will keep getting interrupted and never finish. > I se

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-15 Thread T. E. Lawrence
On 15.01.2013, at 05:45, Jeff Janes wrote: >> Which makes me think that, as we grew the database more than 250 times in >> size over a 2-3 months period, relying on autovacuum (some tables grew from >> 200k to 50m records, other from 1m to 500m records), the autovacuum has >> either let us do

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-14 Thread Jeff Janes
On Monday, January 14, 2013, T. E. Lawrence wrote: > RESOLVED > -- > Dear all, > > Thank you for your great help and multiple advices. > > I discovered the problem and I have to say that it is very stupid and > strange. > > Here is what happened. > > ... > So I decided to try the whole thing pro

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-14 Thread T. E. Lawrence
RESOLVED -- Dear all, Thank you for your great help and multiple advices. I discovered the problem and I have to say that it is very stupid and strange. Here is what happened. >From all advices I tried first partial index. The index was built and there >was no change in the speed of the slow q

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread Alban Hertroys
On 12 January 2013 12:41, T. E. Lawrence wrote: > Hi and thank you for your notes! > > > You really ought to include the output of EXPLAIN ANALYZE in cases such > as these (if it doesn't already point you to the culprit). > > I'll do so, it takes quite long... > > > Most likely you'll find that t

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread Tony Theodore
On 12/01/2013, at 12:47 PM, T. E. Lawrence wrote: > Hello, > > I have a pretty standard query with two tables: > > SELECT table_a.id FROM table_a a, table_b b WHERE ... AND ... AND b.value=...; > > With the last "AND b.value=..." the query is extremely slow (did not wait for > it to end, but

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
Hi and thank you! On 12.01.2013, at 11:52, Eduardo Morras wrote: >> With the last "AND b.value=..." the query is extremely slow (did not wait >> for it to end, but more than a minute), because the value column is not >> indexed (contains items longer than 8K). > > You can construct your own h

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
Hi and thank you for your notes! > You really ought to include the output of EXPLAIN ANALYZE in cases such as > these (if it doesn't already point you to the culprit). I'll do so, it takes quite long... > Most likely you'll find that the last condition added a sequential scan to > the query pl

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
On 12.01.2013, at 07:10, Amit kapila wrote: > You can try once with below query: > Select * from (SELECT a.id,b.value FROM table_a a, table_b b WHERE ... AND > ... ) X where X.value=...; > > If this doesn't work can you send the Explain .. output for both queries(the > query you are using and t

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread Eduardo Morras
On Sat, 12 Jan 2013 02:47:26 +0100 "T. E. Lawrence" wrote: > Hello, > > I have a pretty standard query with two tables: > > SELECT table_a.id FROM table_a a, table_b b WHERE ... AND ... AND b.value=...; > > With the last "AND b.value=..." the query is extremely slow (did not wait for > it to

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread Alban Hertroys
You really ought to include the output of EXPLAIN ANALYZE in cases such as these (if it doesn't already point you to the culprit). Most likely you'll find that the last condition added a sequential scan to the query plan, which can have several causes/reasons. Are the estimated #rows close to the

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-11 Thread Amit kapila
On Saturday, January 12, 2013 7:17 AM T. E. Lawrence wrote: > Hello, > I have a pretty standard query with two tables: > SELECT table_a.id FROM table_a a, table_b b WHERE ... AND ... AND b.value=...; > With the last "AND b.value=..." the query is extremely slow (did not wait for > it to end, b

[GENERAL] reducing number of ANDs speeds up query

2013-01-11 Thread T. E. Lawrence
Hello, I have a pretty standard query with two tables: SELECT table_a.id FROM table_a a, table_b b WHERE ... AND ... AND b.value=...; With the last "AND b.value=..." the query is extremely slow (did not wait for it to end, but more than a minute), because the value column is not indexed (conta