Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-16 Thread T. E. Lawrence
On 15.01.2013, at 17:32, Jeff Janes wrote: > T.E., Fortunately in point releases from August 2012 (9.0.9, 9.1.5, > etc.), the default server log settings will log both the cancel and > the command triggering the cancel. So if you are running an up to > date server, you can just look in the logs

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-16 Thread T. E. Lawrence
On 15.01.2013, at 16:36, Tom Lane wrote: > "T. E. Lawrence" > > So, apparently, we need to interrupt the heavy imports on some reasonable >> intervals and do manual VACUUM ANALYZE? > > Data import as such, no matter how "heavy", shouldn't be a

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-15 Thread T. E. Lawrence
On 15.01.2013, at 05:45, Jeff Janes wrote: >> Which makes me think that, as we grew the database more than 250 times in >> size over a 2-3 months period, relying on autovacuum (some tables grew from >> 200k to 50m records, other from 1m to 500m records), the autovacuum has >> either let us do

Re: [GENERAL] Linux Distribution Preferences?

2013-01-14 Thread T. E. Lawrence
On 15.01.2013, at 00:28, Rich Shepard wrote: > On Tue, 15 Jan 2013, T. E. Lawrence wrote: > >> When forced on Linux we like Debian because it is so conservative (which >> can sometimes drive one crazy, especially if one needs some cutting edge >> feature). >

Re: [GENERAL] Linux Distribution Preferences?

2013-01-14 Thread T. E. Lawrence
> Hey guys, > > I'm not sure the last time I saw this discussion, but I was somewhat curious: > what would be your ideal Linux distribution for a nice solid PostgreSQL > installation? We've kinda bounced back and forth between RHEL, CentOS, and > Ubuntu LTS, so I was wondering what everyone els

Re: [GENERAL] reducing number of ANDs speeds up query RESOLVED

2013-01-14 Thread T. E. Lawrence
RESOLVED -- Dear all, Thank you for your great help and multiple advices. I discovered the problem and I have to say that it is very stupid and strange. Here is what happened. >From all advices I tried first partial index. The index was built and there >was no change in the speed of the slow q

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
Hi and thank you! On 12.01.2013, at 11:52, Eduardo Morras wrote: >> With the last "AND b.value=..." the query is extremely slow (did not wait >> for it to end, but more than a minute), because the value column is not >> indexed (contains items longer than 8K). > > You can construct your own h

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
Hi and thank you for your notes! > You really ought to include the output of EXPLAIN ANALYZE in cases such as > these (if it doesn't already point you to the culprit). I'll do so, it takes quite long... > Most likely you'll find that the last condition added a sequential scan to > the query pl

Re: [GENERAL] reducing number of ANDs speeds up query

2013-01-12 Thread T. E. Lawrence
On 12.01.2013, at 07:10, Amit kapila wrote: > You can try once with below query: > Select * from (SELECT a.id,b.value FROM table_a a, table_b b WHERE ... AND > ... ) X where X.value=...; > > If this doesn't work can you send the Explain .. output for both queries(the > query you are using and t

[GENERAL] reducing number of ANDs speeds up query

2013-01-11 Thread T. E. Lawrence
Hello, I have a pretty standard query with two tables: SELECT table_a.id FROM table_a a, table_b b WHERE ... AND ... AND b.value=...; With the last "AND b.value=..." the query is extremely slow (did not wait for it to end, but more than a minute), because the value column is not indexed (conta

Re: [GENERAL] Using a GIN index on an integer array to model sets of tags

2012-11-17 Thread T. E. Lawrence
> I am researching how to set up a schema for querying a set of tags > associated with an object. I would be interested in hearing your conclusions. I am currently researching in a similar direction. We have streaming replication where the slaves are used for data mining, storing currently abou

Re: [GENERAL] 9.2 streaming replication issue and solution strategy

2012-11-17 Thread T. E. Lawrence
> Have you looked at the below?: > > http://www.postgresql.org/docs/9.2/interactive/hot-standby.html#HOT-STANDBY-CONFLICT > > 25.5.2. Handling Query Conflicts Yes, thank you! I am hoping to hear more from people who have running 9.2 systems w/ between 100m and 1b records, w/ streaming replicat

[GENERAL] 9.2 streaming replication issue and solution strategy

2012-11-17 Thread T. E. Lawrence
Hello, We are running 9.2 w/ streaming replication. The slave is used for heavy tsearch based data mining. Apparently depending on the business of the master the slave queries fail with different frequency with the following message — ERROR: canceling statement due to conflict with recovery D