Re: [PERFORM] Full text search - query plan? PG 8.4.1

2009-10-23 Thread Jesper Krogh
Scott Marlowe wrote: > On Fri, Oct 23, 2009 at 2:32 PM, Jesper Krogh wrote: >> Tom Lane wrote: >>> Jesper Krogh writes: Tom Lane wrote: > ... There's something strange about your tsvector index. Maybe > it's really huge because the documents are huge? huge is a relative term, b

Re: [PERFORM] Table Clustering & Time Range Queries

2009-10-23 Thread Stephen Frost
* Kevin Buckham (kbuck...@applocation.net) wrote: > I came across links to pg_reorg previously but it seemed that the > project was a bit "dead". There is active development but not much > information, and not much in the way of discussions. I will definitely > be testing both partitioning and pg_

Re: [PERFORM] Full text search - query plan? PG 8.4.1

2009-10-23 Thread Scott Marlowe
On Fri, Oct 23, 2009 at 2:32 PM, Jesper Krogh wrote: > Tom Lane wrote: >> Jesper Krogh writes: >>> Tom Lane wrote: ... There's something strange about your tsvector index.  Maybe it's really huge because the documents are huge? >> >>> huge is a relative term, but length(ts_vector(body))

Re: [PERFORM] Full text search - query plan? PG 8.4.1

2009-10-23 Thread Tom Lane
Jesper Krogh writes: > Is is possible to manually set the cost for the @@ operator? You want to set the cost for the underlying function. regards, tom lane -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription

Re: [PERFORM] Full text search - query plan? PG 8.4.1

2009-10-23 Thread Jesper Krogh
Tom Lane wrote: > Jesper Krogh writes: >> Tom Lane wrote: >>> ... There's something strange about your tsvector index. Maybe >>> it's really huge because the documents are huge? > >> huge is a relative term, but length(ts_vector(body)) is about 200 for >> each document. Is that huge? > > It's b

Re: [PERFORM] Calculating selectivity for the query-planner on ts_vector colums.

2009-10-23 Thread Jesper Krogh
Tom Lane wrote: > Jesper Krogh writes: >> It seems to me that the row estimates on a ts_vector search is a bit on >> the low side for terms that is not in th MCV-list in pg_stats: > > tsvector has its own selectivity estimator that's not like plain scalar > equality. Look into src/backend/tsearc

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread Jeff Davis
On Fri, 2009-10-23 at 17:27 +0100, Richard Huxton wrote: > Returns an array of keys given a value to be queried; that is, query is > the value on the right-hand side of an indexable operator whose > left-hand side is the indexed column > > So - that is presumably two separate arrays of keys being

Re: [PERFORM] Table Clustering & Time Range Queries

2009-10-23 Thread Kevin Buckham
I came across links to pg_reorg previously but it seemed that the project was a bit "dead". There is active development but not much information, and not much in the way of discussions. I will definitely be testing both partitioning and pg_reorg. I am curious to see if pg_reorg will be stable eno

Re: [PERFORM] Calculating selectivity for the query-planner on ts_vector colums.

2009-10-23 Thread Tom Lane
Jesper Krogh writes: > It seems to me that the row estimates on a ts_vector search is a bit on > the low side for terms that is not in th MCV-list in pg_stats: tsvector has its own selectivity estimator that's not like plain scalar equality. Look into src/backend/tsearch/ts_selfuncs.c if you wan

[PERFORM] Calculating selectivity for the query-planner on ts_vector colums.

2009-10-23 Thread Jesper Krogh
Hi It seems to me that the row estimates on a ts_vector search is a bit on the low side for terms that is not in th MCV-list in pg_stats: ftstest=# explain select id from ftstest where ftstest_body_fts @@ to_tsquery('nonexistingterm') order by id limit 10;

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread Richard Huxton
Jeff Davis wrote: > On Fri, 2009-10-23 at 09:26 +0100, Richard Huxton wrote: >> That structure isn't exposed to the planner though, so it doesn't >> benefit from any re-ordering the planner would normally do for normal >> (exposed) AND/OR clauses. > > I don't think that explains it, because in the

Re: [PERFORM] query planning different in plpgsql?

2009-10-23 Thread Tom Lane
On Fri, Oct 23, 2009 at 11:38 AM, Michal J. Kubski wrote: >> I am running the following one and it takes 10 minutes in the procedure >> when it is pretty fast standalone. >> >> # EXPLAIN ANALYZE SELECT m.domain_id, nsr_id FROM nsr_meta m, last_snapshot >> l WHERE m.domain_id = l.domain_id; Is it

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread Jeff Davis
On Fri, 2009-10-23 at 09:45 +0200, jes...@krogh.cc wrote: > No, it definately has to go visit the index/table to confirm findings, but > that why I wrote Queryplan in the subject line, because this os only about > the strategy to pursue to obtain the results. And a strategy about > limiting the amo

Re: [PERFORM] query planning different in plpgsql?

2009-10-23 Thread Grzegorz Jaśkiewicz
On Fri, Oct 23, 2009 at 4:49 PM, Scott Mead wrote: > > > Do you not have an index on last_snapshot.domain_id? > that, and also try rewriting a query as JOIN. There might be difference in performance/plan. -- GJ

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread Jeff Davis
On Fri, 2009-10-23 at 09:26 +0100, Richard Huxton wrote: > That structure isn't exposed to the planner though, so it doesn't > benefit from any re-ordering the planner would normally do for normal > (exposed) AND/OR clauses. I don't think that explains it, because in the second plan you only see a

Re: [PERFORM] query planning different in plpgsql?

2009-10-23 Thread Scott Mead
On Fri, Oct 23, 2009 at 11:38 AM, Michal J. Kubski wrote: > > > Hi, > > Is there any way to get the query plan of the query run in the stored > procedure? > I am running the following one and it takes 10 minutes in the procedure > when it is pretty fast standalone. > > Any ideas would be welcome!

[PERFORM] query planning different in plpgsql?

2009-10-23 Thread Michal J . Kubski
Hi, Is there any way to get the query plan of the query run in the stored procedure? I am running the following one and it takes 10 minutes in the procedure when it is pretty fast standalone. Any ideas would be welcome! # EXPLAIN ANALYZE SELECT m.domain_id, nsr_id FROM nsr_meta m, last_snapsho

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread Richard Huxton
jes...@krogh.cc wrote: >> That structure isn't exposed to the planner though, so it doesn't >> benefit from any re-ordering the planner would normally do for normal >> (exposed) AND/OR clauses. >> >> Now, to_tsquery() can't re-order the search terms because it doesn't >> know what column it's being

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread jesper
> jes...@krogh.cc wrote: >> >> So getting them with AND inbetween gives x100 better performance. All >> queries are run on "hot disk" repeated 3-5 times and the number are from >> the last run, so disk-read effects should be filtered away. >> >> Shouldn't it somehow just do what it allready are cap

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread Richard Huxton
jes...@krogh.cc wrote: > > So getting them with AND inbetween gives x100 better performance. All > queries are run on "hot disk" repeated 3-5 times and the number are from > the last run, so disk-read effects should be filtered away. > > Shouldn't it somehow just do what it allready are capable o

Re: [PERFORM] Queryplan within FTS/GIN index -search.

2009-10-23 Thread jesper
> On Fri, 2009-10-23 at 07:18 +0200, Jesper Krogh wrote: >> > In effect, what you want are words that aren't searched (or stored) in >> > the index, but are included in the tsvector (so the RECHECK still >> > works). That sounds like it would solve your problem and it would >> reduce >> > index siz

Re: [PERFORM] Table Clustering & Time Range Queries

2009-10-23 Thread Scara Maccai
> Also, you might want to experiment with things like > pg_reorg: Do you happen to know if that works with 8.4? -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance