Re: [PERFORM] IN or EXISTS

2011-09-22 Thread Jeff Davis
On Wed, 2011-08-31 at 09:33 +0800, Craig Ringer wrote: > On the other hand, the `IN' subquery is uncorrelated needs only run > once, where the `EXISTS' subquery is correlated and has to run once for > every outer record. If the EXISTS looks semantically similar to an IN (aside from NULL semantic

Re: [PERFORM] Query optimization using order by and limit

2011-09-22 Thread Michael Viscuso
Stephen, Yes, I couldn't agree more. The next two things I will be looking at very carefully are the timestamps and indexes. I will reply to this post if either dramatically helps. Thanks again for all your help. My eyes were starting to bleed from staring at explain logs! Mike On Thu, Sep 2

Re: [PERFORM] Query optimization using order by and limit

2011-09-22 Thread Stephen Frost
Mike, * Michael Viscuso (michael.visc...@getcarbonblack.com) wrote: > I spent the better part of the day implementing an application layer > nested loop and it seems to be working well. Of course it's a little > slower than a Postgres only solution because it has to pass data back > and forth for

Re: [PERFORM] Optimizing Trigram searches in PG 9.1

2011-09-22 Thread Dave Crooke
Depending on your needs, you might consider putting the data into a columnar text search engine like Lucene, having it return the integer id's which can then be used for row lookups in PG. On Thu, Sep 22, 2011 at 11:40 AM, Jonathan Bartlett < jonathan.l.bartl...@gmail.com> wrote: > I am working o

Re: [PERFORM] Query optimization using order by and limit

2011-09-22 Thread Michael Viscuso
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Stephen, I spent the better part of the day implementing an application layer nested loop and it seems to be working well. Of course it's a little slower than a Postgres only solution because it has to pass data back and forth for each daily table

[PERFORM] Optimizing Trigram searches in PG 9.1

2011-09-22 Thread Jonathan Bartlett
I am working on a fuzzy search of a large dataset. Basically, it is a list of all of the songs, albums, artists, movies, and celebrities exported from Freebase. Anyway, I was hoping to get a fuzzy search that was nearly as fast as the full-text search with the new nearest-neighbor GIST indexes, b

Re: [PERFORM] Query optimization using order by and limit

2011-09-22 Thread Stephen Frost
* Michael Viscuso (michael.visc...@getcarbonblack.com) wrote: > Adding the final condition hosts_guid = '2007075705813916178' is what > ultimately kills it http://explain.depesz.com/s/8zy. By adding the > host_guid, it spends considerably more time in the older tables than > without this condition

Re: [PERFORM] Query optimization using order by and limit

2011-09-22 Thread Michael Viscuso
Thanks Ken, I'm discussing with my coworker how to best make that change *as we speak*. Do you think this will also resolve the original issue I'm seeing where the query doesn't "limit out properly" and spends time in child tables that won't yield any results? I was hoping that by using the chec

Re: [PERFORM] Query optimization using order by and limit

2011-09-22 Thread k...@rice.edu
On Wed, Sep 21, 2011 at 11:22:53PM -0400, Tom Lane wrote: > Michael Viscuso writes: > > Greg/Tom, you are correct, these columns should be modified to whatever > > is easiest for Postgres to recognize 64-bit unsigned integers. Would > > you still recommend bigint for unsigned integers? I likely