Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Robert Haas
On Fri, Dec 18, 2009 at 7:32 PM, Tom Lane wrote: > Robert Haas writes: >> If at least one column in the subselect is strict, you can rewrite it >> that way yourself, but the optimizer won't do it. I wish it did, but I >> don't wish it badly enough to have written the code myself, and >> apparentl

Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Tom Lane
Robert Haas writes: > If at least one column in the subselect is strict, you can rewrite it > that way yourself, but the optimizer won't do it. I wish it did, but I > don't wish it badly enough to have written the code myself, and > apparently neither does anyone else. I was thinking about this e

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Robert Haas
On Fri, Dec 18, 2009 at 10:51 AM, Sigurgeir Gunnarsson wrote: > The intention was never to talk down postgresql but rather trying to get > some explanation of this difference so that I could do the proper changes. > > After having read the link from Euler's post, which I oversaw, I have > managed

Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Robert Haas
2009/12/18 Grzegorz Jaśkiewicz : > 2009/12/18 Robert Haas : >> 2009/12/18 Grzegorz Jaśkiewicz : >>> On Fri, Dec 18, 2009 at 2:18 PM, Robert Haas wrote: >>> NOT IN is the only that really kills you as far as optimization is concerned.  IN can be transformed to a join.  NOT IN forces a NOT

Re: [PERFORM] Idea how to get rid of Bitmap Heap Scan

2009-12-18 Thread Greg Stark
On Fri, Dec 18, 2009 at 4:18 PM, Michael N. Mikhulya wrote: > Thank you very much. I catch the point why it is done so. > > But I'm curious whether it is still possible to don't fetch data from > files table just because inappropriate ids (e.g. removed ones) will > not produce any wrong effect jus

Re: [PERFORM] Idea how to get rid of Bitmap Heap Scan

2009-12-18 Thread Michael N. Mikhulya
Thank you very much. I catch the point why it is done so. But I'm curious whether it is still possible to don't fetch data from files table just because inappropriate ids (e.g. removed ones) will not produce any wrong effect just because them indirectly "checked" on downloads table? Here I mean th

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Sigurgeir Gunnarsson
The intention was never to talk down postgresql but rather trying to get some explanation of this difference so that I could do the proper changes. After having read the link from Euler's post, which I oversaw, I have managed to shorten the import time. My problem was with the indexes. I was able

Re: [PERFORM] Idea how to get rid of Bitmap Heap Scan

2009-12-18 Thread Matthew Wakeling
On Fri, 18 Dec 2009, Michael N. Mikhulya wrote: The problem here is that we are forced to fetch "files" in Bitmap Heap Scan. But actually there is no need for the whole "files" record. The necessary data is only "files" ids. The idea is to avoid fetching data from "files" table, and get the ids

[PERFORM] Idea how to get rid of Bitmap Heap Scan

2009-12-18 Thread Michael N. Mikhulya
Hello! There are many questions on internet about whether it is possible to optimize "Bitmap Heap Scan" somehow without answer, so seems like problem is rather important. The query I need to optimize is: EXPLAIN SELECT date_trunc('day', d.created_at) AS day, COUNT(*) AS download FROM downloads d

Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Grzegorz Jaśkiewicz
2009/12/18 Robert Haas : > 2009/12/18 Grzegorz Jaśkiewicz : >> On Fri, Dec 18, 2009 at 2:18 PM, Robert Haas wrote: >> >>> NOT IN is the only that really kills you as far as optimization is >>> concerned.  IN can be transformed to a join.  NOT IN forces a NOT >>> (subplan)-type plan, which bites -

Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Robert Haas
2009/12/18 Grzegorz Jaśkiewicz : > On Fri, Dec 18, 2009 at 2:18 PM, Robert Haas wrote: > >> NOT IN is the only that really kills you as far as optimization is >> concerned.  IN can be transformed to a join.  NOT IN forces a NOT >> (subplan)-type plan, which bites - hard. > > in a well designed dat

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Robert Haas
On Fri, Dec 18, 2009 at 7:46 AM, Sigurgeir Gunnarsson wrote: > I hope the issue is still open though I haven't replied to it before. > > Euler mentioned that I did not provide any details about my system. I'm > using version 8.3 and with most settings default on an old machine with 2 GB > of mem.

Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Grzegorz Jaśkiewicz
On Fri, Dec 18, 2009 at 2:18 PM, Robert Haas wrote: > NOT IN is the only that really kills you as far as optimization is > concerned.  IN can be transformed to a join.  NOT IN forces a NOT > (subplan)-type plan, which bites - hard. in a well designed database (read: not abusing NULLs) - it can b

Re: [PERFORM] Automatic optimization of IN clauses via INNER JOIN

2009-12-18 Thread Robert Haas
On Thu, Dec 17, 2009 at 9:20 PM, Craig Ringer wrote: > On 17/12/2009 11:57 PM, Tom Lane wrote: >> >> Thomas Hamilton  writes: >>> >>> But in our testing under the same optimization and conditions INNER JOIN >>> is significantly outperforming IN. >> >> [ shrug... ]  You haven't provided any details

Re: [PERFORM] Issues with \copy from file

2009-12-18 Thread Sigurgeir Gunnarsson
I hope the issue is still open though I haven't replied to it before. Euler mentioned that I did not provide any details about my system. I'm using version 8.3 and with most settings default on an old machine with 2 GB of mem. The table definition is simple, four columns; id, value, x, y where id