Re: [HACKERS] WITH RECUSIVE patches 0723

2008-08-02 Thread Michael Meskes
On Sat, Aug 02, 2008 at 12:33:38AM -0400, Tom Lane wrote: > grammar. Right now the situation is that Michael Meskes makes a manual > cleanup pass every so often :-(. I would like to see that get automated > sooner rather than later, but in the near term it is not the I received a very promising

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-08-02 Thread Michael Meskes
On Sat, Aug 02, 2008 at 12:35:55AM +0100, Andrew Gierth wrote: > One more oversight: the patch isn't updating the ECPG preproc.y other > than trivially, so WITH queries aren't working in ecpg: I'll take care of this as soon as the patch settles down enough, gets included into our CVS or into its o

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-08-01 Thread Tom Lane
Andrew Gierth <[EMAIL PROTECTED]> writes: > One more oversight: the patch isn't updating the ECPG preproc.y other > than trivially, so WITH queries aren't working in ecpg: > test.pgc:111: ERROR: syntax error at or near "recursive" By and large we don't expect core patches to worry about fixing th

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-08-01 Thread Andrew Gierth
> "Tatsuo" == Tatsuo Ishii <[EMAIL PROTECTED]> writes: >> At David's request I've been looking through this patch. >> >> Regarding documentation: if it would help, I can write some; I >> have already made a start on writing down what is going on >> internally in order to understand it my

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-29 Thread David Fetter
On Mon, Jul 28, 2008 at 02:49:01PM -0400, Tom Lane wrote: > Andrew Gierth <[EMAIL PROTECTED]> writes: > > "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: > > Tom> That whole business of using the EState to pass tuplestores back > > Tom> and forth looks fundamentally broken to me anyway; there's jus

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Pavel Stehule
Hello 2008/7/29 Martijn van Oosterhout <[EMAIL PROTECTED]>: > On Mon, Jul 28, 2008 at 07:57:16PM +0100, Andrew Gierth wrote: >> Which will be a serious pessimization in many common cases if you do >> it all the time. Googling for examples of non-recursive WITH queries >> shows that it is very wide

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: > Martijn van Oosterhout <[EMAIL PROTECTED]> writes: >> Since the problem is using the result of a WITH clause more than >> once, would it be sufficient to simply detect that case and bail? >> You don't want materialisation is most cases, the

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> My thought is that we could optimize away materialization in cases where >> we can tell it's not needed (no volatile functions and/or no multiple >> scans of the subquery). But not being able to do it means we've >> implemented the fe

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Dunstan
Tom Lane wrote: My thought is that we could optimize away materialization in cases where we can tell it's not needed (no volatile functions and/or no multiple scans of the subquery). But not being able to do it means we've implemented the feature incorrectly. I'm

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
Martijn van Oosterhout <[EMAIL PROTECTED]> writes: > Since the problem is using the result of a WITH clause more than once, > would it be sufficient to simply detect that case and bail? You don't > want materialisation is most cases, there's just a few where it is > needed. Really? I tried googli

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Martijn van Oosterhout
On Mon, Jul 28, 2008 at 07:57:16PM +0100, Andrew Gierth wrote: > Which will be a serious pessimization in many common cases if you do > it all the time. Googling for examples of non-recursive WITH queries > shows that it is very widely used for clarity or convenience, in > contexts where you _don't

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Gregory" == Gregory Stark <[EMAIL PROTECTED]> writes: Gregory> I just wonder where all these examples of real-world queries Gregory> were when I posted this patch and asked for such feedback Gregory> originally. sigh. Gregory> In any case I think we've already made this decision. If we

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Gregory Stark
"Andrew Gierth" <[EMAIL PROTECTED]> writes: >> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: > > Tom> This isn't going to be a particularly simple fix :-(. The basic > Tom> implementation clearly ought to be to dump the result of the > Tom> subquery into a tuplestore and then have the uppe

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: Tom> We are not in the business of getting spec-required semantics Tom> 80% right; I guess that's why we still fold identifiers to lowercase, why our timestamp implementation differs from the standard, why we used to require AS for select-list

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
Andrew Gierth <[EMAIL PROTECTED]> writes: > Recursive WITH queries that self-join the recursion result seem to be > rare in practice. We are not in the business of getting spec-required semantics 80% right; and as I took pains to point out to start with, there are good functionality reasons to wan

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: [snip spec] Just out of curiosity, since I don't have a copy of the spec handy, how does the language for WITH compare to that for views? Tom> I think this is a "must fix" because of the point about volatile Tom> functions --- changing it la

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
Andrew Gierth <[EMAIL PROTECTED]> writes: > "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: > Tom> That whole business of using the EState to pass tuplestores back > Tom> and forth looks fundamentally broken to me anyway; there's just > Tom> no way it'll be certain to link the right nodes together

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Tom" == Tom Lane <[EMAIL PROTECTED]> writes: >> I think it needs this change in addition; without it, incorrect >> results are returned when you reference a recursive view from >> within the recursive query, due to the RecursionScan nodes >> becoming linked to the wrong tuplestores. T

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
Andrew Gierth <[EMAIL PROTECTED]> writes: > "Tatsuo" == Tatsuo Ishii <[EMAIL PROTECTED]> writes: > Tatsuo> Included patches from Yoshiyuki should fix 1) and 2). I also > Tatsuo> add your SQLs to the regression test. Thanks. > I think it needs this change in addition; without it, incorrect > resu

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: >> This isn't going to be a particularly simple fix :-(. The basic >> implementation clearly ought to be to dump the result of the subquery >> into a tuplestore and then have the upper level read out from that. >> H

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: "Tom Lane" <[EMAIL PROTECTED]> writes: > I think what this is saying is that the subquery defined by a WITH > clause is to be evaluated only once, even if it is referenced in > multiple places in the upper query. This is sensible because if there > is no su

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Tatsuo" == Tatsuo Ishii <[EMAIL PROTECTED]> writes: Tatsuo> Included patches from Yoshiyuki should fix 1) and 2). I also Tatsuo> add your SQLs to the regression test. Thanks. I think it needs this change in addition; without it, incorrect results are returned when you reference a recursi

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tom Lane
I spent some time reading the SQL spec over the weekend, and I believe I've identified a fairly serious problem in the WITH patch. SQL99 7.12 General Rule 1 is 1) If a non-recursive is specified, then: a) For every WLE, let WQN be the immediately contained in WLE. Let WQ

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Andrew Gierth
> "Tatsuo" == Tatsuo Ishii <[EMAIL PROTECTED]> writes: >> This behaviour is clearly intentional, since the entire mechanism of >> estate-> es_disallow_tuplestore exists for no other reason, but it >> seems to me to be clearly wrong. What is the justification for it? Tatsuo> Yes, this is d

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Robert Haas
> Now we think that we were wrong. This type of query should run into > infinit recursion and it's user's responsibility that he does not make > such a query. > > Another idea would be prohibiting *any* outer joins in the recursive > term (DB2 style), but this may be overkill. Even if you were to

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-28 Thread Tatsuo Ishii
> At David's request I've been looking through this patch. > > Regarding documentation: if it would help, I can write some; I have > already made a start on writing down what is going on internally in > order to understand it myself. > > I've found three more bugs so far: > > 1) > > create view

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-27 Thread Pavel Stehule
Hello I played with CTE and I have to say, it's great feature - great work. One questions - can I enforce materialisation of query? It would be usefull for some analytical queries like: with tmp as (select a, sum(b) as b from test) select * from tmp union all select 'all', sum(b) from tmp; reg

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-27 Thread Tatsuo Ishii
> At David's request I've been looking through this patch. > > Regarding documentation: if it would help, I can write some; I have > already made a start on writing down what is going on internally in > order to understand it myself. Thanks. There was some docs written in Japanese by Yoshiyuki. R

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-26 Thread Andrew Gierth
At David's request I've been looking through this patch. Regarding documentation: if it would help, I can write some; I have already made a start on writing down what is going on internally in order to understand it myself. I've found three more bugs so far: 1) create view v2(id) as values (1);

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-25 Thread Tatsuo Ishii
> Thanks for the patch :) > > Now, I get a different problem, this time with the following code > intended to materialize paths on the fly and summarize down to a > certain depth in a tree: > > CREATE TABLE tree( > id INTEGER PRIMARY KEY, > parent_id INTEGER REFERENCES tree(id) > ); > >

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-24 Thread Tatsuo Ishii
> Now, I get a different problem, this time with the following code > intended to materialize paths on the fly and summarize down to a > certain depth in a tree: > > CREATE TABLE tree( > id INTEGER PRIMARY KEY, > parent_id INTEGER REFERENCES tree(id) > ); > > INSERT INTO tree > VALUES (1,

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-24 Thread David Fetter
On Thu, Jul 24, 2008 at 01:55:37PM +0900, Tatsuo Ishii wrote: > > Program received signal SIGSEGV, Segmentation fault. > > Thanks for the report. Here is the new patches from Yoshiyuki. Thanks for the patch :) Now, I get a different problem, this time with the following code intended to material

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-24 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > Reviewers, please let me know if you find problems with the > patches. If none, I would like to commit this weekend. Given that everyone who has tested this has found a different way to crash it, and that the frequency of crash reports shows no signs of s

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
> On Wed, Jul 23, 2008 at 10:59:20AM -0400, Tom Lane wrote: > > Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > > Reviewers, please let me know if you find problems with the > > > patches. If none, I would like to commit this weekend. > > > > Has this patch actually been reviewed yet? The only repor

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
> On Wed, Jul 23, 2008 at 10:59:20AM -0400, Tom Lane wrote: > > Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > > Reviewers, please let me know if you find problems with the > > > patches. If none, I would like to commit this weekend. > > > > Has this patch actually been reviewed yet? The only repor

Re: [PATCHES] [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread David Fetter
On Wed, Jul 23, 2008 at 10:59:20AM -0400, Tom Lane wrote: > Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > Reviewers, please let me know if you find problems with the > > patches. If none, I would like to commit this weekend. > > Has this patch actually been reviewed yet? The only reports I've > se

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
> Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > Reviewers, please let me know if you find problems with the > > patches. If none, I would like to commit this weekend. > > Has this patch actually been reviewed yet? The only reports I've > seen are from testing; nothing from anyone actually reading

Re: [HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > Reviewers, please let me know if you find problems with the > patches. If none, I would like to commit this weekend. Has this patch actually been reviewed yet? The only reports I've seen are from testing; nothing from anyone actually reading the code. I

[HACKERS] WITH RECUSIVE patches 0723

2008-07-23 Thread Tatsuo Ishii
> Hi, > > Here is the lastest WITH RECURSIVE patches against 2007/07/17 CVS (CVS > HEAD won't compile for me). > > This version includes regression tests and is almost ready for commit > IMO. I pulled fresh CVS HEAD and it seems the problem is gone. Here is the lastest WITH RECURSIVE patches ag