Re: [HACKERS] Aggregate ORDER BY patch

2009-12-20 Thread Pavel Stehule
2009/12/19 Marko Tiikkaja : > On 2009-12-15 23:10 +0200, Tom Lane wrote: >> >> Andrew Gierth  writes: >>> >>> Notice that there are cases where agg(distinct x order by x) is >>> nondeterministic while agg(distinct x order by x,y) is deterministic. >> >> Well, I think what you're really describing i

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-19 Thread Tom Lane
Marko Tiikkaja writes: > On 2009-12-15 23:10 +0200, Tom Lane wrote: >> If we really wanted to take the above seriously, my opinion is that >> we ought to introduce DISTINCT ON in aggregates. > FWIW, in my opinion the idea behind this patch is to not fall back on > hacks like that. This patch al

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-19 Thread Marko Tiikkaja
On 2009-12-15 23:10 +0200, Tom Lane wrote: Andrew Gierth writes: Notice that there are cases where agg(distinct x order by x) is nondeterministic while agg(distinct x order by x,y) is deterministic. Well, I think what you're really describing is a case where you're using the wrong sort opclas

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-15 Thread Tom Lane
Andrew Gierth writes: > Notice that there are cases where agg(distinct x order by x) is > nondeterministic while agg(distinct x order by x,y) is deterministic. Well, I think what you're really describing is a case where you're using the wrong sort opclass. If the aggregate can distinguish two va

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-15 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> Query-level DISTINCT shouldn't allow columns in the order by that >> aren't in the select list because those columns _do not exist_ at >> the point that ordering logically takes place (even though in the >> implementation, they might). >> This isn't the ca

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-15 Thread Tom Lane
Andrew Gierth writes: > Query-level DISTINCT shouldn't allow columns in the order by that > aren't in the select list because those columns _do not exist_ at the > point that ordering logically takes place (even though in the > implementation, they might). > This isn't the case for aggregate orde

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-15 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> Andrew Gierth writes: >> Updated version of the aggregate order by patch. Tom> Applied with some editorialization. The main change I made was Tom> to get rid of all the ad-hoc DISTINCT handling in parse_agg.c Tom> and use transformDistinctClause() inst

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-15 Thread Tom Lane
Andrew Gierth writes: > Updated version of the aggregate order by patch. Applied with some editorialization. The main change I made was to get rid of all the ad-hoc DISTINCT handling in parse_agg.c and use transformDistinctClause() instead. This exposed what I believe to be a bug in the submitt

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-14 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> and I would also expect there to be a nonzero performance hit > Tom> from the extra TargetEntry expression nodes, even when the > Tom> feature is not in use. > I tested for that and couldn't reliably detect any (certainly <2% > on count

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-14 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> Updated version of the aggregate order by patch. Tom> I'm starting to look at this now. I find it rather bizarre to Tom> merge both the actual arguments of an aggregate and the optional Tom> ORDER BY expressions into a single targetlist. It doesn't seem

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-14 Thread Tom Lane
Andrew Gierth writes: > Updated version of the aggregate order by patch. I'm starting to look at this now. I find it rather bizarre to merge both the actual arguments of an aggregate and the optional ORDER BY expressions into a single targetlist. It doesn't seem like that would be an especially

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-03 Thread Alvaro Herrera
Hitoshi Harada escribió: > I found only trivial favors such like that a blank line is added > around line 595 in the patch, and "proj" in peraggstate sounds a > little weird to me because of surrounding "evaldesc" and "evalslot" > ("evalproj" seems better to me). Also catversion update doesn't mea

Re: [HACKERS] Aggregate ORDER BY patch

2009-12-02 Thread Hitoshi Harada
2009/11/30 Andrew Gierth : > Updated version of the aggregate order by patch. > > Includes docs + regression tests all in the same patch. > > Changes: > >  - removed SortGroupClause.implicit as per review comments, >    replacing it with separate lists for Aggref.aggorder and >    Aggref.aggdistinc

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-30 Thread Andrew Gierth
Updated version of the aggregate order by patch. Includes docs + regression tests all in the same patch. Changes: - removed SortGroupClause.implicit as per review comments, replacing it with separate lists for Aggref.aggorder and Aggref.aggdistinct. - Refactored in order to move the

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> What about error handling? If the user specifies agg(distinct x) >> where x is not sortable, do we leave it to the planner to detect >> that (which means not reporting the error position?) Tom> Well, at the moment there's only going to be a sort-based Tom>

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Hitoshi Harada
2009/11/16 Andrew Gierth : >> "Hitoshi" == Hitoshi Harada writes: > >  >> What case exactly would you consider an error? When an order by >  >> expression references a lower (more deeply nested) query level >  >> than any of the actual arguments? > >  Hitoshi> It's only that I felt not intuiti

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Tom Lane
Andrew Gierth writes: > What about error handling? If the user specifies agg(distinct x) where > x is not sortable, do we leave it to the planner to detect that (which > means not reporting the error position?) Well, at the moment there's only going to be a sort-based implementation, so I don't o

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> I agree with Hitoshi that this seems to be a hack to deal with the [snip] So copying the way that SELECT DISTINCT works would be the way to go? i.e. keeping separate lists in the parse node for sorting and distinct? What about error handling? If the user s

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Andrew Gierth
> "Hitoshi" == Hitoshi Harada writes: >> What case exactly would you consider an error? When an order by >> expression references a lower (more deeply nested) query level >> than any of the actual arguments? Hitoshi> It's only that I felt not intuitive. To me, arguments are Hitoshi> reg

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Tom Lane
Hitoshi Harada writes: > 2009/11/16 Andrew Gierth : > "Hitoshi" == Hitoshi Harada writes: >>>  Hitoshi>  - SortGroupClause.implicit >>>  Hitoshi> "implicit" member was added in SortGroupClause. I didn't >>>  Hitoshi> find clear reason to add this. Could you show a case to >>>  Hitoshi> clarify th

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Hitoshi Harada
2009/11/16 Andrew Gierth : >> "Hitoshi" == Hitoshi Harada writes: > >  Hitoshi> Questions here: >  Hitoshi> - agglevelsup? > What case exactly would you consider an error? When an order by > expression references a lower (more deeply nested) query level than > any of the actual arguments? It'

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Greg Stark
On Sun, Nov 15, 2009 at 11:23 PM, Andrew Gierth wrote: > Future performance enhancements (which I have no particular plans to > tackle) would involve having the planner consult the desired aggregate > orderings and estimating the cost of sorting as opposed to the cost of > producing a plan with th

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Andrew Gierth
> "Andrew" == Andrew Gierth writes: Andrew> Performance. Andrew> tuplesort_getdatum etc. seems to be substantially faster than Andrew> tuplesort_gettupleslot especially for the case where you're Andrew> sorting a pass-by-value datum such as an integer (since the Andrew> datum is then st

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Andrew Gierth
> "Hitoshi" == Hitoshi Harada writes: Hitoshi> Questions here: Hitoshi> - agglevelsup? Hitoshi> We have aggregate capability that all arguments from upper Hitoshi> level query in downer level aggregate makes aggregate call Hitoshi> itself to upper level call, as a constant value in down

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-15 Thread Hitoshi Harada
Here's my first review. The patch was in context diff format and applied cleanly with a little 3 hunks in parse_expr.c. make succeeded without any warnings and make check passed all 121 tests. It implements as advertised, following SQL spec with extension of both DISTINCT clause and ORDER BY clau

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Greg Stark
On Fri, Nov 13, 2009 at 5:09 PM, Tom Lane wrote: > Quite.  This is another instance of the thing I complained of before, > that the SQL committee likes to define the behavior of specific > aggregates instead of inducing a generic aggregate-behavior definition. I think this makes sense from the po

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Hitoshi Harada
2009/11/14 Tom Lane : > Andrew Gierth writes: >> "Peter" == Peter Eisentraut writes: >>  Peter> This is exactly the syntax that is in the spec AFAICT. > >> Right. The spec defines this syntax for array_agg and xmlagg (only). > > Cool, I had forgotten that they added that in the latest revisions.

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Hitoshi Harada
2009/11/14 Andrew Gierth : >> "Heikki" == Heikki Linnakangas >> writes: > >  >> No artificial restrictions are imposed on what syntactical >  >> combinations are allowed. However, ORDER BY is not allowed with >  >> aggregates used as window functions (as per the existing >  >> restriction

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Andrew Gierth
> "Heikki" == Heikki Linnakangas > writes: >> No artificial restrictions are imposed on what syntactical >> combinations are allowed. However, ORDER BY is not allowed with >> aggregates used as window functions (as per the existing >> restriction on DISTINCT). Heikki> How is this d

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Andres Freund
On Friday 13 November 2009 16:35:08 Tom Lane wrote: > Greg Stark writes: > > On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas > > > > wrote: > >> Andrew Gierth wrote: > >>> Herewith a patch to implement agg(foo ORDER BY bar) with or without > >>> DISTINCT, etc. > >> > >> What does that mean? A

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Tom Lane
Andrew Gierth writes: > "Peter" == Peter Eisentraut writes: > Peter> This is exactly the syntax that is in the spec AFAICT. > Right. The spec defines this syntax for array_agg and xmlagg (only). Cool, I had forgotten that they added that in the latest revisions. I withdraw the complaint that t

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Andrew Gierth
> "Peter" == Peter Eisentraut writes: >> I'm not entirely convinced that adding ORDER BY here is a good idea, >> partly because it goes so far beyond the spec Peter> This is exactly the syntax that is in the spec AFAICT. Right. The spec defines this syntax for array_agg and xmlagg (only)

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Andrew Gierth
> "Heikki" == Heikki Linnakangas > writes: >> Herewith a patch to implement agg(foo ORDER BY bar) with or >> without DISTINCT, etc. Heikki> What does that mean? Aggregate functions are supposed to be Heikki> commutative, right? The SQL spec defines two non-commutative aggregates th

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Robert Haas
On Fri, Nov 13, 2009 at 10:35 AM, Tom Lane wrote: > Greg Stark writes: >> On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas >> wrote: >>> Andrew Gierth wrote: Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. >>> >>> What does that mean? Aggregate fun

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Peter Eisentraut
On fre, 2009-11-13 at 10:35 -0500, Tom Lane wrote: > I'm not entirely convinced that adding ORDER BY here is a good idea, > partly because it goes so far beyond the spec This is exactly the syntax that is in the spec AFAICT. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Peter Eisentraut
On fre, 2009-11-13 at 10:01 -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On fre, 2009-11-13 at 03:16 +, Andrew Gierth wrote: > >> Caveat: as discussed earlier, this patch changes the behaviour of > >> array_agg(DISTINCT x) when applied to NULL inputs. Formerly, the NULLs > >> were unc

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Tom Lane
Greg Stark writes: > On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas > wrote: >> Andrew Gierth wrote: >>> Herewith a patch to implement agg(foo ORDER BY bar) with or without >>> DISTINCT, etc. >> >> What does that mean? Aggregate functions are supposed to be commutative, >> right? > We cert

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Tom Lane
Peter Eisentraut writes: > On fre, 2009-11-13 at 03:16 +, Andrew Gierth wrote: >> Caveat: as discussed earlier, this patch changes the behaviour of >> array_agg(DISTINCT x) when applied to NULL inputs. Formerly, the NULLs >> were unconditionally skipped; now, they are treated just like DISTINC

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Peter Eisentraut
On fre, 2009-11-13 at 03:16 +, Andrew Gierth wrote: > Caveat: as discussed earlier, this patch changes the behaviour of > array_agg(DISTINCT x) when applied to NULL inputs. Formerly, the NULLs > were unconditionally skipped; now, they are treated just like DISTINCT > or GROUP BY normally do. T

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Greg Stark
On Fri, Nov 13, 2009 at 7:54 AM, Heikki Linnakangas wrote: > Andrew Gierth wrote: >> Herewith a patch to implement agg(foo ORDER BY bar) with or without >> DISTINCT, etc. > > What does that mean? Aggregate functions are supposed to be commutative, > right? We certainly have non-commutative agggre

Re: [HACKERS] Aggregate ORDER BY patch

2009-11-13 Thread Heikki Linnakangas
Andrew Gierth wrote: > Herewith a patch to implement agg(foo ORDER BY bar) with or without > DISTINCT, etc. What does that mean? Aggregate functions are supposed to be commutative, right? > No artificial restrictions are imposed on what > syntactical combinations are allowed. However, ORDER BY is

[HACKERS] Aggregate ORDER BY patch

2009-11-12 Thread Andrew Gierth
Herewith a patch to implement agg(foo ORDER BY bar) with or without DISTINCT, etc. No artificial restrictions are imposed on what syntactical combinations are allowed. However, ORDER BY is not allowed with aggregates used as window functions (as per the existing restriction on DISTINCT). Included