Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-09-03 Thread Hitoshi Harada
2009/9/3 Pavel Stehule : > 2009/9/3 Joshua Tolley : >> On Thu, Sep 03, 2009 at 01:19:25AM +0400, Олег Царев wrote: >>> After week-lengthed investigation, now i 'm sure - my level of >>> qualification not enough for implementation task "GROUPING SETS". >>> I require documentation about the executor

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-09-02 Thread Pavel Stehule
2009/9/3 Joshua Tolley : > On Thu, Sep 03, 2009 at 01:19:25AM +0400, Олег Царев wrote: >> After week-lengthed investigation, now i 'm sure - my level of >> qualification not enough for implementation task "GROUPING SETS". >> I require documentation about the executor and the planner, i can't >> und

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-09-02 Thread Pavel Stehule
2009/9/2 Олег Царев : > After week-lengthed investigation, now i 'm sure - my level of > qualification not enough for implementation task "GROUPING SETS". > I require documentation about the executor and the planner, i can't > understand scheme of work by source code. > Many code, many cases, but v

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-09-02 Thread Joshua Tolley
On Thu, Sep 03, 2009 at 01:19:25AM +0400, Олег Царев wrote: > After week-lengthed investigation, now i 'm sure - my level of > qualification not enough for implementation task "GROUPING SETS". > I require documentation about the executor and the planner, i can't > understand scheme of work by sourc

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-09-02 Thread Олег Царев
After week-lengthed investigation, now i 'm sure - my level of qualification not enough for implementation task "GROUPING SETS". I require documentation about the executor and the planner, i can't understand scheme of work by source code. Many code, many cases, but very little information "what is

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Pavel Stehule
2009/8/14 Олег Царев : > 2009/8/14 Hitoshi Harada : >> 2009/8/14 Pavel Stehule : >>> 2009/8/13 Hitoshi Harada : 2009/8/14 Pavel Stehule : > I prefered using CTE, because this way was the most short to small > bugs less prototype - with full functionality. You could make it by

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Олег Царев
2009/8/14 Hitoshi Harada : > 2009/8/14 Pavel Stehule : >> 2009/8/13 Hitoshi Harada : >>> 2009/8/14 Pavel Stehule : I prefered using CTE, because this way was the most short to small bugs less prototype - with full functionality. >>> >>> You could make it by query rewriting, but as you say

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Hitoshi Harada
2009/8/14 Pavel Stehule : > 2009/8/13 Hitoshi Harada : >> 2009/8/14 Pavel Stehule : >>> I prefered using CTE, because this way was the most short to small >>> bugs less prototype - with full functionality. >> >> You could make it by query rewriting, but as you say the best cleanest >> way is total

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Pavel Stehule
2009/8/13 Hitoshi Harada : > 2009/8/14 Pavel Stehule : >> I prefered using CTE, because this way was the most short to small >> bugs less prototype - with full functionality. > > You could make it by query rewriting, but as you say the best cleanest > way is total refactoring of existing nodeAgg. H

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Hitoshi Harada
2009/8/14 Pavel Stehule : > I prefered using CTE, because this way was the most short to small > bugs less prototype - with full functionality. You could make it by query rewriting, but as you say the best cleanest way is total refactoring of existing nodeAgg. How easy to implement is not convinci

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Pavel Stehule
2009/8/13 Олег Царев : > 2009/8/13 Hitoshi Harada : >> 2009/8/8 Alvaro Herrera : >>> Олег Царев escribió: Hello all! If no one objecte (all agree, in other say) i continue work on patch - particulary, i want support second strategy (tuple store instead of hash-table) for save or

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Hitoshi Harada
2009/8/14 Олег Царев : > All rights, exclude >> Because GROUP BY we have today is a subset of GROUPING SETS by >> definition, I suppose we'll refactor nodeAgg.c so that it is allowed >> to take multiple group definitions. And we must support both of >> HashAgg and GroupAgg. For HashAgg, it is easie

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Pavel Stehule
2009/8/13 Hitoshi Harada : > 2009/8/8 Alvaro Herrera : >> Олег Царев escribió: >>> Hello all! >>> If no one objecte (all agree, in other say) i continue work on patch - >>> particulary, i want support second strategy (tuple store instead of >>> hash-table) for save order of source (more cheap solut

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Олег Царев
2009/8/13 Hitoshi Harada : > 2009/8/8 Alvaro Herrera : >> Олег Царев escribió: >>> Hello all! >>> If no one objecte (all agree, in other say) i continue work on patch - >>> particulary, i want support second strategy (tuple store instead of >>> hash-table) for save order of source (more cheap solut

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-13 Thread Hitoshi Harada
2009/8/8 Alvaro Herrera : > Олег Царев escribió: >> Hello all! >> If no one objecte (all agree, in other say) i continue work on patch - >> particulary, i want support second strategy (tuple store instead of >> hash-table) for save order of source (more cheap solution in case with >> grouping sets

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-08-07 Thread Alvaro Herrera
Олег Царев escribió: > Hello all! > If no one objecte (all agree, in other say) i continue work on patch - > particulary, i want support second strategy (tuple store instead of > hash-table) for save order of source (more cheap solution in case with > grouping sets + order by), investigate and brai

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-13 Thread David Fetter
On Wed, May 13, 2009 at 03:12:51PM +0200, Pavel Stehule wrote: > 2009/5/13 Joshua Tolley : > > On Wed, May 13, 2009 at 06:29:41AM +0200, Pavel Stehule wrote: > >> 2009/5/13 Joshua Tolley : > >> > On Tue, May 12, 2009 at 11:20:14PM +0200, Pavel Stehule wrote: > >> >> this patch has some bugs but it

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-13 Thread Pavel Stehule
2009/5/13 Joshua Tolley : > On Wed, May 13, 2009 at 06:29:41AM +0200, Pavel Stehule wrote: >> 2009/5/13 Joshua Tolley : >> > On Tue, May 12, 2009 at 11:20:14PM +0200, Pavel Stehule wrote: >> >> this patch has some bugs but it is good prototype (it's more stable >> >> than old patch): >> > >> > I'm

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-13 Thread Joshua Tolley
On Wed, May 13, 2009 at 06:29:41AM +0200, Pavel Stehule wrote: > 2009/5/13 Joshua Tolley : > > On Tue, May 12, 2009 at 11:20:14PM +0200, Pavel Stehule wrote: > >> this patch has some bugs but it is good prototype (it's more stable > >> than old patch): > > > > I'm not sure if you're at the point th

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-13 Thread Tom Lane
Robert Haas writes: > But that leads me to a question - does the existing HashAggregate code > make any attempt to obey work_mem? No. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-13 Thread Robert Haas
On Tue, May 12, 2009 at 2:21 AM, Pavel Stehule wrote: >> Moreover, I guess you don't even need to buffer tuples to aggregate by >> different keys. What you have to do is only to prepare more than one >> hash tables (, or set up sort order if the plan detects hash table is >> too large to fit in th

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-12 Thread Pavel Stehule
2009/5/13 Joshua Tolley : > On Tue, May 12, 2009 at 11:20:14PM +0200, Pavel Stehule wrote: >> this patch has some bugs but it is good prototype (it's more stable >> than old patch): > > I'm not sure if you're at the point that you're interested in bug reports, but > here's something that didn't beh

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-12 Thread Joshua Tolley
On Tue, May 12, 2009 at 11:20:14PM +0200, Pavel Stehule wrote: > this patch has some bugs but it is good prototype (it's more stable > than old patch): I'm not sure if you're at the point that you're interested in bug reports, but here's something that didn't behave as expected: 5432 j...@josh*#

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-12 Thread Олег Царев
Hello all! If no one objecte (all agree, in other say) i continue work on patch - particulary, i want support second strategy (tuple store instead of hash-table) for save order of source (more cheap solution in case with grouping sets + order by), investigate and brainstorm another optimisation, wr

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-11 Thread Pavel Stehule
2009/5/12 Hitoshi Harada : > 2009/5/11 Pavel Stehule : >> I am thinking so Grouping Sets based on CTE should be more commitable >> code. It doesn't mean so your ideas are wrong, but these >> optimalization should to work on CTE too. >> >> select * from table group by rollup(a,b,c) >> >> have to hav

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-11 Thread Hitoshi Harada
2009/5/11 Pavel Stehule : > I am thinking so Grouping Sets based on CTE should be more commitable > code. It doesn't mean so your ideas are wrong, but these > optimalization should to work on CTE too. > > select * from table group by rollup(a,b,c) > > have to have generate same plan as > > with q a

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-10 Thread Олег Царев
I will write separated mail about rollup. Can you send me some links or information about "CTE"? What is it? Also, I need some deep knownledge about postgresql aggregation calculation (executor part) - can you help me (links, books, name of source files, etc)?. After get additional information i wi

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-10 Thread Pavel Stehule
2009/5/10 Олег Царев : > Hello, Pavel. > > I read you letter, and found some points: > > a) Simple transformation any GROUPING SETS in GROUP BY + UNION ALL > require clone source for every group. > It's so slow. > My point: we can make for start simple implementation. > b) Your sollution it's so go

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-10 Thread Олег Царев
Hello, Pavel. I read you letter, and found some points: a) Simple transformation any GROUPING SETS in GROUP BY + UNION ALL require clone source for every group. It's so slow. My point: we can make for start simple implementation. b) Your sollution it's so good, IMHO WITH q AS (SELECT * FROM some)

Re: [HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-10 Thread Pavel Stehule
Hello some other info is on http://wiki.postgresql.org/wiki/Grouping_Sets Maybe in e few weak I'll have some a prototype based on CTE. My older technique based on hashed tables should be well, but it carry lot of unshared code. Using CTE means so we can optimize CTE and GROUPING SETS together. I

[HACKERS] Implementation of GROUPING SETS (T431: Extended grouping capabilities)

2009-05-10 Thread Олег Царев
Hello all. Please, approve my ideas for implementation. Standart has feature T431: Extended grouping capabilities. This feature i found in TODO-list: http://wiki.postgresql.org/wiki/Todo -> SQL Commands -> TO DO MS SQL 2005 partial support this feature: http://www.kodyaz.com/articles/sql-server-2