Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 20:51 -0500, Tom Lane wrote: > No. The current code involves two pallocs per cycle, one inside the > aggregate function to construct its result value, and then one in > datumCopy to copy the result into the proper context. Ah, true -- missed the fact that PG_RETURN_INT64() d

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > - yours would mean that int8inc() and similar aggregates wouldn't ever > need to do palloc(); mine would require a palloc() every k calls to the > transition function. No. The current code involves two pallocs per cycle, one inside the aggregate function

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 19:07 -0500, Tom Lane wrote: > True, but you still have to palloc if it returns the second argument. Is that common? In any case, I don't see how you can _ever_ avoid a palloc if the aggregate returns the second argument. The second argument is in a per-tuple memory context:

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > On Thu, 2004-12-02 at 10:45 -0500, Tom Lane wrote: >> (2) I think you lose much of the performance >> benefit as soon as you have to distinguish cases (b) and (c). > Why wouldn't a simple comparison work? We're passing two arguments into > the aggregate fu

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Neil Conway
On Thu, 2004-12-02 at 10:45 -0500, Tom Lane wrote: > (2) I think you lose much of the performance > benefit as soon as you have to distinguish cases (b) and (c). Ideally > you would use MemoryContextContains for this, but that doesn't work > reliably on pointers that point to fields of a tuple. W

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-02 Thread Alvaro Herrera
On Wed, Dec 01, 2004 at 10:04:43PM -0500, Tom Lane wrote: > "Andrew Dunstan" <[EMAIL PROTECTED]> writes: > > I've raised this before, but would like to suggest again that there might be > > virtue in branching earlier in the dev cycle - maybe around the time of the > > first beta. > > Given the am

Re: Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-02 Thread simon
Neil Conway <[EMAIL PROTECTED]> wrote on 02.12.2004, 05:05:12: > On Wed, 2004-12-01 at 19:34 +0100, [EMAIL PROTECTED] wrote: > > I regard performance testing as an essential part of the > > release process of any performance critical software. Most earlier beta > > releases were fixing functional

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-02 Thread Andrew Dunstan
Tom Lane wrote: "Andrew Dunstan" <[EMAIL PROTECTED]> writes: I've raised this before, but would like to suggest again that there might be virtue in branching earlier in the dev cycle - maybe around the time of the first beta. Given the amount of patching that's gone on, branching 8.1 earli

Re: [HACKERS] nodeAgg perf tweak

2004-12-02 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > ISTM it would be reasonable to mandate that aggregate authors return one > of three things from their state transition functions: > (a) return the previous state value > (b) return the "next data item" value > (c) return some other value; if by a pas

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 15:54 -0500, Tom Lane wrote: > This seems like it might work. Instead of copying the result into the > aggcontext on every cycle, we could copy it only when we intend to reset > the working context. Right. > This is > problematic since the source tuple will go away on the n

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 19:34 +0100, [EMAIL PROTECTED] wrote: > I regard performance testing as an essential part of the > release process of any performance critical software. Most earlier beta > releases were fixing functional problems and now the focus has moved to > performance related issues. I

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Tom Lane
"Andrew Dunstan" <[EMAIL PROTECTED]> writes: > I've raised this before, but would like to suggest again that there might be > virtue in branching earlier in the dev cycle - maybe around the time of the > first beta. Given the amount of patching that's gone on, branching 8.1 earlier would have mean

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > The attached patch invokes the transition function in the current memory > context. I don't think that's right: a memory leak in an aggregate's > transition function would be problematic when we're invoked from a > per-query memory context, as is the case w

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Andrew Dunstan
Neil Conway said: > On Wed, 2004-12-01 at 11:08 -0300, Alvaro Herrera wrote: > >> There's real development waiting only for release to happen ... > > There is real development being done as we speak, that will be landed > to 8.1 when it branches :) > I've raised this before, but would like to sugg

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 11:08 -0300, Alvaro Herrera wrote: > When would the experimentation end, and 8.0 be finally released? As I said, this is work for 8.1; I don't think it ought to affect the release schedule of 8.0 (perhaps in some marginal way because Tom might spend some time discussing issue

Re: Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread simon
Alvaro Herrera <[EMAIL PROTECTED]> wrote on 01.12.2004, 15:08:11: > On Wed, Dec 01, 2004 at 10:03:40AM +, Simon Riggs wrote: > > > Please shave 20% off everybody's aggregation queries, ugly or not. > > You'll see a lot of happy people. > > When would the experimentation end, and 8.0 be final

Please release (was Re: [HACKERS] nodeAgg perf tweak)

2004-12-01 Thread Alvaro Herrera
On Wed, Dec 01, 2004 at 10:03:40AM +, Simon Riggs wrote: > Please shave 20% off everybody's aggregation queries, ugly or not. > You'll see a lot of happy people. When would the experimentation end, and 8.0 be finally released? There's real development waiting only for release to happen ... I

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Simon Riggs
On Wed, 2004-12-01 at 08:37, Neil Conway wrote: > On Wed, 2004-12-01 at 08:25 +, Simon Riggs wrote: > > I'd be a little twitchy about new memory contexts at this stage of beta, > > but if the code is fairly well isolated that could be good. > > This would be for 8.1 anyway. > > > Would it pos

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Neil Conway
On Wed, 2004-12-01 at 08:25 +, Simon Riggs wrote: > I'd be a little twitchy about new memory contexts at this stage of beta, > but if the code is fairly well isolated that could be good. This would be for 8.1 anyway. > Would it possible to differentiate between well-known builtin functions >

Re: [HACKERS] nodeAgg perf tweak

2004-12-01 Thread Simon Riggs
On Wed, 2004-12-01 at 02:48, Neil Conway wrote: > I noticed that we have a bottleneck in aggregate performance in > advance_transition_function(): according to callgrind, doing datumCopy() > and pfree() for every tuple produced by the transition function is > pretty expensive. Some queries bare thi

Re: [HACKERS] nodeAgg perf tweak

2004-11-30 Thread Neil Conway
On Tue, 2004-11-30 at 23:15 -0500, Tom Lane wrote: > And how badly does it leak memory? I do not believe this patch is > tenable. Did you read the rest of my mail? > Something that occurred to me the other morning in the shower is that we > could trivially inline MemoryContextSwitchTo() when usi

Re: [HACKERS] nodeAgg perf tweak

2004-11-30 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > I've attached a quick and dirty hack that avoids the need to palloc() > and pfree() for every tuple produced by the aggregate's transition > function. And how badly does it leak memory? I do not believe this patch is tenable. Something that occurred to m