Re: [HACKERS] COPY and Volatile default expressions

2014-01-15 Thread Simon Riggs
On 16 April 2013 14:37, Simon Riggs wrote: > On 16 April 2013 13:57, Heikki Linnakangas wrote: > >> You still need to check the args, if the function is nextval, otherwise you >> incorrectly perform the optimization for something like >> "nextval(myvolatilefunc())". > > Guess so. At least its an

Re: [HACKERS] COPY and Volatile default expressions

2013-06-28 Thread Simon Riggs
On 24 June 2013 10:21, Kohei KaiGai wrote: > Hi Simon, > > I checked this patch. One thing I could comment on is, do you think it is > a good > idea to have oid of exception function list on > contain_volatile_functions_walker()? > > The walker function is static thus here is no impact for other

Re: [HACKERS] COPY and Volatile default expressions

2013-06-24 Thread Kohei KaiGai
Hi Simon, I checked this patch. One thing I could comment on is, do you think it is a good idea to have oid of exception function list on contain_volatile_functions_walker()? The walker function is static thus here is no impact for other caller, and its "context" argument is unused. My propositio

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Simon Riggs
On 16 April 2013 15:07, Tom Lane wrote: > Bruce Momjian writes: >> I found Simon's nextval()/COPY timings without this patch sobering. I >> assume he can apply this for 9.3, right? I believe it is a fix for a >> new 9.3 feature. > > It is not a "fix", it is not for a 9.3 feature (the multi-inse

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Bruce Momjian
On Tue, Apr 16, 2013 at 10:07:07AM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I found Simon's nextval()/COPY timings without this patch sobering. I > > assume he can apply this for 9.3, right? I believe it is a fix for a > > new 9.3 feature. > > It is not a "fix", it is not for a 9.3 fe

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Tom Lane
Bruce Momjian writes: > I found Simon's nextval()/COPY timings without this patch sobering. I > assume he can apply this for 9.3, right? I believe it is a fix for a > new 9.3 feature. It is not a "fix", it is not for a 9.3 feature (the multi-insert thing went in in 9.2), and personally I'd vote

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Bruce Momjian
On Tue, Apr 16, 2013 at 02:37:33PM +0100, Simon Riggs wrote: > On 16 April 2013 13:57, Heikki Linnakangas wrote: > > > You still need to check the args, if the function is nextval, otherwise you > > incorrectly perform the optimization for something like > > "nextval(myvolatilefunc())". > > Gues

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Simon Riggs
On 16 April 2013 13:57, Heikki Linnakangas wrote: > You still need to check the args, if the function is nextval, otherwise you > incorrectly perform the optimization for something like > "nextval(myvolatilefunc())". Guess so. At least its an easy change. Thanks for checking. -- Simon Riggs

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Heikki Linnakangas
On 16.04.2013 14:38, Simon Riggs wrote: On 15 April 2013 21:53, Simon Riggs wrote: So I'll treat this as two separate cases: * add special case for sequences Patch attached. + if (IsA(node, FuncExpr)) + { + FuncExpr *expr = (FuncExpr *) node; + + /*

Re: [HACKERS] COPY and Volatile default expressions

2013-04-16 Thread Simon Riggs
On 15 April 2013 21:53, Simon Riggs wrote: > So I'll treat this as two separate cases: > * add special case for sequences Patch attached. > * use the NO SQL mechanism, as described, which implies no reads or > writes of database state. We could test that, but its somewhat harder > and we'd need

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 21:32, Jaime Casanova wrote: > On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane wrote: >> >> OTOH, the notion that a UUID generator doesn't touch *any* database >> state seems like it might be worth treating as a general function >> property: it's simple to understand and applies to a l

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 4:21 PM, Tom Lane wrote: > I think plenty of people would be upset if row serial numbers assigned > with nextval() were not assigned in the order of the incoming rows. > The argument that you can get gaps in the sequence in some corner cases > (none of which apply within a

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 20:52, Robert Haas wrote: > On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane wrote: >>> I claim this is a common class, since sequence next_val functions and >>> uuid generators meet that criteria and most common forms of auditing >>> trigger, as well as any other form of data-reformat

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Jaime Casanova
On Mon, Apr 15, 2013 at 3:21 PM, Tom Lane wrote: > > OTOH, the notion that a UUID generator doesn't touch *any* database > state seems like it might be worth treating as a general function > property: it's simple to understand and applies to a lot of other > volatile functions such as random() and

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Robert Haas writes: > On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane wrote: >> I don't believe that it's a good idea to consider nextval() to be >> reorderable, so I'm not convinced by your argument here. > Why not? > I admit that I can't convince myself that it's safe. But I can't > think of a co

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Robert Haas
On Mon, Apr 15, 2013 at 11:49 AM, Tom Lane wrote: >> I claim this is a common class, since sequence next_val functions and >> uuid generators meet that criteria and most common forms of auditing >> trigger, as well as any other form of data-reformatting trigger. > > I don't believe that it's a goo

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 07:04:55PM +0100, Simon Riggs wrote: > On 15 April 2013 18:41, David Fetter wrote: > > > The difference between HEAD and patch in the "COPY, with sequence" > > case is pretty remarkable. What's the patch? > > Attached. Thanks! :) > This is usable only for this test. It

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 18:41, David Fetter wrote: > The difference between HEAD and patch in the "COPY, with sequence" > case is pretty remarkable. What's the patch? Attached. This is usable only for this test. It is not anywhere remotely close to being applied. -- Simon Riggs ht

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 06:30:55PM +0100, Simon Riggs wrote: > On 15 April 2013 17:04, Simon Riggs wrote: > > > I will implement as a kluge, test and report the results. > > Test is COPY 1 million rows on a table with 2 columns, both bigint. > Verified no checkpoints triggered during load. > No

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:04, Simon Riggs wrote: > I will implement as a kluge, test and report the results. Test is COPY 1 million rows on a table with 2 columns, both bigint. Verified no checkpoints triggered during load. No other work active on database, tests condicted on laptop Autovacuum disable

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
David Fetter writes: > On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote: >> Loading data into a table with a SERIAL or UUID column is the main >> use case, so I'll measure that. > The former is common enough a use case to optimize specifically, > should the numbers come out right. Yea

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Hannu Krosing
On 04/15/2013 06:04 PM, Simon Riggs wrote: On 15 April 2013 16:55, Tom Lane wrote: Simon Riggs writes: On 15 April 2013 16:24, David Fetter wrote: Do you have numbers on this, or ways to gather same? In other words, how do we know what resources (time, CPU cycles, disk seeks, etc.) are bei

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 17:08, David Fetter wrote: >> Loading data into a table with a SERIAL or UUID column is the main >> use case, so I'll measure that. > > The former is common enough a use case to optimize specifically, > should the numbers come out right. Do you suppose that an in-core > UUID gen

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 05:04:16PM +0100, Simon Riggs wrote: > On 15 April 2013 16:55, Tom Lane wrote: > > Simon Riggs writes: > >> On 15 April 2013 16:24, David Fetter wrote: > >>> Do you have numbers on this, or ways to gather same? In other > >>> words, how do we know what resources (time, C

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 11:49:42AM -0400, Tom Lane wrote: > Simon Riggs writes: > > COPY cannot be optimised correctly if we have before triggers or > > volatile default expressions. > > > The multi-insert code detects those cases and falls back to the single > > row mechanism in those cases. >

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:55, Tom Lane wrote: > Simon Riggs writes: >> On 15 April 2013 16:24, David Fetter wrote: >>> Do you have numbers on this, or ways to gather same? In other words, >>> how do we know what resources (time, CPU cycles, disk seeks, etc.) are >>> being consumed here? > >> The mult

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs writes: > On 15 April 2013 16:24, David Fetter wrote: >> Do you have numbers on this, or ways to gather same? In other words, >> how do we know what resources (time, CPU cycles, disk seeks, etc.) are >> being consumed here? > The multi-insert optimisation for COPY is already there a

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:41, Heikki Linnakangas wrote: >> What I'd like to do is to invent a new form of labelling that allows >> us to understand that COPY can still be optimised. > > It would be even nicer to detect at runtime, when a default expression or > before trigger tries to access the same t

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Tom Lane
Simon Riggs writes: > COPY cannot be optimised correctly if we have before triggers or > volatile default expressions. > The multi-insert code detects those cases and falls back to the single > row mechanism in those cases. > There a common class of volatile functions that wouldn't cause > probl

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
On 15 April 2013 16:24, David Fetter wrote: >> I claim this is a common class, since sequence next_val functions and >> uuid generators meet that criteria and most common forms of auditing >> trigger, as well as any other form of data-reformatting trigger. Since >> this is a common case, it seems

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Heikki Linnakangas
On 15.04.2013 17:00, Simon Riggs wrote: COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that wouldn't ca

Re: [HACKERS] COPY and Volatile default expressions

2013-04-15 Thread David Fetter
On Mon, Apr 15, 2013 at 03:00:34PM +0100, Simon Riggs wrote: > COPY cannot be optimised correctly if we have before triggers or > volatile default expressions. > > The multi-insert code detects those cases and falls back to the > single row mechanism in those cases. > > There a common class of vo

[HACKERS] COPY and Volatile default expressions

2013-04-15 Thread Simon Riggs
COPY cannot be optimised correctly if we have before triggers or volatile default expressions. The multi-insert code detects those cases and falls back to the single row mechanism in those cases. There a common class of volatile functions that wouldn't cause problems: any volatile function that d