Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-19 Thread Peter Geoghegan
On Sun, Oct 19, 2014 at 2:52 PM, Greg Stark wrote: > Well OLD and NEW are also not joins yet we expose them this way. It > always seemed like a hack to me but better one hack than two different > inconsistent hacks, no? In my opinion, no. Those hacks do not appear in the parse analysis of ordinar

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-19 Thread Greg Stark
On Thu, Oct 16, 2014 at 8:00 PM, Peter Geoghegan wrote: > Basically, it's difficult to make this work for technical reasons > precisely because what I have here isn't join-like. Can I easily > disallow "OLD.*" in a RETURNING clause (recall that we only project > inserted tuples, as always)? Even i

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-16 Thread Peter Geoghegan
On Thu, Oct 16, 2014 at 11:39 AM, Robert Haas wrote: > On Thu, Oct 16, 2014 at 2:01 PM, Peter Geoghegan wrote: >> I want to retain CONFLICTING(), although I'm thinking about changing >> the spelling to EXCLUDED(). While CONFLICTING() is more or less a new >> and unprecedented style of expression,

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-16 Thread Robert Haas
On Thu, Oct 16, 2014 at 2:01 PM, Peter Geoghegan wrote: > I want to retain CONFLICTING(), although I'm thinking about changing > the spelling to EXCLUDED(). While CONFLICTING() is more or less a new > and unprecedented style of expression, and in general that's something > to be skeptical of, I th

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-16 Thread Peter Geoghegan
On Thu, Oct 16, 2014 at 11:01 AM, Peter Geoghegan wrote: > It is? In any case, I'm working on a revision with this syntax: By the way, in my next revision, barring any objections, the ON CONFLICT (column/expression) syntax is mandatory in the case of ON CONFLICT UPDATE, and optional in the case o

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-16 Thread Peter Geoghegan
On Thu, Oct 16, 2014 at 6:48 AM, Robert Haas wrote: > If that seems too complicated, leave it out for v1: just insist that > there must be at least one unique non-partial index on the relevant > set of columns. That's what I'll do. > There seems to be some confusion here. This part was about th

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-16 Thread Robert Haas
On Mon, Oct 13, 2014 at 2:02 PM, Peter Geoghegan wrote: >> If the user issues INSERT .. ON DUPLICATE (a) UPDATE, we'll fire >> before-insert triggers and then inspect the tuple to be inserted. If >> b is neither 1 nor 2, then we'll fail with an error saying that we >> can't support ON DUPLICATE w

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-13 Thread Peter Geoghegan
On Mon, Oct 13, 2014 at 7:46 AM, Robert Haas wrote: > Come on. You've had the syntax that way for a while, multiple people > have objected to it, and it didn't get changed. When people renewed > their objections, you said that they were not having a "realistic > conversation". I'm tired of gett

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-13 Thread Robert Haas
On Fri, Oct 10, 2014 at 4:41 PM, Peter Geoghegan wrote: > On Fri, Oct 10, 2014 at 12:09 PM, Robert Haas wrote: >> I think what's realistic here is that the patch isn't going to get >> committed the way you have it today, because nobody else likes that >> design. That may be harsh, but I think it

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-12 Thread Matthew Woodcraft
On 2014-10-12 13:40, Marko Tiikkaja wrote: > On 10/12/14, 2:36 PM, Matthew Woodcraft wrote: >> On 2014-10-10 19:44, Kevin Grittner wrote: >>> To restate: to do so is conflating the logical definition of the >>> database with a particular implementation detail. As just one >>> reason that is a bad

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-12 Thread Marko Tiikkaja
On 10/12/14, 2:36 PM, Matthew Woodcraft wrote: On 2014-10-10 19:44, Kevin Grittner wrote: To restate: to do so is conflating the logical definition of the database with a particular implementation detail. As just one reason that is a bad idea: we can look up unique indexes on the specified colu

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-12 Thread Matthew Woodcraft
On 2014-10-10 19:44, Kevin Grittner wrote: > Peter Geoghegan wrote: >> People keep remarking that they don't like that you can (optionally) >> name a unique index explicitly, [...] > To restate: to do so is conflating the logical definition of the > database with a particular implementation de

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 2:16 PM, Kevin Grittner wrote: >> Would you be okay with this never working with partial unique >> indexes? That gives me something to work with. > > That seems like the only sensible course, to me. Okay, great. If that's the consensus, then that's all I need to know. >>

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 2:17 PM, Jim Nasby wrote: > That said, since the use case for UPSERT is different than both INSERT and > UPDATE maybe it would be a good idea to have a separate trigger for them > anyway. -1. I think that having a separate UPSERT trigger is a bad idea. I'll need to change

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Kevin Grittner
Peter Geoghegan wrote: > On Fri, Oct 10, 2014 at 12:09 PM, Robert Haas wrote: >> I think what's realistic here is that the patch isn't going to get >> committed the way you have it today, because nobody else likes that >> design. That may be harsh, but I think it's accurate. > > I do think that'

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Jim Nasby
On 10/9/14, 6:59 PM, Gavin Flower wrote: On 10/10/14 12:38, Jim Nasby wrote: On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >Although the last go-around does suggest that there is at least one >point of difference on the semantics. You seem t

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 12:09 PM, Robert Haas wrote: > I think what's realistic here is that the patch isn't going to get > committed the way you have it today, because nobody else likes that > design. That may be harsh, but I think it's accurate. I do think that's harsh, because it's unnecessar

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 12:04 PM, Kevin Grittner wrote: > Peter Geoghegan wrote: > >> There is no danger of UPDATE before row-level triggers firing >> without then updating (unless the xact aborts, but you know what >> I mean). > > Well, let's make sure I do know what you mean. If a BEFORE UPDAT

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 11:55 AM, Kevin Grittner wrote: > I haven't read the patch, but the discussion has made that clear, > yes. Try to take agreement on a point gracefully, will ya? ;-) Heh, sorry. I did literally mean what I said - it wasn't 100% clear to me that you got that. It's safest t

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Kevin Grittner
Peter Geoghegan wrote: > On Fri, Oct 10, 2014 at 11:44 AM, Kevin Grittner wrote: >> [discussion on committers rejecting the notion of a syntax >> involving specification of an index name] > as I've said many times, there are problems with naming > columns directly that need to be addressed. T

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Robert Haas
On Fri, Oct 10, 2014 at 2:29 PM, Peter Geoghegan wrote: > People keep remarking that they don't like that you can (optionally) > name a unique index explicitly, and I keep telling them why I've done > it that way [1]. There is a trade-off here. I am willing to go another > way in that trade-off, b

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Kevin Grittner
Peter Geoghegan wrote: > There is no danger of UPDATE before row-level triggers firing > without then updating (unless the xact aborts, but you know what > I mean). Well, let's make sure I do know what you mean. If a BEFORE UPDATE ... FOR EACH ROW trigger returns NULL, the UPDATE will be skippe

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Kevin Grittner
Peter Geoghegan wrote: > On Fri, Oct 10, 2014 at 11:30 AM, Kevin Grittner wrote: >> That seems a lot cleaner than the proposal on the Wiki page. If we >> go that route, it makes sense to fire the BEFORE INSERT triggers >> before attempting the insert and then fire BEFORE UPDATE triggers >> befo

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 11:44 AM, Kevin Grittner wrote: > We've all read that, and your repeated arguments for that point of > view. We disagree and have said why. What in that is not a > realistic conversation? Because, as I've said many times, there are problems with naming columns directly t

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 11:38 AM, Peter Geoghegan wrote: >> That seems a lot cleaner than the proposal on the Wiki page. If we >> go that route, it makes sense to fire the BEFORE INSERT triggers >> before attempting the insert and then fire BEFORE UPDATE triggers >> before attempting the UPDATE.

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Kevin Grittner
Peter Geoghegan wrote: > On Fri, Oct 10, 2014 at 11:05 AM, Robert Haas wrote: >> I actually like this syntax reasonably well in some ways, but I don't >> like that we're mentioning the index name, and the CONFLICTING() >> notation is decidedly odd. > > People keep remarking that they don't like

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 11:30 AM, Kevin Grittner wrote: > That seems a lot cleaner than the proposal on the Wiki page. If we > go that route, it makes sense to fire the BEFORE INSERT triggers > before attempting the insert and then fire BEFORE UPDATE triggers > before attempting the UPDATE. If e

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Kevin Grittner
Robert Haas wrote: > Anything we do about triggers will by definition be novel. Right > now, we have INSERT, UPDATE, and DELETE. If we add a new > operation, whether it's called UPSERT or MERGE or FROB, [ ... ] If we call it MERGE, then we had better follow the rules the standard lays out for

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Peter Geoghegan
On Fri, Oct 10, 2014 at 11:05 AM, Robert Haas wrote: > Anything we do about triggers will by definition be novel. Right now, > we have INSERT, UPDATE, and DELETE. If we add a new operation, > whether it's called UPSERT or MERGE or FROB, or if we add a flag to > insert that makes it possibly do s

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-10 Thread Robert Haas
On Wed, Oct 8, 2014 at 5:01 PM, Kevin Grittner wrote: > Peter Geoghegan wrote: >> On Mon, Oct 6, 2014 at 8:35 AM, Robert Haas wrote: >>> I think the problem is that it's not possible to respect the "usual >>> guarantees" even at READ COMMITTED level when performing an INSERT OR >>> UPDATE operat

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-09 Thread Gavin Flower
On 10/10/14 12:38, Jim Nasby wrote: On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >Although the last go-around does suggest that there is at least one >point of difference on the semantics. You seem to want to fire the >BEFORE INSERT trigger

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-09 Thread Jim Nasby
On 10/8/14, 5:51 PM, Peter Geoghegan wrote: On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >Although the last go-around does suggest that there is at least one >point of difference on the semantics. You seem to want to fire the >BEFORE INSERT triggers before determining whether this wil

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 7:04 PM, Marti Raudsepp wrote: > Oops, I missed "for each row" here. Note that I have an open item for what to do about statement level triggers here: https://wiki.postgresql.org/wiki/UPSERT I'm not satisfied with the current behavior. It needs more thought. But I thin

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 4:56 AM, Marti Raudsepp wrote: > create trigger ev1 before insert on evt_type execute procedure ins(); > create trigger ev2 before update on evt_type execute procedure upd(); > create trigger ev3 after insert on evt_type execute procedure ins(); > create trigger ev4 after up

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 4:25 AM, Peter Geoghegan wrote: > On Wed, Oct 8, 2014 at 6:12 PM, Marti Raudsepp wrote: >> Skipping >> BEFORE UPDATE entirely seems to violate POLA. > Good thing that the patch doesn't do that, then. I clearly documented > this in a few places, including: > http://postgres

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 6:12 PM, Marti Raudsepp wrote: > Oh, one more consideration: I believe you will run into the same issue > if you want to implement BEFORE UPDATE triggers in any form. Skipping > BEFORE UPDATE entirely seems to violate POLA. Good thing that the patch doesn't do that, then. I

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 3:47 AM, Peter Geoghegan wrote: > On Wed, Oct 8, 2014 at 5:37 PM, Marti Raudsepp wrote: >> Only in case the trigger changes *key* columns necessary for atomicity >> (i.e. from the WITHIN index). Other columns are fair game. The >> restriction seems justifiable to me: it's u

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 5:37 PM, Marti Raudsepp wrote: > Only in case the trigger changes *key* columns necessary for atomicity > (i.e. from the WITHIN index). Other columns are fair game. The > restriction seems justifiable to me: it's unreasonable to be atomic > with respect to values that change

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 2:46 AM, Peter Geoghegan wrote: >> Indeed, the current behavior breaks even the canonical "keep track of >> how many posts are in a thread" trigger example > use an AFTER trigger for this kind of thing, and all of these > issues go away. In the latest patches from CommitFes

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 4:30 PM, Marti Raudsepp wrote: > On Thu, Oct 9, 2014 at 1:51 AM, Peter Geoghegan wrote: >> On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >>> Although the last go-around does suggest that there is at least one >>> point of difference on the semantics. You seem to w

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Marti Raudsepp
On Thu, Oct 9, 2014 at 1:51 AM, Peter Geoghegan wrote: > On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: >> Although the last go-around does suggest that there is at least one >> point of difference on the semantics. You seem to want to fire the >> BEFORE INSERT triggers before determining

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Peter Geoghegan
On Wed, Oct 8, 2014 at 2:01 PM, Kevin Grittner wrote: > Although the last go-around does suggest that there is at least one > point of difference on the semantics. You seem to want to fire the > BEFORE INSERT triggers before determining whether this will be an > INSERT or an UPDATE. That seems l

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Kevin Grittner
Peter Geoghegan wrote: > On Mon, Oct 6, 2014 at 8:35 AM, Robert Haas wrote: >> I think the problem is that it's not possible to respect the "usual >> guarantees" even at READ COMMITTED level when performing an INSERT OR >> UPDATE operation (however spelled). You may find that there's a tuple >>

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-08 Thread Peter Geoghegan
On Mon, Oct 6, 2014 at 8:35 AM, Robert Haas wrote: > I think the problem is that it's not possible to respect the "usual > guarantees" even at READ COMMITTED level when performing an INSERT OR > UPDATE operation (however spelled). You may find that there's a tuple > with the same PK which is comm

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-06 Thread Peter Geoghegan
On Mon, Oct 6, 2014 at 8:35 AM, Robert Haas wrote: > I think the problem is that it's not possible to respect the "usual > guarantees" even at READ COMMITTED level when performing an INSERT OR > UPDATE operation (however spelled). That's definitely the main problem. Also, there could be garden va

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-06 Thread Robert Haas
On Fri, Oct 3, 2014 at 4:16 PM, Kevin Grittner wrote: >> ... the SQL standard does not require that MERGE be atomic in the >> sense of atomically providing either an INSERT or UPDATE, ... > > My understanding is that the standard logically requires (without > concern for implementation details) th

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 3:42 PM, Peter Geoghegan wrote: >> We routinely pick very >> different plans based on the presence or absence of an index, and >> we use special snapshots in the course of executing many DML >> statements (if FK triggers are fired) Apart from FK snapshots, we also use dirty

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 3:42 PM, Peter Geoghegan wrote: > Yes, it is. I am opposed to using the MERGE syntax for this *because* > MERGE is useful (independently useful, when done properly), not > because it is not useful. As I've mentioned, there is also the practical argument: No one else does t

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 2:44 PM, Kevin Grittner wrote: > I've never seen "atomic" used to mean "you never get an error" > before. > When you are saying "atomic" you mean something quite different. Perhaps I should have been more careful on that point. Just to be crystal clear: I don't intend that

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Kevin Grittner
Peter Geoghegan wrote: > On Fri, Oct 3, 2014 at 1:16 PM, Kevin Grittner wrote: >> I'm having trouble parsing what this statement means: >> >>> ... the SQL standard does not require that MERGE be atomic in >>> the sense of atomically providing either an INSERT or UPDATE, ... > ... always getting

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 1:16 PM, Kevin Grittner wrote: > I have two questions I hope you can clarify. I'm having trouble > parsing what this statement means: > >> ... the SQL standard does not require that MERGE be atomic in the >> sense of atomically providing either an INSERT or UPDATE, ... > >

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Kevin Grittner
Peter Geoghegan wrote: > The page is: https://wiki.postgresql.org/wiki/UPSERT Thanks! I have two questions I hope you can clarify. I'm having trouble parsing what this statement means: > ... the SQL standard does not require that MERGE be atomic in the > sense of atomically providing eithe

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:54 AM, Peter Geoghegan wrote: > I've started off by adding varied examples of the use of the existing > proposed syntax. I'll expand on this soon. I spent some time today expanding on the details, and commenting on the issues around the custom syntax (exactly what it doe

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:07 AM, Heikki Linnakangas wrote: > Could you write down all of the discussed syntaxes, using a similar notation > we use in the manual, with examples on how to use them? And some examples on > what is possible with some syntaxes, and not with others? That would make it >

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 02:52 AM, Peter Geoghegan wrote: Having been surprisingly successful at advancing our understanding of arguments for and against various approaches to "value locking", I decided to try the same thing out elsewhere. I have created a general-purpose UPSERT wiki page. The page is: htt

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-01 Thread Craig Ringer
On 10/02/2014 07:52 AM, Peter Geoghegan wrote: > Having been surprisingly successful at advancing our understanding of > arguments for and against various approaches to "value locking", I > decided to try the same thing out elsewhere. I have created a > general-purpose UPSERT wiki page. > > The pa

[HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-01 Thread Peter Geoghegan
Having been surprisingly successful at advancing our understanding of arguments for and against various approaches to "value locking", I decided to try the same thing out elsewhere. I have created a general-purpose UPSERT wiki page. The page is: https://wiki.postgresql.org/wiki/UPSERT Right now,