Re: [HACKERS] ask for review of MERGE

2010-11-12 Thread Bruce Momjian
Kevin Grittner wrote: > Robert Haas wrote: > > > rhaas=# create table concurrent (x integer primary key); > > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > > "concurrent_pkey" for table "concurrent" > > CREATE TABLE > > rhaas=# insert into x values (1); > > rhaas=# begin; > >

Re: [HACKERS] ask for review of MERGE

2010-10-26 Thread Simon Riggs
On Tue, 2010-10-26 at 16:08 -0400, Robert Haas wrote: > On Tue, Oct 26, 2010 at 8:10 AM, Simon Riggs wrote: > > I agree with your analysis. Let me review... > > [review] > > Sounds like we're on the same page. > > > Two options for resolution are > > > > 1) Throw SERIALIZABLE error > > > > 2) Im

Re: [HACKERS] ask for review of MERGE

2010-10-26 Thread Robert Haas
On Tue, Oct 26, 2010 at 8:10 AM, Simon Riggs wrote: > I agree with your analysis. Let me review... > [review] Sounds like we're on the same page. > Two options for resolution are > > 1) Throw SERIALIZABLE error > > 2) Implement something similar to EvalPlanQual > As you say, we already resolve t

Re: [HACKERS] ask for review of MERGE

2010-10-26 Thread Simon Riggs
On Mon, 2010-10-25 at 16:58 -0400, Robert Haas wrote: > On Mon, Oct 25, 2010 at 4:10 PM, Greg Stark wrote: > > On Mon, Oct 25, 2010 at 12:40 PM, Robert Haas wrote: > >> Now, as Greg says, that might be what some people want, but it's > >> certainly monumentally unserializable. > > > > To be clear

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Robert Haas
On Mon, Oct 25, 2010 at 4:10 PM, Greg Stark wrote: > On Mon, Oct 25, 2010 at 12:40 PM, Robert Haas wrote: >> Now, as Greg says, that might be what some people want, but it's >> certainly monumentally unserializable. > > To be clear when I said it's what people want what I meant was that in > the

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> I would have thought that the INSERT would have >> created an "in doubt" tuple which would block the UPDATE. > This is just standard MVCC - readers don't block writers, nor > writers readers. Sure, but I tend to think of both INSERT and UPDATE a

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Greg Stark
On Mon, Oct 25, 2010 at 12:40 PM, Robert Haas wrote: > Now, as Greg says, that might be what some people want, but it's > certainly monumentally unserializable. To be clear when I said it's what people want what I meant was that in the common cases it's doing exactly what people want. As opposed

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Robert Haas
On Mon, Oct 25, 2010 at 3:15 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> rhaas=# create table concurrent (x integer primary key); >> NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index >> "concurrent_pkey" for table "concurrent" >> CREATE TABLE >> rhaas=# insert into x values (

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Robert Haas wrote: > rhaas=# create table concurrent (x integer primary key); > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "concurrent_pkey" for table "concurrent" > CREATE TABLE > rhaas=# insert into x values (1); > rhaas=# begin; > BEGIN > rhaas=# insert into concurrent v

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Robert Haas
On Mon, Oct 25, 2010 at 1:42 PM, Greg Stark wrote: > On Sun, Oct 24, 2010 at 10:43 AM, Robert Haas wrote: >> But let's back up and talk about MVCC for a minute.  Suppose we have >> three source tuples, (1), (2), and (3); and the target table contains >> tuples (1) and (2), of which only (1) is vi

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Greg Stark wrote: > Robert Haas wrote: >> But let's back up and talk about MVCC for a minute. Suppose we >> have three source tuples, (1), (2), and (3); and the target table >> contains tuples (1) and (2), of which only (1) is visible to our >> MVCC snapshot; suppose also an equijoin. Clearly,

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Greg Stark
On Sun, Oct 24, 2010 at 10:43 AM, Robert Haas wrote: > But let's back up and talk about MVCC for a minute.  Suppose we have > three source tuples, (1), (2), and (3); and the target table contains > tuples (1) and (2), of which only (1) is visible to our MVCC snapshot; > suppose also an equijoin.  

Re: [HACKERS] ask for review of MERGE

2010-10-25 Thread Kevin Grittner
Robert Haas wrote: > What we're talking about is what happens when there are concurrent > table modifications in progress; and the answer is that you might > get serialization anomalies. But we have serialization anomalies > without MERGE, too - see the discussions around Kevin Grittner's > SSI

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 7:11 PM, Greg Stark wrote: > On Sun, Oct 24, 2010 at 2:50 AM, Martijn van Oosterhout > wrote: >> Can we please not get MERGE hung up on trying to make it atomic. The >> standard requires no such thing and there are plenty of uses of MERGE >> that don't involve high concurr

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sun, Oct 24, 2010 at 2:39 PM, Greg Smith wrote: > Sure, but in the most common use case I think we're targeting at first, no > indexes means there's also no unique constraints either.  I don't think > people can reasonable expect to MERGE or anything else to guarantee they > won't accidentally

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sun, Oct 24, 2010 at 2:50 AM, Martijn van Oosterhout wrote: > Can we please not get MERGE hung up on trying to make it atomic. The > standard requires no such thing and there are plenty of uses of MERGE > that don't involve high concurrency updates of the same row by > different processes. If w

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Smith
Robert Haas wrote: Well, there's no guarantee that any index at all exists on the target table, so any solution based on index locks can't be fully general. Sure, but in the most common use case I think we're targeting at first, no indexes means there's also no unique constraints either. I

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 12:15 PM, Greg Smith wrote: > Robert Haas wrote: >> I am also wondering exactly what the semantics are supposed to be >> under concurrency.  We can't assume that it's OK to treat WHEN NOT >> MATCHED as WHEN MATCHED if a unique-key violation is encountered.  The >> join cond

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Smith
Robert Haas wrote: I am also wondering exactly what the semantics are supposed to be under concurrency. We can't assume that it's OK to treat WHEN NOT MATCHED as WHEN MATCHED if a unique-key violation is encountered. The join condition could be something else entirely. I guess we could scan th

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Robert Haas
On Sun, Oct 24, 2010 at 5:50 AM, Martijn van Oosterhout wrote: > On Sat, Oct 23, 2010 at 03:59:13PM -0700, Greg Stark wrote: >> I think the blocker with MERGE has always been that the standard is >> *so* general that it could apply to conditions that *aren't* covered >> by a unique constraint or e

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Martijn van Oosterhout
On Sat, Oct 23, 2010 at 03:59:13PM -0700, Greg Stark wrote: > I think the blocker with MERGE has always been that the standard is > *so* general that it could apply to conditions that *aren't* covered > by a unique constraint or exclusion constriant. Personally, I'm fine > saying that those cases w

Re: [HACKERS] ask for review of MERGE

2010-10-24 Thread Greg Stark
On Sat, Oct 23, 2010 at 4:03 PM, Josh Berkus wrote: > I think that such a lock would also be useful for improving the FK deadlock > issues we have. I don't see how. I think the problem you're referring to occurs when different plans update rows in different orders and the resulting locks on forei

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Josh Berkus
So the trick for MERGE on equality would be to refactor the btree api so that you could find the matching leaf page and *keep* that page locked. Then do an update against the conflicting row found there if any without ever releasing that page lock. Someone else can come along and delete the row

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Greg Stark
On Sat, Oct 23, 2010 at 9:12 AM, Tom Lane wrote: >> "PostgreSQL doesn't have a good way to lock access to a key value that >> doesn't exist yet--what other databases call key range >> locking...Improvements to the index implementation are needed to allow >> this feature." > > This seems to be pres

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Boxuan Zhai
> > > This did seem to find a bug in the implementation after running for a > while: > > TRAP: FailedAssertion("!(epqstate->origslot != ((void *)0))", File: > "execMain.c", Line: 1732) > > Line number there is relative to what you can see at > http://github.com/greg2ndQuadrant/postgres/blob/merge/s

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Tom Lane
Greg Smith writes: > Marko Tiikkaja wrote: >> What's been bothering me is that so far there has not been an >> agreement on whether we need to protect against concurrency issues or >> not. In fact, there has been almost no discussion about the >> concurrency issues which AIUI have been the big

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Greg Smith
Marko Tiikkaja wrote: What's been bothering me is that so far there has not been an agreement on whether we need to protect against concurrency issues or not. In fact, there has been almost no discussion about the concurrency issues which AIUI have been the biggest single reason we don't have

Re: [HACKERS] ask for review of MERGE

2010-10-23 Thread Marko Tiikkaja
On 2010-10-23 8:34 AM +0300, Greg Smith wrote: While the performance doesn't need to be great in V1, there needs to be at least some minimal protection against concurrency issues before this is commit quality. What's been bothering me is that so far there has not been an agreement on whether w

Re: [HACKERS] ask for review of MERGE

2010-10-22 Thread Greg Smith
There are now two branches of MERGE code in review progress available. http://github.com/greg2ndQuadrant/postgres/tree/merge-unstable has the bit-rotted version that doesn't quite work against HEAD yet, while http://github.com/greg2ndQuadrant/postgres/tree/merge is what I'm still testing again

Re: [HACKERS] ask for review of MERGE

2010-10-22 Thread Stefan Kaltenbrunner
On 10/21/2010 08:36 PM, Greg Smith wrote: Robert Haas wrote: I think the right way to write UPSERT is something along the lines of: MERGE INTO Stock t USING (VALUES (10, 1)) s(item_id, balance) ON s.item_id = t.item_id ... [...] Here's what the query plan looks like on a MATCH: Merge (cost

Re: [HACKERS] ask for review of MERGE

2010-10-21 Thread Greg Smith
Robert Haas wrote: I think the right way to write UPSERT is something along the lines of: MERGE INTO Stock t USING (VALUES (10, 1)) s(item_id, balance) ON s.item_id = t.item_id ... That led in the right direction, after a bit more fiddling I was finally able to get something that does what

Re: [HACKERS] ask for review of MERGE

2010-10-19 Thread Boxuan Zhai
Hi, I considered the empty source table situation again. I think it is correct to upsert nothing in this case. Back to the original logic of MERGE command, it is main purpose is to add the supplementary data from the source table into the target table. Thus, an empty source table means no input d

Re: [HACKERS] ask for review of MERGE

2010-10-18 Thread Robert Haas
On Mon, Oct 18, 2010 at 10:09 AM, Boxuan Zhai wrote: > > > On Mon, Oct 18, 2010 at 9:54 PM, Robert Haas wrote: >> >> I think that MERGE is supposed to trigger one rule for each row in the >> source data.  So: >> >> On Sun, Oct 17, 2010 at 8:20 PM, Greg Smith wrote: >> > MERGE INTO Stock t >> >  

Re: [HACKERS] ask for review of MERGE

2010-10-18 Thread Boxuan Zhai
On Mon, Oct 18, 2010 at 9:54 PM, Robert Haas wrote: > I think that MERGE is supposed to trigger one rule for each row in the > source data. So: > > On Sun, Oct 17, 2010 at 8:20 PM, Greg Smith wrote: > > MERGE INTO Stock t > > USING (SELECT * FROM Stock WHERE item_id=10) AS s > > ON s.item_id=

Re: [HACKERS] ask for review of MERGE

2010-10-18 Thread Robert Haas
I think that MERGE is supposed to trigger one rule for each row in the source data. So: On Sun, Oct 17, 2010 at 8:20 PM, Greg Smith wrote: > MERGE INTO Stock t >  USING (SELECT * FROM Stock WHERE item_id=10) AS s >  ON s.item_id=t.item_id >  WHEN MATCHED THEN UPDATE SET balance=s.balance + 1 >  

Re: [HACKERS] ask for review of MERGE

2010-10-14 Thread Greg Smith
Robert Haas wrote: Greg, are you still working on a review of this patch? Yes, just had more distractions while coming to speed up on this area than I'd hoped. I'll get a second round of looking at this done by the weekend. -- Greg Smith, 2ndQuadrant US g...@2ndquadrant.com Baltimore, M

Re: [HACKERS] ask for review of MERGE

2010-10-14 Thread Robert Haas
On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith wrote: > Starting looking at the latest MERGE patch from Boxuan here tonight. The > regression tests pass for me here, good starting sign. I expect to move onto > trying to break it more creatively next, then onto performance tests. > Nothing much more e

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Boxuan Zhai
On Wed, Sep 29, 2010 at 9:16 PM, Robert Haas wrote: > On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith wrote: > > One compiler warning I noticed that needs to get resolved: > > > > src/backend/commands/explain.c: > > > > explain.c: In function ‘ExplainMergeActions’: > > explain.c:1528: warning: compa

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Robert Haas
On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith wrote: > One compiler warning I noticed that needs to get resolved: > > src/backend/commands/explain.c: > > explain.c: In function ‘ExplainMergeActions’: > explain.c:1528: warning: comparison of distinct pointer types lacks a cast > > That is complaining

Re: [HACKERS] ask for review of MERGE

2010-09-29 Thread Josh Kupershmidt
On Wed, Sep 29, 2010 at 2:44 AM, Greg Smith wrote: > The rest of the compiler warnings I saw didn't look related to his code, > maybe stuff my picky Ubuntu compiler is noticing that was done recently to > HEAD. I haven't checked HEAD without this patch yet to confirm, and am done > for the night

Re: [HACKERS] ask for review of MERGE

2010-09-28 Thread Greg Smith
Starting looking at the latest MERGE patch from Boxuan here tonight. The regression tests pass for me here, good starting sign. I expect to move onto trying to break it more creatively next, then onto performance tests. Nothing much more exciting than that to report so far. It had suffered som

Re: [HACKERS] ask for review of MERGE

2010-09-24 Thread Greg Smith
Finding time for a review as large as this one is a bit tough, but I've managed to set aside a couple of days for it over the next week. I'm delivering a large project tonight and intend to start in on the review work tomorrow onced that's cleared up. If you're ever not sure who is working on

Re: [HACKERS] ask for review of MERGE

2010-09-23 Thread Marko Tiikkaja
On 2010-09-23 1:31 PM +0300, Boxuan Zhai wrote: I have just generate a new patch of MERGE command. I haven't followed the discussion very closely, but this part in the regression tests caught my attention: +-- we now have a duplicate key in Buy, so when we join to +-- Stock we will generate

Re: [HACKERS] ask for review of MERGE

2010-09-23 Thread Thom Brown
On 23 September 2010 11:31, Boxuan Zhai wrote: > Dear All, > I have just generate a new patch of MERGE command. > One main change in this edition is the removal of RASIE ERROR action from > MEREG, because its semantics is not well defined yet. > I also rewrote the regress test file merge.sql, to m