Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-02-04 Thread Robert Haas
On Sat, Feb 2, 2013 at 4:38 PM, Andres Freund wrote: > On 2013-01-28 16:55:52 -0500, Steve Singer wrote: >> If your using non-surragate /natural primary keys this tends to come up >> occasionally due to data-entry errors or renames. I'm looking at this from >> the point of view of what do I need

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-02-02 Thread Andres Freund
On 2013-01-28 16:55:52 -0500, Steve Singer wrote: > If your using non-surragate /natural primary keys this tends to come up > occasionally due to data-entry errors or renames. I'm looking at this from > the point of view of what do I need to use this as a source for a production > replication sys

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-28 Thread Steve Singer
On 13-01-28 06:17 AM, Andres Freund wrote: Hi, 3. Pass the delete (with no key values) onto the replication client and let it deal with it (see 1 and 2) Hm. While I agree that nicer behaviour would be good I think the real enforcement should happen on a higher level, e.g. with event triggers

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-28 Thread Andres Freund
Hi, On 2013-01-27 23:07:51 -0500, Steve Singer wrote: > A few more comments; > > In decode.c DecodeDelete > > + if (r->xl_len <= (SizeOfHeapDelete + SizeOfHeapHeader)) > + { > + elog(DEBUG2, "huh, no primary key for a delete on wal_level = > logical?"); > + return; > + } > +

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-28 Thread Andres Freund
On 2013-01-26 16:20:33 -0500, Steve Singer wrote: > On 13-01-24 11:15 AM, Steve Singer wrote: > >On 13-01-24 06:40 AM, Andres Freund wrote: > >> > >>Fair enough. I am also working on a user of this infrastructure but that > >>doesn't help you very much. Steve Singer seemed to make some stabs at > >

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-27 Thread Steve Singer
On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get there?

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-26 Thread Steve Singer
On 13-01-24 11:15 AM, Steve Singer wrote: On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get there?

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Bruce Momjian
On Fri, Jan 25, 2013 at 02:16:09AM +0100, Andres Freund wrote: > What I am afraid though is that it basically goes on like this in the > next commitfests: > * 9.4-CF1: no "serious" reviewer comments because they are busy doing release > work > * 9.4-CF2: all are relieved that the release is over a

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Andres Freund
Hi! On 2013-01-24 13:27:00 -0500, Robert Haas wrote: > On Thu, Jan 24, 2013 at 6:14 AM, Andres Freund wrote: > Before getting bogged down in technical commentary, let me say this > very clearly: I am enormously grateful for your work on this project. > Logical replication based on WAL decoding i

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Heikki Linnakangas
On 24.01.2013 20:27, Robert Haas wrote: Before getting bogged down in technical commentary, let me say this very clearly: I am enormously grateful for your work on this project. Logical replication based on WAL decoding is a feature of enormous value that PostgreSQL has needed for a long time, an

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Now, the bad news is, I don't think it's very reasonable to try to > commit this to 9.3. I think it is just too much stuff too late in the > cycle. I've reviewed some of the patches from time to time but there > is a lot more stuff and it's big and c

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Robert Haas
On Thu, Jan 24, 2013 at 6:14 AM, Andres Freund wrote: > Thats way much more along the lines of what I am afraid of than the > performance stuff - but Heikki cited those, so I replied to that. > > Note that I didn't say this must, must go in - I just don't think > Heikki's reasoning about why not h

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Steve Singer
On 13-01-24 06:40 AM, Andres Freund wrote: Fair enough. I am also working on a user of this infrastructure but that doesn't help you very much. Steve Singer seemed to make some stabs at writing an output plugin as well. Steve, how far did you get there? I was able to get something that generat

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Andres Freund
On 2013-01-24 12:38:25 +0200, Heikki Linnakangas wrote: > On 24.01.2013 00:30, Andres Freund wrote: > >Hi, > > > >I decided to reply on the patches thread to be able to find this later. > > > >On 2013-01-23 22:48:50 +0200, Heikki Linnakangas wrote: > >>"logical changeset generation v4" > >>This is

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Heikki Linnakangas
One random thing that caught my eye in the patch, I though I'd mention it while I still remember: In heap_delete, you call heap_form_tuple() in a critical section. That's a bad idea, because if it runs out of memory -> PANIC. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Andres Freund
Hi Robert, Hi all, On 2013-01-23 20:17:04 -0500, Robert Haas wrote: > On Wed, Jan 23, 2013 at 5:30 PM, Andres Freund wrote: > > The only reason the submitted version of logical decoding is > > comparatively slow is that its xmin update policy is braindamaged, > > working on that right now. > > I

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Heikki Linnakangas
On 24.01.2013 00:30, Andres Freund wrote: Hi, I decided to reply on the patches thread to be able to find this later. On 2013-01-23 22:48:50 +0200, Heikki Linnakangas wrote: "logical changeset generation v4" This is a boatload of infrastructure for supporting logical replication, yet we have n

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-24 Thread Simon Riggs
On 24 January 2013 01:17, Robert Haas wrote: > I agree. The thing that scares me about the logical replication stuff > is not that it might be slow (and if your numbers are to be believed, > it isn't), but that I suspect it's riddled with bugs and possibly some > questionable design decisions.

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-23 Thread Joshua D. Drake
On 01/23/2013 05:17 PM, Robert Haas wrote: Of course, I have no evidence that that will happen. But it is a really big piece of code, and therefore unless you are superman, it's probably got a really large number of bugs. The scary thing is that it is not as if we can say, well, this is a big

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-23 Thread Robert Haas
On Wed, Jan 23, 2013 at 5:30 PM, Andres Freund wrote: > pgbench upstream: > tps: 22275.941409 > space overhead: 0% > pgbench logical-submitted > tps: 16274.603046 > space overhead: 2.1% > pgbench logical-HEAD (will submit updated version tomorrow or so): > tps: 20853.341551 > space overhead: 2.3%

Re: [HACKERS] logical changeset generation v4 - Heikki's thoughts about the patch state

2013-01-23 Thread Andres Freund
Hi, I decided to reply on the patches thread to be able to find this later. On 2013-01-23 22:48:50 +0200, Heikki Linnakangas wrote: > "logical changeset generation v4" > This is a boatload of infrastructure for supporting logical replication, yet > we have no code actually implementing logical re