Re: [HACKERS] Changeset Extraction v7.6.1

2014-06-01 Thread Jim Nasby
On 6/1/14, 10:49 AM, Euler Taveira wrote: On 01-06-2014 02:57, Andres Freund wrote: On 2014-06-01 00:50:58 -0500, Jim Nasby wrote: On 5/31/14, 9:11 AM, Andres Freund wrote: On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep t

Re: [HACKERS] Changeset Extraction v7.6.1

2014-06-01 Thread Euler Taveira
On 01-06-2014 02:57, Andres Freund wrote: > On 2014-06-01 00:50:58 -0500, Jim Nasby wrote: >> On 5/31/14, 9:11 AM, Andres Freund wrote: >>> On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: > But do you really want to keep that snapshot around long enou

Re: [HACKERS] Changeset Extraction v7.6.1

2014-05-31 Thread Andres Freund
On 2014-06-01 00:50:58 -0500, Jim Nasby wrote: > On 5/31/14, 9:11 AM, Andres Freund wrote: > >On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: > >>On 2/17/14, 7:31 PM, Robert Haas wrote: > >>>But do you really want to keep that snapshot around long enough to > >>>copy the entire database? I bet you

Re: [HACKERS] Changeset Extraction v7.6.1

2014-05-31 Thread Jim Nasby
On 5/31/14, 9:11 AM, Andres Freund wrote: On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy the entire database? I bet you don't: if the database is big, holding back xmin for long enou

Re: [HACKERS] Changeset Extraction v7.6.1

2014-05-31 Thread Andres Freund
On 2014-02-21 15:14:15 -0600, Jim Nasby wrote: > On 2/17/14, 7:31 PM, Robert Haas wrote: > >But do you really want to keep that snapshot around long enough to > >copy the entire database? I bet you don't: if the database is big, > >holding back xmin for long enough to copy the whole thing isn't li

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-27 Thread Robert Haas
On Thu, Feb 27, 2014 at 11:06 AM, Andres Freund wrote: > On 2014-02-24 12:50:03 -0500, Robert Haas wrote: >> On Mon, Feb 24, 2014 at 9:48 AM, Andres Freund >> wrote: >> > On 2014-02-15 17:29:04 -0500, Robert Haas wrote: >> >> On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund >> >> wrote: >> > >>

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-27 Thread Andres Freund
On 2014-02-24 12:50:03 -0500, Robert Haas wrote: > On Mon, Feb 24, 2014 at 9:48 AM, Andres Freund wrote: > > On 2014-02-15 17:29:04 -0500, Robert Haas wrote: > >> On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund > >> wrote: > > > >> + /* > >> +* XXX: It's impolite to ignore our argum

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-24 Thread Robert Haas
On Mon, Feb 24, 2014 at 9:48 AM, Andres Freund wrote: > On 2014-02-15 17:29:04 -0500, Robert Haas wrote: >> On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund >> wrote: > >> + /* >> +* XXX: It's impolite to ignore our argument and keep decoding until >> the >> +* current posit

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-24 Thread Andres Freund
On 2014-02-15 17:29:04 -0500, Robert Haas wrote: > On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund wrote: > + /* > +* XXX: It's impolite to ignore our argument and keep decoding until > the > +* current position. > +*/ > > Eh, what? So, the background here is that

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Jim Nasby
On 2/17/14, 7:31 PM, Robert Haas wrote: But do you really want to keep that snapshot around long enough to copy the entire database? I bet you don't: if the database is big, holding back xmin for long enough to copy the whole thing isn't likely to be fun. I can confirm that this would be epic

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-21 08:51:03 -0500, Robert Haas wrote: > On Fri, Feb 21, 2014 at 8:27 AM, Andres Freund wrote: > > On 2014-02-21 08:16:59 -0500, Robert Haas wrote: > >> On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund > >> wrote: > >> > I can sympathize with the "too much during init" argument, but I do

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Robert Haas
On Fri, Feb 21, 2014 at 8:27 AM, Andres Freund wrote: > On 2014-02-21 08:16:59 -0500, Robert Haas wrote: >> On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund >> wrote: >> > I can sympathize with the "too much during init" argument, but I don't >> > see how moving stuff to the first call would get r

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-21 08:16:59 -0500, Robert Haas wrote: > On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund wrote: > > I can sympathize with the "too much during init" argument, but I don't > > see how moving stuff to the first call would get rid of the problems. If > > we fail later it's going to be just a

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Robert Haas
On Fri, Feb 21, 2014 at 6:07 AM, Andres Freund wrote: > I can sympathize with the "too much during init" argument, but I don't > see how moving stuff to the first call would get rid of the problems. If > we fail later it's going to be just as confusing. No, it isn't. If you fail during init the

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-19 13:07:11 -0500, Robert Haas wrote: > On Tue, Feb 18, 2014 at 4:07 AM, Andres Freund wrote: > >> 2. I think the snapshot-export code is fundamentally misdesigned. As > >> I said before, the idea that we're going to export one single snapshot > >> at one particular point in time strik

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
On 2014-02-19 13:31:06 -0500, Robert Haas wrote: > TBH, as compared to what you've got now, I think this mostly boils > down to a question of quoting and escaping. I'm not really concerned > with whether we ship something that's perfectly efficient, or that has > filtering capabilities, or that ha

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-21 Thread Andres Freund
Hi, On 2014-02-19 13:01:02 -0500, Robert Haas wrote: > > I think it should be fairly easy to relax the restriction to creating a > > slot, but not getting data from it. Do you think that would that be > > sufficient? > > That would be a big improvement, for sure, and might be entirely sufficient.

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Euler Taveira
On 18-02-2014 06:33, Andres Freund wrote: > I really hope there will be nicer ones by the time 9.4 is > released. Euler did send in a json plugin > http://archives.postgresql.org/message-id/52A5BFAE.1040209%2540timbira.com.br > , but there hasn't too much feedback yet. It's hard to start discussing

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Tue, Feb 18, 2014 at 4:33 AM, Andres Freund wrote: > On 2014-02-17 21:35:23 -0500, Robert Haas wrote: >> What >> I don't understand is why we're not taking the test_decoding module, >> polishing it up a little to produce some nice, easily >> machine-parseable output, calling it basic_decoding,

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Tue, Feb 18, 2014 at 4:07 AM, Andres Freund wrote: >> 2. I think the snapshot-export code is fundamentally misdesigned. As >> I said before, the idea that we're going to export one single snapshot >> at one particular point in time strikes me as extremely short-sighted. > > I don't think so. I

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Sun, Feb 16, 2014 at 12:12 PM, Andres Freund wrote: > On 2014-02-15 17:29:04 -0500, Robert Haas wrote: >> On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund >> wrote: >> > [ new patches ] >> >> 0001 already needs minor >> >> + * copied stuff from tuptoaster.c. Perhaps there should be toast_intern

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-19 Thread Robert Haas
On Sat, Feb 15, 2014 at 6:59 PM, Andres Freund wrote: > On 2014-02-15 17:29:04 -0500, Robert Haas wrote: >> On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund >> wrote: >> > [ new patches ] >> >> 0001 already needs minor > > Hm? > > If there are conflicts, I'll push/send a rebased tomorrow or monday

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
On 2014-02-17 21:35:23 -0500, Robert Haas wrote: > What > I don't understand is why we're not taking the test_decoding module, > polishing it up a little to produce some nice, easily > machine-parseable output, calling it basic_decoding, and shipping > that. Then people who want something else can

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
On 2014-02-17 18:49:59 -0800, Peter Geoghegan wrote: > On Mon, Feb 17, 2014 at 6:35 PM, Robert Haas wrote: > > What I actually suspect is going to happen if we ship this as-is is > > that people are going to start building logical replication solutions > > on top of the test_decoding module even t

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
On 2014-02-17 21:10:26 -0500, Tom Lane wrote: > Robert Haas writes: > > 1. How safe is it to try to do decoding inside of a regular backend? > > What we're doing here is entering a special mode where we forbid the > > use of regular snapshots in favor of requiring the use of "decoding > > snapshot

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-18 Thread Andres Freund
Hi Robert, On 2014-02-17 20:31:34 -0500, Robert Haas wrote: > 1. How safe is it to try to do decoding inside of a regular backend? > What we're doing here is entering a special mode where we forbid the > use of regular snapshots in favor of requiring the use of "decoding > snapshots", and forbid a

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Peter Geoghegan
On Mon, Feb 17, 2014 at 6:35 PM, Robert Haas wrote: > What I actually suspect is going to happen if we ship this as-is is > that people are going to start building logical replication solutions > on top of the test_decoding module even though it explicitly says that > it's just test code. This is

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Robert Haas
On Mon, Feb 17, 2014 at 9:10 PM, Tom Lane wrote: >> 3. As this feature is proposed, the only plugin we'll ship with 9.4 is >> a test_decoding plugin which, as its own documentation says, "doesn't >> do anything especially useful." What exactly do we gain by forcing >> users who want to make use o

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Tom Lane
Robert Haas writes: > Having now had a little bit of opportunity to reflect on the State Of > This Patch, I'd like to step back from the minutia upon which I've > been commenting in my previous emails and articulate three high-level > concerns about this patch. In so doing, I would like to specif

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-17 Thread Robert Haas
On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund wrote: > [ patches ] Having now had a little bit of opportunity to reflect on the State Of This Patch, I'd like to step back from the minutia upon which I've been commenting in my previous emails and articulate three high-level concerns about this pa

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-16 Thread Andres Freund
On 2014-02-15 17:29:04 -0500, Robert Haas wrote: > On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund wrote: > > [ new patches ] > > 0001 already needs minor > > + * copied stuff from tuptoaster.c. Perhaps there should be toast_internal.h? > > Yes, please. If you can submit a separate patch creati

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-15 Thread Andres Freund
On 2014-02-15 17:29:04 -0500, Robert Haas wrote: > On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund wrote: > > [ new patches ] > > 0001 already needs minor Hm? If there are conflicts, I'll push/send a rebased tomorrow or monday. > +* the transaction's invalidations. This currently won't

Re: [HACKERS] Changeset Extraction v7.6.1

2014-02-15 Thread Robert Haas
On Fri, Feb 14, 2014 at 4:55 AM, Andres Freund wrote: > [ new patches ] 0001 already needs minor + * copied stuff from tuptoaster.c. Perhaps there should be toast_internal.h? Yes, please. If you can submit a separate patch creating this file and relocating this stuff there, I will commit it.