Re: logical replication snapshots

2018-07-30 Thread Dimitri Maziuk
On 07/27/2018 05:20 PM, Adrian Klaver wrote: > On 07/27/2018 03:04 PM, Dimitri Maziuk wrote: >> ... but if I did the publication for ALL, I could just use streaming >> replication ... > > Well I was just showing the extremes from a single table publication to > ALL tables. You can also do subsets

Re: logical replication snapshots

2018-07-27 Thread Adrian Klaver
On 07/27/2018 03:04 PM, Dimitri Maziuk wrote: On 07/26/2018 07:11 PM, Adrian Klaver wrote: On 07/26/2018 04:48 PM, Dimitri Maziuk wrote: ... The publication foopub is at this point fubar I take it? And needs to be re-created on the publisher and reconnected on the subscriber? Complete with wit

Re: logical replication snapshots

2018-07-27 Thread Dimitri Maziuk
On 07/26/2018 07:11 PM, Adrian Klaver wrote: > On 07/26/2018 04:48 PM, Dimitri Maziuk wrote: ... >> The publication foopub is at this point fubar I take it? And needs to be >> re-created on the publisher and reconnected on the subscriber? Complete >> with with inital resync? > > Not sure. Personal

Re: logical replication snapshots

2018-07-26 Thread Adrian Klaver
On 07/26/2018 04:48 PM, Dimitri Maziuk wrote: On 07/26/2018 05:34 PM, Adrian Klaver wrote: On 07/26/2018 03:01 PM, Dimitri Maziuk wrote: Let me ask a different question: if I drop and re-create a published table on the publisher without doing anything to the publication and subscription, what

Re: logical replication snapshots

2018-07-26 Thread Dimitri Maziuk
On 07/26/2018 05:34 PM, Adrian Klaver wrote: > On 07/26/2018 03:01 PM, Dimitri Maziuk wrote: >> Let me ask a different question: if I drop and re-create a published >> table on the publisher without doing anything to the publication and >> subscription, what happens? > > Take a look at: > > http

Re: logical replication snapshots

2018-07-26 Thread Adrian Klaver
On 07/26/2018 03:01 PM, Dimitri Maziuk wrote: On 07/26/2018 04:39 PM, Adrian Klaver wrote: The thing that has me somewhat confused is: ERROR: permission denied for schema macromolecules I would thought the replication user could access that. The more I look at the errors, the less sense it

Re: logical replication snapshots

2018-07-26 Thread Dimitri Maziuk
On 07/26/2018 04:39 PM, Adrian Klaver wrote: > The thing that has me somewhat confused is: > > ERROR: permission denied for schema macromolecules > > I would thought the replication user could access that. The more I look at the errors, the less sense it all makes. Let me ask a different quest

Re: logical replication snapshots

2018-07-26 Thread Adrian Klaver
On 07/26/2018 02:06 PM, Dimitri Maziuk wrote: On 07/26/2018 02:54 PM, Adrian Klaver wrote: On 07/26/2018 10:54 AM, Dimitri Maziuk wrote: I'm not sure what happened, I remember the initial sync of that particular schema failing on one table only, but looking at it now, all tables are empty on

Re: logical replication snapshots

2018-07-26 Thread Dimitri Maziuk
On 07/26/2018 02:54 PM, Adrian Klaver wrote: > On 07/26/2018 10:54 AM, Dimitri Maziuk wrote: >> I'm not sure what happened, I remember the initial sync of that >> particular schema failing on one table only, but looking at it now, all >> tables are empty on the subscriber. > > To me that indicate

Re: logical replication snapshots

2018-07-26 Thread Andres Freund
On 2018-07-26 12:54:19 -0500, Dimitri Maziuk wrote: > >> "LOG: logical decoding found consistent point at 19/E6942440" > >> DETAIL: There are no running transactions." > >> > >> Others with more experience in this area would need to fill whether that > >> might account for the 13 million files in

Re: logical replication snapshots

2018-07-26 Thread Adrian Klaver
On 07/26/2018 10:54 AM, Dimitri Maziuk wrote: On 07/25/2018 07:57 PM, Andres Freund wrote: On 2018-07-25 12:31:01 -0700, Adrian Klaver wrote: Would "permission denied" be relevant? Logical decoding is something I am still learning. The "permission denied" would to me be relevant only to the

Re: logical replication snapshots

2018-07-26 Thread Dimitri Maziuk
On 07/25/2018 07:57 PM, Andres Freund wrote: > On 2018-07-25 12:31:01 -0700, Adrian Klaver wrote: >>> Would "permission denied" be relevant? >> >> Logical decoding is something I am still learning. The "permission denied" >> would to me be relevant only to the extent that it seems to be provoking:

Re: logical replication snapshots

2018-07-25 Thread Andres Freund
On 2018-07-25 12:31:01 -0700, Adrian Klaver wrote: > On 07/25/2018 11:10 AM, Dimitri Maziuk wrote: > > On 07/25/2018 11:16 AM, Adrian Klaver wrote: > > > > > Where you using pg_export_snapshot() by any chance?: > > > > If it's not in chapter 31 of TFM then I have not touched it. The only > > nota

Re: logical replication snapshots

2018-07-25 Thread Adrian Klaver
On 07/25/2018 11:10 AM, Dimitri Maziuk wrote: On 07/25/2018 11:16 AM, Adrian Klaver wrote: Where you using pg_export_snapshot() by any chance?: If it's not in chapter 31 of TFM then I have not touched it. The only notable thing I changed from 31.9 quickstart is that replica identity is full o

Re: logical replication snapshots

2018-07-25 Thread Dimitri Maziuk
On 07/25/2018 02:31 PM, Adrian Klaver wrote: > In your previous post you said: > > "(To somebody else as I'm obviously not turning logical replication back > on until I know it won't kill my server again.) " > > So what are you working off now? I have that database rebuilt with no publications

Re: logical replication snapshots

2018-07-25 Thread Adrian Klaver
On 07/25/2018 11:10 AM, Dimitri Maziuk wrote: On 07/25/2018 11:16 AM, Adrian Klaver wrote: Where you using pg_export_snapshot() by any chance?: If it's not in chapter 31 of TFM then I have not touched it. The only notable thing I changed from 31.9 quickstart is that replica identity is full o

Re: logical replication snapshots

2018-07-25 Thread Dimitri Maziuk
On 07/25/2018 11:16 AM, Adrian Klaver wrote: > Where you using pg_export_snapshot() by any chance?: If it's not in chapter 31 of TFM then I have not touched it. The only notable thing I changed from 31.9 quickstart is that replica identity is full on all tables (the data's a mess). > Where there

Re: logical replication snapshots

2018-07-25 Thread Adrian Klaver
On 07/25/2018 08:40 AM, Dimitri Maziuk wrote: On 7/25/2018 10:28 AM, Andres Freund wrote: Are you really expecting us to be able to reproduce the problem based on the above description? Our test suites do setup plain replications setups, and the problem doesn't occur there. I don't, by defini

Re: logical replication snapshots

2018-07-25 Thread Dimitri Maziuk
On 7/25/2018 10:28 AM, Andres Freund wrote: Are you really expecting us to be able to reproduce the problem based on the above description? Our test suites do setup plain replications setups, and the problem doesn't occur there. I don't, by definition, have a reproducible case: it only happene

Re: logical replication snapshots

2018-07-25 Thread Andres Freund
On 2018-07-24 14:02:26 -0500, Dimitri Maziuk wrote: > On 07/24/2018 01:43 PM, Andres Freund wrote: > > On 2018-07-24 12:22:24 -0500, Dimitri Maziuk wrote: > >> On 07/24/2018 12:21 PM, Alvaro Herrera wrote: > >>> > >>> Are you serious about us trying to diagnose a bug with this description? > >> > >

Re: logical replication snapshots

2018-07-24 Thread Dimitri Maziuk
On 07/24/2018 01:43 PM, Andres Freund wrote: > On 2018-07-24 12:22:24 -0500, Dimitri Maziuk wrote: >> On 07/24/2018 12:21 PM, Alvaro Herrera wrote: >>> >>> Are you serious about us trying to diagnose a bug with this description? >> >> What do you want to know, exactly? > > A recipe that we can fol

Re: logical replication snapshots

2018-07-24 Thread Andres Freund
On 2018-07-24 12:22:24 -0500, Dimitri Maziuk wrote: > On 07/24/2018 12:21 PM, Alvaro Herrera wrote: > > > > Are you serious about us trying to diagnose a bug with this description? > > What do you want to know, exactly? A recipe that we can follow and reproduce the issue. Greetings, Andres Fre

Re: logical replication snapshots

2018-07-24 Thread Dimitri Maziuk
On 07/24/2018 12:21 PM, Alvaro Herrera wrote: > > Are you serious about us trying to diagnose a bug with this description? What do you want to know, exactly? -- Dimitri Maziuk Programmer/sysadmin BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu signature.asc Description: OpenPGP digital

Re: logical replication snapshots

2018-07-24 Thread Alvaro Herrera
On 2018-Jul-24, Dimitri Maziuk wrote: > On 7/23/2018 6:51 PM, Andres Freund wrote: > > Could you describe what exactly you did into that situation? > > Created a database, few publications, as per TFM, and a few subscriptions on > another host. Went on vacation for 3 weeks. The problem host is c

Re: logical replication snapshots

2018-07-24 Thread Dimitri Maziuk
On 7/23/2018 6:51 PM, Andres Freund wrote: What precedes that "loop"? systemctl start postgresql-10 Could you describe what exactly you did into that situation? Created a database, few publications, as per TFM, and a few subscriptions on another host. Went on vacation for 3 weeks. The pro

Re: logical replication snapshots

2018-07-23 Thread Andres Freund
Hi, On 2018-07-23 15:23:04 -0500, Dimitri Maziuk wrote: > I'm playing with logical replication in postgres-10 and I got it to hang > on startup with > > > 2018-07-23 13:15:06.577 CDT [18624] FATAL: the database system is starting > > up > > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: shmem_exit

logical replication snapshots

2018-07-23 Thread Dimitri Maziuk
Hi everyone, I'm playing with logical replication in postgres-10 and I got it to hang on startup with > 2018-07-23 13:15:06.577 CDT [18624] FATAL: the database system is starting up > 2018-07-23 13:15:06.577 CDT [18624] DEBUG: shmem_exit(1): 0 > before_shmem_exit callbacks to make > 2018-07-23