Repeated, never-ending deadlock

2022-04-19 Thread andrew cooke
Hi All, OK, apologies in advance, I feel like this is a rather unintelligent question that I probably know the answer to, but I am banging my head against a wall trying to understand what is happening. I have an ETL program, written in Python, that uses mutiple processes to read and load a bunc

Logical subscription / publication lifetimes

2022-04-22 Thread andrew cooke
If I define a publication at time Tp, then load some data on the publisher, then start a subscription at time Ts, then load some more data on the publisher, does the subscriber get data from Tp or Ts onwards? Also, if a subscription is disabled and then re-enabled does it lose the data inbetween

Re: Logical subscription / publication lifetimes

2022-04-22 Thread andrew cooke
see how else it could be practically implemented, but just want to be sure I am understanding. The idea that there are two phases (copy existing data then replicate operations) is a big help. Thanks again, Andrew On Fri, Apr 22, 2022 at 09:13:15AM -0700, David G. Johnston wrote: > On Fri, Ap

How many max_replication_slots?

2022-04-26 Thread andrew cooke
Hi, How can I estimate the correct value for max_replication_slots (logical replication, pg 14)? As far as I understand, they're needed to keep WAL files on the publisher when something goes wrong. But we seem to need way, way more slots than publishers. Is it one slot per publisher per table

Appending data locally to a logical replication subscriber

2022-05-30 Thread andrew cooke
Hi All, I would appreciate some advice on adding data to logically replicated tables on the subscriber. I am worried about contention between writes from local data loading and replication. We have 14 publisher databases (all with identical schema) that are constantly receiving new data. The