Re: [GENERAL] Logical Decoding Failover

2016-08-10 Thread Simon Riggs
On 10 August 2016 at 09:03, Colin Morelli wrote: > That's too bad. Logical decoding is incredibly powerful, but if it > ultimately comes down to picking between HA or logical decoding the choice > to use it becomes a lot more difficult. Agreed. > Are you saying that logical decoding is receivin

Re: [GENERAL] Logical Decoding Failover

2016-08-10 Thread Colin Morelli
Simon, That's too bad. Logical decoding is incredibly powerful, but if it ultimately comes down to picking between HA or logical decoding the choice to use it becomes a lot more difficult. Are you saying that logical decoding is receiving a different design in 10.0? (For clarify - is 10.0 the nex

Re: [GENERAL] Logical Decoding Failover

2016-08-10 Thread Simon Riggs
On 6 August 2016 at 04:17, Colin Morelli wrote: > I'm using logical decoding in my application to capture change streams and > ship them to Kafka. However, logical replication slots aren't included in > the WAL and thus don't make it to replicas. In the case of a failover, it's > not clear what (

Re: [GENERAL] Logical Decoding Failover

2016-08-09 Thread Venkata Balaji N
> Now the logical decoding client connects to B (the new primary). The > replication slot doesn't exist. So, it creates it and starts streaming. > This is where the problem lies - as it would begin streaming from LSN 4 > (anything after what has already been committed), because I have no way > (tha

Re: [GENERAL] Logical Decoding Failover

2016-08-08 Thread Colin Morelli
Venkata, No problem - thanks for replying. I'll try to clarify. Say I have databases A and B, where A is primary and B is a replica. I have an application that executes 3 transactions against A, resulting in WAL log sequence numbers 1, 2, and 3. The WAL is successfully replicated to B. I have a

Re: [GENERAL] Logical Decoding Failover

2016-08-08 Thread Venkata Balaji N
On Sun, Aug 7, 2016 at 9:29 PM, Colin Morelli wrote: > Venkata, > > Thanks for the reply. Unfortunately something like PgPool still won't > create the replication slots on all hosts, and record the LSN in a way that > is reusable on the secondary. > Yes, thats correct, pgPool does not have anyth

Re: [GENERAL] Logical Decoding Failover

2016-08-07 Thread Colin Morelli
Venkata, Thanks for the reply. Unfortunately something like PgPool still won't create the replication slots on all hosts, and record the LSN in a way that is reusable on the secondary. This sort of puts logical decoding at odds with HA, correct? In the case of master failover, there's no way to:

Re: [GENERAL] Logical Decoding Failover

2016-08-06 Thread Venkata Balaji N
On Sat, Aug 6, 2016 at 1:17 PM, Colin Morelli wrote: > Hey all, > > I'm using logical decoding in my application to capture change streams and > ship them to Kafka. However, logical replication slots aren't included in > the WAL and thus don't make it to replicas. In the case of a failover, it's

[GENERAL] Logical Decoding Failover

2016-08-05 Thread Colin Morelli
Hey all, I'm using logical decoding in my application to capture change streams and ship them to Kafka. However, logical replication slots aren't included in the WAL and thus don't make it to replicas. In the case of a failover, it's not clear what (if anything) can be done to continue receiving a