Re: Allow logical failover slots to wait on synchronous replication

2024-09-23 Thread Amit Kapila
On Sat, Sep 21, 2024 at 6:34 AM John H wrote: > > On Fri, Sep 20, 2024 at 2:44 AM shveta malik wrote: > > > > > > > > > > The difference is that the purpose of 'synchronized_standby_slots' is > > > to mention slot names for which the user expects logical walsenders to > > > wait before sending th

Re: Allow logical failover slots to wait on synchronous replication

2024-09-20 Thread John H
Hi, On Fri, Sep 20, 2024 at 2:44 AM shveta malik wrote: > > > > > > > The difference is that the purpose of 'synchronized_standby_slots' is > > to mention slot names for which the user expects logical walsenders to > > wait before sending the logical changes to subscribers. OTOH, > > 'synchronous

Re: Allow logical failover slots to wait on synchronous replication

2024-09-20 Thread John H
Hi, On Mon, Sep 16, 2024 at 2:25 AM shveta malik wrote: > > > > > > If we don't do something similar, then aren't there chances that we > > > keep on waiting on the wrong lsn[mode] for the case when mode = > > > SYNC_REP_WAIT_APPLY while sync-rep-wait infrastructure is updating > > > different m

Re: Allow logical failover slots to wait on synchronous replication

2024-09-20 Thread shveta malik
On Thu, Sep 19, 2024 at 12:02 PM Amit Kapila wrote: > > On Tue, Sep 17, 2024 at 9:08 AM shveta malik wrote: > > > > On Mon, Sep 16, 2024 at 4:04 PM Amit Kapila wrote: > > > > > > On Mon, Sep 16, 2024 at 2:55 PM shveta malik > > > wrote: > > > > > > > > On Mon, Sep 16, 2024 at 11:13 AM Amit Kap

Re: Allow logical failover slots to wait on synchronous replication

2024-09-18 Thread Amit Kapila
On Tue, Sep 17, 2024 at 9:08 AM shveta malik wrote: > > On Mon, Sep 16, 2024 at 4:04 PM Amit Kapila wrote: > > > > On Mon, Sep 16, 2024 at 2:55 PM shveta malik wrote: > > > > > > On Mon, Sep 16, 2024 at 11:13 AM Amit Kapila > > > wrote: > > > > > > > > > > > Another question aside from the abo

Re: Allow logical failover slots to wait on synchronous replication

2024-09-16 Thread shveta malik
On Mon, Sep 16, 2024 at 4:04 PM Amit Kapila wrote: > > On Mon, Sep 16, 2024 at 2:55 PM shveta malik wrote: > > > > On Mon, Sep 16, 2024 at 11:13 AM Amit Kapila > > wrote: > > > > > > > > Another question aside from the above point, what if someone has > > > specified logical subscribers in sync

Re: Allow logical failover slots to wait on synchronous replication

2024-09-16 Thread Amit Kapila
On Mon, Sep 16, 2024 at 2:55 PM shveta malik wrote: > > On Mon, Sep 16, 2024 at 11:13 AM Amit Kapila wrote: > > > > > Another question aside from the above point, what if someone has > > specified logical subscribers in synchronous_standby_names? In the > > case of synchronized_standby_slots, we

Re: Allow logical failover slots to wait on synchronous replication

2024-09-16 Thread shveta malik
On Mon, Sep 16, 2024 at 11:13 AM Amit Kapila wrote: > > On Fri, Sep 13, 2024 at 3:13 PM shveta malik wrote: > > > > On Thu, Sep 12, 2024 at 3:04 PM shveta malik wrote: > > > > > > On Wed, Sep 11, 2024 at 2:40 AM John H wrote: > > > > > > > > Hi Shveta, > > > > > > > > On Sun, Sep 8, 2024 at 11:

Re: Allow logical failover slots to wait on synchronous replication

2024-09-15 Thread Amit Kapila
On Fri, Sep 13, 2024 at 3:13 PM shveta malik wrote: > > On Thu, Sep 12, 2024 at 3:04 PM shveta malik wrote: > > > > On Wed, Sep 11, 2024 at 2:40 AM John H wrote: > > > > > > Hi Shveta, > > > > > > On Sun, Sep 8, 2024 at 11:16 PM shveta malik > > > wrote: > > > > > > > > > > > I was trying to h

Re: Allow logical failover slots to wait on synchronous replication

2024-09-13 Thread shveta malik
On Thu, Sep 12, 2024 at 3:04 PM shveta malik wrote: > > On Wed, Sep 11, 2024 at 2:40 AM John H wrote: > > > > Hi Shveta, > > > > On Sun, Sep 8, 2024 at 11:16 PM shveta malik wrote: > > > > > > > > I was trying to have a look at the patch again, it doesn't apply on > > > the head, needs rebase. >

Re: Allow logical failover slots to wait on synchronous replication

2024-09-12 Thread shveta malik
On Wed, Sep 11, 2024 at 2:40 AM John H wrote: > > Hi Shveta, > > On Sun, Sep 8, 2024 at 11:16 PM shveta malik wrote: > > > > > I was trying to have a look at the patch again, it doesn't apply on > > the head, needs rebase. > > > > Rebased with the latest changes. > > > Regarding 'mode = SyncRepWa

Re: Allow logical failover slots to wait on synchronous replication

2024-09-10 Thread John H
Hi Shveta, On Sun, Sep 8, 2024 at 11:16 PM shveta malik wrote: > > I was trying to have a look at the patch again, it doesn't apply on > the head, needs rebase. > Rebased with the latest changes. > Regarding 'mode = SyncRepWaitMode', FWIW, SyncRepWaitForLSN() also > does in a similar way. It g

Re: Allow logical failover slots to wait on synchronous replication

2024-09-08 Thread shveta malik
On Fri, Aug 30, 2024 at 12:56 AM John H wrote: > > Hi Shveta, > > Thanks for reviewing it so quickly. > > On Thu, Aug 29, 2024 at 2:35 AM shveta malik wrote: > > > > Thanks for the patch. Few comments and queries: > > > > 1) > > + static XLogRecPtr lsn[NUM_SYNC_REP_WAIT_MODE]; > > > > We sh

Re: Allow logical failover slots to wait on synchronous replication

2024-08-29 Thread John H
Hi Shveta, Thanks for reviewing it so quickly. On Thu, Aug 29, 2024 at 2:35 AM shveta malik wrote: > > Thanks for the patch. Few comments and queries: > > 1) > + static XLogRecPtr lsn[NUM_SYNC_REP_WAIT_MODE]; > > We shall name it as 'lsns' as there are multiple. > This follows the same na

Re: Allow logical failover slots to wait on synchronous replication

2024-08-29 Thread shveta malik
On Thu, Aug 29, 2024 at 2:31 AM John H wrote: > > Hi Amit, > > On Mon, Aug 26, 2024 at 11:00 PM Amit Kapila wrote: > > I wanted a simple test where in the first case you use > > synchronous_standby_names = 'ANY 3 (A,B,C,D,E)' and in the second case > > use standby_slot_names = A_slot, B_slot, C_s

Re: Allow logical failover slots to wait on synchronous replication

2024-08-28 Thread John H
Hi Amit, On Mon, Aug 26, 2024 at 11:00 PM Amit Kapila wrote: > I wanted a simple test where in the first case you use > synchronous_standby_names = 'ANY 3 (A,B,C,D,E)' and in the second case > use standby_slot_names = A_slot, B_slot, C_slot, D_slot, E_slot. You > can try some variations of it as

Re: Allow logical failover slots to wait on synchronous replication

2024-08-27 Thread shveta malik
On Tue, Aug 27, 2024 at 12:56 AM John H wrote: > > Hi Shveta, > > On Sun, Jul 21, 2024 at 8:42 PM shveta malik wrote: > > > > Ah that's a gap. Let me add some logging/warning in a similar fashion. > > > Although I think I'd have the warning be relatively generic (e.g. > > > changes are blocked be

Re: Allow logical failover slots to wait on synchronous replication

2024-08-26 Thread Amit Kapila
On Tue, Aug 27, 2024 at 12:58 AM John H wrote: > > For instance, in Shveta's suggestion of > > > > > We can perform this test with both of the below settings and say make > > > > D and E slow in sending responses: > > > > 1) synchronous_standby_names = 'ANY 3 (A,B,C,D,E)' > > > > 2) standby_slot_n

Re: Allow logical failover slots to wait on synchronous replication

2024-08-26 Thread John H
Hi Bertrand, On Sun, Jul 28, 2024 at 10:00 PM Bertrand Drouvot wrote: > Yeah, at the same place as the static lsn[] declaration, something like: > > static XLogRecPtr lsn[NUM_SYNC_REP_WAIT_MODE]; /* cached LSNs */ > > but that may just be a matter of taste. > I've updated the patch to ref

Re: Allow logical failover slots to wait on synchronous replication

2024-08-26 Thread John H
Hi Shveta, Amit, > > > > ... We should try to > > > > find out if there is a performance benefit with the use of > > > > synchronous_standby_names in the normal configurations like the one > > > > you used in the above tests to prove the value of this patch. I don't expect there to be a performan

Re: Allow logical failover slots to wait on synchronous replication

2024-08-26 Thread John H
Hi Shveta, On Sun, Jul 21, 2024 at 8:42 PM shveta malik wrote: > > Ah that's a gap. Let me add some logging/warning in a similar fashion. > > Although I think I'd have the warning be relatively generic (e.g. > > changes are blocked because > > they're not synchronously committed) > > > > okay, s

Re: Allow logical failover slots to wait on synchronous replication

2024-07-28 Thread Bertrand Drouvot
Hi John, On Thu, Jul 18, 2024 at 02:22:08PM -0700, John H wrote: > Hi Bertrand, > > > 1 === > > ... > > That's worth additional comments in the code. > > There's this comment already about caching the value already, not sure > if you prefer something more? > > /* Cache values to reduce contenti

Re: Allow logical failover slots to wait on synchronous replication

2024-07-28 Thread shveta malik
On Fri, Jul 26, 2024 at 5:11 PM Amit Kapila wrote: > > On Fri, Jul 26, 2024 at 3:28 PM shveta malik wrote: > > > > On Tue, Jul 23, 2024 at 10:35 AM Amit Kapila > > wrote: > > > > > > On Tue, Jul 9, 2024 at 12:39 AM John H wrote: > > > > > > > > > Out of curiosity, did you compare with > > > >

Re: Allow logical failover slots to wait on synchronous replication

2024-07-26 Thread Amit Kapila
On Fri, Jul 26, 2024 at 3:28 PM shveta malik wrote: > > On Tue, Jul 23, 2024 at 10:35 AM Amit Kapila wrote: > > > > On Tue, Jul 9, 2024 at 12:39 AM John H wrote: > > > > > > > Out of curiosity, did you compare with standby_slot_names_from_syncrep > > > > set to off > > > > and standby_slot_name

Re: Allow logical failover slots to wait on synchronous replication

2024-07-26 Thread shveta malik
On Tue, Jul 23, 2024 at 10:35 AM Amit Kapila wrote: > > On Tue, Jul 9, 2024 at 12:39 AM John H wrote: > > > > > Out of curiosity, did you compare with standby_slot_names_from_syncrep > > > set to off > > > and standby_slot_names not empty? > > > > I didn't think 'standby_slot_names' would impact

Re: Allow logical failover slots to wait on synchronous replication

2024-07-22 Thread Amit Kapila
On Tue, Jul 9, 2024 at 12:39 AM John H wrote: > > > Out of curiosity, did you compare with standby_slot_names_from_syncrep set > > to off > > and standby_slot_names not empty? > > I didn't think 'standby_slot_names' would impact TPS as much since > it's not grabbing the SyncRepLock but here's a q

Re: Allow logical failover slots to wait on synchronous replication

2024-07-22 Thread Amit Kapila
On Mon, Jul 22, 2024 at 9:12 AM shveta malik wrote: > > On Fri, Jul 19, 2024 at 2:52 AM John H wrote: > > > > Hi Shveta, > > > > Thanks for taking a look at the patch. > > > > > > will leave user no option to unlink failover-enabled logical > > > > subscribers from the wait on synchronous standby

Re: Allow logical failover slots to wait on synchronous replication

2024-07-21 Thread shveta malik
On Fri, Jul 19, 2024 at 2:52 AM John H wrote: > > Hi Shveta, > > Thanks for taking a look at the patch. > > > > will leave user no option to unlink failover-enabled logical > > > subscribers from the wait on synchronous standbys. > > That's a good point. I'm a bit biased in that I don't think ther

Re: Allow logical failover slots to wait on synchronous replication

2024-07-18 Thread John H
Hi Bertrand, > 1 === > ... > That's worth additional comments in the code. There's this comment already about caching the value already, not sure if you prefer something more? /* Cache values to reduce contention on lock */ > 2 === > ... > Looks like setting initialized to true is missing once

Re: Allow logical failover slots to wait on synchronous replication

2024-07-18 Thread John H
Hi Shveta, Thanks for taking a look at the patch. > > will leave user no option to unlink failover-enabled logical > > subscribers from the wait on synchronous standbys. That's a good point. I'm a bit biased in that I don't think there's a great reason why someone would want to replicate logical

Re: Allow logical failover slots to wait on synchronous replication

2024-07-17 Thread shveta malik
On Thu, Jul 18, 2024 at 9:25 AM shveta malik wrote: > > On Tue, Jul 9, 2024 at 12:42 AM John H wrote: > > > > > > > Can we make it a default > > > behavior that logical slots marked with a failover option will wait > > > for 'synchronous_standby_names' as per your patch's idea unless > > > 'stand

Re: Allow logical failover slots to wait on synchronous replication

2024-07-17 Thread shveta malik
On Tue, Jul 9, 2024 at 12:42 AM John H wrote: > > > > Can we make it a default > > behavior that logical slots marked with a failover option will wait > > for 'synchronous_standby_names' as per your patch's idea unless > > 'standby_slot_names' is specified? I don't know if there is any value > > i

Re: Allow logical failover slots to wait on synchronous replication

2024-07-10 Thread Bertrand Drouvot
Hi, On Mon, Jul 08, 2024 at 12:08:58PM -0700, John H wrote: > I took a deeper look at this with GDB and I think it's necessary to > cache the value of "mode". > We first check: > > if (mode == SYNC_REP_NO_WAIT) > return true; > > However after this check it's possible to receive a SIGHUP changin

Re: Allow logical failover slots to wait on synchronous replication

2024-07-08 Thread John H
Hi Amit, Thanks for taking a look. On Tue, Jun 18, 2024 at 10:34 PM Amit Kapila wrote: > > > The reading indicates when you set 'standby_slot_names_from_syncrep', > the TPS reduces as compared to when it is not set. It would be better > to see the data comparing 'standby_slot_names_from_syncrep

Re: Allow logical failover slots to wait on synchronous replication

2024-07-08 Thread John H
Hi, Thanks Bertrand for taking a look at the patch. On Mon, Jun 17, 2024 at 8:19 AM Bertrand Drouvot wrote: > > + int mode = SyncRepWaitMode; > > It's set to SyncRepWaitMode and then never change. Worth to get rid of "mode"? > I took a deeper look at this with GDB and I think it'

Re: Allow logical failover slots to wait on synchronous replication

2024-06-18 Thread Amit Kapila
On Tue, Jun 11, 2024 at 4:21 AM John H wrote: > > Building on bf279ddd1c, this patch introduces a GUC > 'standby_slot_names_from_syncrep' which allows logical failover slots > to wait for changes to have been synchronously replicated before sending > the decoded changes to logical subscribers. > >

Re: Allow logical failover slots to wait on synchronous replication

2024-06-17 Thread Bertrand Drouvot
Hi, On Tue, Jun 11, 2024 at 10:00:46AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Jun 10, 2024 at 09:25:10PM -0500, Nathan Bossart wrote: > > On Mon, Jun 10, 2024 at 03:51:05PM -0700, John H wrote: > > > The existing 'standby_slot_names' isn't great for users who are running > > > clusters

Re: Allow logical failover slots to wait on synchronous replication

2024-06-11 Thread Bertrand Drouvot
Hi, On Mon, Jun 10, 2024 at 09:25:10PM -0500, Nathan Bossart wrote: > On Mon, Jun 10, 2024 at 03:51:05PM -0700, John H wrote: > > The existing 'standby_slot_names' isn't great for users who are running > > clusters with quorum-based synchronous replicas. For instance, if > > the user has synchron

Re: Allow logical failover slots to wait on synchronous replication

2024-06-10 Thread Nathan Bossart
On Mon, Jun 10, 2024 at 03:51:05PM -0700, John H wrote: > The existing 'standby_slot_names' isn't great for users who are running > clusters with quorum-based synchronous replicas. For instance, if > the user has synchronous_standby_names = 'ANY 3 (A,B,C,D,E)' it's a > bit tedious to have to recon