Re: Allow async standbys wait for sync replication

2022-03-17 Thread Bharath Rupireddy
On Sat, Mar 5, 2022 at 1:26 AM Nathan Bossart wrote: > > My point is that there are existing tools for alerting processes when an > LSN is synchronously replicated and for waking up WAL senders. What I am > proposing wouldn't involve spinning in XLogSendPhysical() waiting for > synchronous replic

Re: Allow async standbys wait for sync replication

2022-03-15 Thread Bharath Rupireddy
On Wed, Mar 9, 2022 at 7:31 AM Andres Freund wrote: > > Hi, > > On 2022-03-06 12:27:52 +0530, Bharath Rupireddy wrote: > > On Sun, Mar 6, 2022 at 1:57 AM Andres Freund wrote: > > > > > > Hi, > > > > > > On 2022-03-05 14:14:54 +0530, Bharath Rupireddy wrote: > > > > I understand. Even if we use th

Re: Allow async standbys wait for sync replication

2022-03-14 Thread Andres Freund
Hi, On 2022-03-14 11:30:02 +0900, Kyotaro Horiguchi wrote: > That is, (as my understanding) async standbys are required to allow > overwriting existing unreplayed records after reconnection. But, > putting aside how to remember that LSN, if that happens at a segment > boundary, the async replica

Re: Allow async standbys wait for sync replication

2022-03-13 Thread Kyotaro Horiguchi
At Mon, 14 Mar 2022 11:30:02 +0900 (JST), Kyotaro Horiguchi wrote in > At Sat, 12 Mar 2022 14:33:32 -0800, Nathan Bossart > wrote in > > On Tue, Mar 08, 2022 at 06:01:23PM -0800, Andres Freund wrote: > > > To me it's architecturally the completely wrong direction. We should move > > > in > >

Re: Allow async standbys wait for sync replication

2022-03-13 Thread Kyotaro Horiguchi
At Sat, 12 Mar 2022 14:33:32 -0800, Nathan Bossart wrote in > On Tue, Mar 08, 2022 at 06:01:23PM -0800, Andres Freund wrote: > > To me it's architecturally the completely wrong direction. We should move in > > the *other* direction, i.e. allow WAL to be sent to standbys before the > > primary ha

Re: Allow async standbys wait for sync replication

2022-03-12 Thread Nathan Bossart
On Tue, Mar 08, 2022 at 06:01:23PM -0800, Andres Freund wrote: > To me it's architecturally the completely wrong direction. We should move in > the *other* direction, i.e. allow WAL to be sent to standbys before the > primary has finished flushing it locally. Which requires similar > infrastructure

Re: Allow async standbys wait for sync replication

2022-03-08 Thread Andres Freund
Hi, On 2022-03-06 12:27:52 +0530, Bharath Rupireddy wrote: > On Sun, Mar 6, 2022 at 1:57 AM Andres Freund wrote: > > > > Hi, > > > > On 2022-03-05 14:14:54 +0530, Bharath Rupireddy wrote: > > > I understand. Even if we use the SyncRepWaitForLSN approach, the async > > > walsenders will have to do

Re: Allow async standbys wait for sync replication

2022-03-05 Thread Bharath Rupireddy
On Sun, Mar 6, 2022 at 1:57 AM Andres Freund wrote: > > Hi, > > On 2022-03-05 14:14:54 +0530, Bharath Rupireddy wrote: > > I understand. Even if we use the SyncRepWaitForLSN approach, the async > > walsenders will have to do nothing in WalSndLoop() until the sync > > walsender wakes them up via Sy

Re: Allow async standbys wait for sync replication

2022-03-05 Thread Andres Freund
Hi, On 2022-03-05 14:14:54 +0530, Bharath Rupireddy wrote: > I understand. Even if we use the SyncRepWaitForLSN approach, the async > walsenders will have to do nothing in WalSndLoop() until the sync > walsender wakes them up via SyncRepWakeQueue. I still think we should flat out reject this appr

Re: Allow async standbys wait for sync replication

2022-03-05 Thread Bharath Rupireddy
On Sat, Mar 5, 2022 at 1:26 AM Nathan Bossart wrote: > > On Wed, Mar 02, 2022 at 09:47:09AM +0530, Bharath Rupireddy wrote: > > On Wed, Mar 2, 2022 at 2:57 AM Nathan Bossart > > wrote: > >> I think there are a couple of advantages. For one, spinning is probably > >> not the best from a resource

Re: Allow async standbys wait for sync replication

2022-03-04 Thread Nathan Bossart
On Wed, Mar 02, 2022 at 09:47:09AM +0530, Bharath Rupireddy wrote: > On Wed, Mar 2, 2022 at 2:57 AM Nathan Bossart > wrote: >> I think there are a couple of advantages. For one, spinning is probably >> not the best from a resource perspective. > > Just to be on the same page - by spinning do yo

Re: Allow async standbys wait for sync replication

2022-03-01 Thread Bharath Rupireddy
On Wed, Mar 2, 2022 at 2:57 AM Nathan Bossart wrote: > > On Tue, Mar 01, 2022 at 11:09:57PM +0530, Bharath Rupireddy wrote: > > On Tue, Mar 1, 2022 at 10:35 PM Nathan Bossart > > wrote: > >> Yes, perhaps the synchronous replication framework will need to alert WAL > >> senders when the syncrep L

Re: Allow async standbys wait for sync replication

2022-03-01 Thread Nathan Bossart
On Tue, Mar 01, 2022 at 11:09:57PM +0530, Bharath Rupireddy wrote: > On Tue, Mar 1, 2022 at 10:35 PM Nathan Bossart > wrote: >> Yes, perhaps the synchronous replication framework will need to alert WAL >> senders when the syncrep LSN advances so that the WAL is sent to the async >> standbys. I'm

Re: Allow async standbys wait for sync replication

2022-03-01 Thread Bharath Rupireddy
On Tue, Mar 1, 2022 at 10:35 PM Nathan Bossart wrote: > > On Tue, Mar 01, 2022 at 04:34:31PM +0900, Kyotaro Horiguchi wrote: > > At Mon, 28 Feb 2022 22:05:28 -0800, Nathan Bossart > > wrote in > >> replicated LSN. TBH there are probably other things that need to be > >> considered (e.g., how do

Re: Allow async standbys wait for sync replication

2022-03-01 Thread Nathan Bossart
On Tue, Mar 01, 2022 at 04:34:31PM +0900, Kyotaro Horiguchi wrote: > At Mon, 28 Feb 2022 22:05:28 -0800, Nathan Bossart > wrote in >> replicated LSN. TBH there are probably other things that need to be >> considered (e.g., how do we ensure that the WAL sender sends the rest once >> it is replic

Re: Allow async standbys wait for sync replication

2022-02-28 Thread Kyotaro Horiguchi
(Now I understand what "async" mean here..) At Mon, 28 Feb 2022 22:05:28 -0800, Nathan Bossart wrote in > On Tue, Mar 01, 2022 at 11:10:09AM +0530, Bharath Rupireddy wrote: > > On Tue, Mar 1, 2022 at 12:27 AM Nathan Bossart > > wrote: > >> My feedback is specifically about this behavior. I d

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-28 Thread Nathan Bossart
On Tue, Mar 01, 2022 at 11:10:09AM +0530, Bharath Rupireddy wrote: > On Tue, Mar 1, 2022 at 12:27 AM Nathan Bossart > wrote: >> My feedback is specifically about this behavior. I don't think we should >> spin in XLogSend*() waiting for an LSN to be synchronously replicated. I >> think we should

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-28 Thread Bharath Rupireddy
On Tue, Mar 1, 2022 at 12:27 AM Nathan Bossart wrote: > > On Mon, Feb 28, 2022 at 06:45:51PM +0530, Bharath Rupireddy wrote: > > On Sat, Feb 26, 2022 at 9:37 PM Nathan Bossart > > wrote: > >> For > >> this feature, I think we always need to consider what the primary considers > >> synchronously

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-28 Thread Nathan Bossart
On Mon, Feb 28, 2022 at 06:45:51PM +0530, Bharath Rupireddy wrote: > On Sat, Feb 26, 2022 at 9:37 PM Nathan Bossart > wrote: >> For >> this feature, I think we always need to consider what the primary considers >> synchronously replicated. My suggested approach doesn't change that. I'm >> sayin

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-28 Thread Bharath Rupireddy
On Sat, Feb 26, 2022 at 9:37 PM Nathan Bossart wrote: > > On Sat, Feb 26, 2022 at 02:17:50PM +0530, Bharath Rupireddy wrote: > > A global min LSN of SendRqstPtr of all the sync standbys can be > > calculated and the async standbys can send WAL up to global min LSN. > > This is unlike what the v1 p

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-26 Thread Nathan Bossart
On Sat, Feb 26, 2022 at 02:17:50PM +0530, Bharath Rupireddy wrote: > A global min LSN of SendRqstPtr of all the sync standbys can be > calculated and the async standbys can send WAL up to global min LSN. > This is unlike what the v1 patch does i.e. async standbys will wait > until the sync standbys

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-26 Thread Bharath Rupireddy
On Sat, Feb 26, 2022 at 3:22 AM Hsu, John wrote: > > > On Fri, Feb 25, 2022 at 08:31:37PM +0530, Bharath Rupireddy wrote: > >> Thanks Satya and others for the inputs. Here's the v1 patch that > >> basically allows async wal senders to wait until the sync standbys > >> report their flush lsn back t

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-26 Thread Bharath Rupireddy
On Sat, Feb 26, 2022 at 1:08 AM Nathan Bossart wrote: > > On Fri, Feb 25, 2022 at 08:31:37PM +0530, Bharath Rupireddy wrote: > > Thanks Satya and others for the inputs. Here's the v1 patch that > > basically allows async wal senders to wait until the sync standbys > > report their flush lsn back t

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-25 Thread Nathan Bossart
On Fri, Feb 25, 2022 at 08:31:37PM +0530, Bharath Rupireddy wrote: > Thanks Satya and others for the inputs. Here's the v1 patch that > basically allows async wal senders to wait until the sync standbys > report their flush lsn back to the primary. Please let me know your > thoughts. I haven't had