Re: Unnecessary delay in streaming replication due to replay lag

2021-12-16 Thread Kyotaro Horiguchi
At Wed, 15 Dec 2021 17:01:24 -0800, Soumyadeep Chakraborty wrote in > Sure, that makes more sense. Fixed. As I played with this briefly. I started a standby from a backup that has an access to archive. I had the following log lines steadily. [139535:postmaster] LOG: database system is read

Re: Unnecessary delay in streaming replication due to replay lag

2021-12-15 Thread Soumyadeep Chakraborty
Hi Bharath, Thanks for the review! On Sat, Nov 27, 2021 at 6:36 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > 1) A memory leak: add FreeDir(dir); before returning. > + ereport(LOG, > + (errmsg("Could not start streaming WAL eagerly"), > + errdetail("There are timeline c

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-27 Thread Bharath Rupireddy
On Tue, Nov 23, 2021 at 1:39 AM Soumyadeep Chakraborty wrote: > > Hi Bharath, > > Yes, that thread has been discussed here. Asim had x-posted the patch to [1]. > This thread > was more recent when Ashwin and I picked up the patch in Aug 2021, so we > continued here. > The patch has been signific

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-22 Thread Soumyadeep Chakraborty
Hi Bharath, Yes, that thread has been discussed here. Asim had x-posted the patch to [1]. This thread was more recent when Ashwin and I picked up the patch in Aug 2021, so we continued here. The patch has been significantly updated by us, addressing Michael's long outstanding feedback. Regards, S

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-22 Thread Bharath Rupireddy
On Fri, Nov 19, 2021 at 2:05 PM Soumyadeep Chakraborty wrote: > > Hi Daniel, > > Thanks for checking in on this patch. > Attached rebased version. Hi, I've not gone through the patch or this thread entirely, yet, can you please confirm if there's any relation between this thread and another one a

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-19 Thread Soumyadeep Chakraborty
Hi Daniel, Thanks for checking in on this patch. Attached rebased version. Regards, Soumyadeep (VMware) From 51149e4f877dc2f8bf47a1356fc8b0ec2512ac6d Mon Sep 17 00:00:00 2001 From: Soumyadeep Chakraborty Date: Fri, 19 Nov 2021 00:33:17 -0800 Subject: [PATCH v4 1/1] Introduce feature to start WAL

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-15 Thread Daniel Gustafsson
> On 10 Nov 2021, at 00:41, Soumyadeep Chakraborty > wrote: > Thanks for the detailed review! Attached is a rebased patch that addresses > most of the feedback. This patch no longer applies after e997a0c64 and associated follow-up commits, please submit a rebased version. -- Daniel Gustafsson

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-09 Thread Soumyadeep Chakraborty
Hi Michael, Thanks for the detailed review! Attached is a rebased patch that addresses most of the feedback. On Mon, Nov 8, 2021 at 1:41 AM Michael Paquier wrote: > +static void > +StartWALReceiverEagerly() > +{ > The patch fails to apply because of the recent changes from Robert to > eliminate

Re: Unnecessary delay in streaming replication due to replay lag

2021-11-08 Thread Michael Paquier
On Tue, Aug 24, 2021 at 09:51:25PM -0700, Soumyadeep Chakraborty wrote: > Ashwin and I recently got a chance to work on this and we addressed all > outstanding feedback and suggestions. PFA a significantly reworked patch. +static void +StartWALReceiverEagerly() +{ The patch fails to apply because

Re: Unnecessary delay in streaming replication due to replay lag

2021-10-25 Thread Soumyadeep Chakraborty
Rebased and added a CF entry for Nov CF: https://commitfest.postgresql.org/35/3376/.

Re: Unnecessary delay in streaming replication due to replay lag

2021-08-24 Thread Soumyadeep Chakraborty
Hello, Ashwin and I recently got a chance to work on this and we addressed all outstanding feedback and suggestions. PFA a significantly reworked patch. On 20.11.2020 11:21, Michael Paquier wrote: > This patch thinks that it is fine to request streaming even if > PrimaryConnInfo is not set, but

Re: Unnecessary delay in streaming replication due to replay lag

2020-12-01 Thread Anastasia Lubennikova
On 20.11.2020 11:21, Michael Paquier wrote: On Tue, Sep 15, 2020 at 05:30:22PM +0800, lchch1...@sina.cn wrote: I read the code and test the patch, it run well on my side, and I have several issues on the patch. + RequestXLogStreaming(ThisTimeLineID, +

Re: Unnecessary delay in streaming replication due to replay lag

2020-11-20 Thread Michael Paquier
On Tue, Sep 15, 2020 at 05:30:22PM +0800, lchch1...@sina.cn wrote: > I read the code and test the patch, it run well on my side, and I have > several issues on the > patch. + RequestXLogStreaming(ThisTimeLineID, +startpoint, +

Re: Unnecessary delay in streaming replication due to replay lag

2020-09-15 Thread lchch1...@sina.cn
Hello I read the code and test the patch, it run well on my side, and I have several issues on the patch. 1. When call RequestXLogStreaming() during replay, you pick timeline straightly from control file, do you think it should pick timeline from timeline history file? 2. In archive recovery m

Re: Unnecessary delay in streaming replication due to replay lag

2020-08-10 Thread Asim Praveen
> On 10-Aug-2020, at 12:27 PM, Masahiko Sawada > wrote: > > The patch can be applied cleanly to the current HEAD but I got the > error on building the code with this patch: > > xlog.c: In function 'StartupXLOG': > xlog.c:7315:6: error: too few arguments to function 'RequestXLogStreaming' > 73

Re: Unnecessary delay in streaming replication due to replay lag

2020-08-09 Thread Masahiko Sawada
On Sun, 9 Aug 2020 at 14:54, Asim Praveen wrote: > > I would like to revive this thready by submitting a rebased patch to start > streaming replication without waiting for startup process to finish replaying > all WAL. The start LSN for streaming is determined to be the LSN that points > to th

Re: Unnecessary delay in streaming replication due to replay lag

2020-08-09 Thread Asim Praveen
> On 09-Aug-2020, at 2:11 PM, Michael Paquier wrote: > > I have not really looked at the proposed patch, but it would be good > to have some documentation. > Ah, right. The basic idea is to reuse the logic to allow read-only connections to also start WAL streaming. The patch borrows a new

Re: Unnecessary delay in streaming replication due to replay lag

2020-08-09 Thread Michael Paquier
On Sun, Aug 09, 2020 at 05:54:32AM +, Asim Praveen wrote: > I would like to revive this thready by submitting a rebased patch to > start streaming replication without waiting for startup process to > finish replaying all WAL. The start LSN for streaming is determined > to be the LSN that point

Re: Unnecessary delay in streaming replication due to replay lag

2020-08-08 Thread Asim Praveen
I would like to revive this thready by submitting a rebased patch to start streaming replication without waiting for startup process to finish replaying all WAL. The start LSN for streaming is determined to be the LSN that points to the beginning of the most recently flushed WAL segment. The p

Re: Unnecessary delay in streaming replication due to replay lag

2020-01-17 Thread Asim R P
On Fri, Jan 17, 2020 at 11:08 AM Michael Paquier wrote: > > On Fri, Jan 17, 2020 at 09:34:05AM +0530, Asim R P wrote: > > > > 0001 - TAP test to demonstrate the problem. > > There is no real need for debug_replay_delay because we have already > recovery_min_apply_delay, no? That would count o

Re: Unnecessary delay in streaming replication due to replay lag

2020-01-16 Thread Michael Paquier
On Fri, Jan 17, 2020 at 09:34:05AM +0530, Asim R P wrote: > Standby does not start walreceiver process until startup process > finishes WAL replay. The more WAL there is to replay, longer is the > delay in starting streaming replication. If replication connection is > temporarily disconnected, th

Unnecessary delay in streaming replication due to replay lag

2020-01-16 Thread Asim R P
Hi Standby does not start walreceiver process until startup process finishes WAL replay. The more WAL there is to replay, longer is the delay in starting streaming replication. If replication connection is temporarily disconnected, this delay becomes a major problem and we are proposing a soluti