On Thu, Jan 02, 2025 at 11:12:37PM +0500, Andrey M. Borodin wrote:
> In my observation restore from archive is many orders of magnitude
> faster than streaming replication. Advanced archive tools employ
> compression (x6 to speed), download parallelism (x4), are not
> constrained be primary's netwo
> On 23 Mar 2024, at 14:22, Bharath Rupireddy
> wrote:
>
> IMHO, it makes sense to have something like replay_source_order if
> there's any use case that arises in future requiring the standby to
> intentionally switch to pg_wal or archive. But not as part of this
> feature.
IMO, it's vital
Hi,
On Thu, Aug 29, 2024 at 6:32 AM Bharath Rupireddy
wrote:
> In synchronous replication setup, until standby finishes fetching WAL
> from the archive, the commits on the primary have to wait which can
> increase the query latency. If the standby can connect to the primary
> as soon as the broke
Hi,
Thanks for looking into this.
On Fri, Aug 23, 2024 at 5:03 AM John H wrote:
>
> For a motivation aspect I can see this being useful
> synchronous_replicas if you have commit set to flush mode.
In synchronous replication setup, until standby finishes fetching WAL
from the archive, the commit
Hi,
I took a brief look at the patch.
For a motivation aspect I can see this being useful
synchronous_replicas if you have commit set to flush mode.
So +1 on feature, easier configurability, although thinking about it
more you could probably have the restore script be smarter and provide
non-zero
On Mon, Mar 18, 2024 at 11:38 AM Michael Paquier wrote:
>
> On Sun, Mar 17, 2024 at 11:37:58AM +0530, Bharath Rupireddy wrote:
> > Rebase needed after 071e3ad59d6fd2d6d1277b2bd9579397d10ded28 due to a
> > conflict in meson.build. Please see the attached v23 patch.
>
> I've been reading this patch,
On Sun, Mar 17, 2024 at 11:37:58AM +0530, Bharath Rupireddy wrote:
> Rebase needed after 071e3ad59d6fd2d6d1277b2bd9579397d10ded28 due to a
> conflict in meson.build. Please see the attached v23 patch.
I've been reading this patch, and this is a very tricky one. Please
be *very* cautious.
+#strea
On Wed, Mar 6, 2024 at 9:49 PM Nathan Bossart wrote:
>
> > I played with that idea and it came out very nice. Please see the
> > attached v22 patch. Note that personally I didn't like "FORCE" being
> > there in the names, so I've simplified them a bit.
>
> Thanks. I'd like to spend some time test
On Wed, Mar 06, 2024 at 10:02:43AM +0530, Bharath Rupireddy wrote:
> On Wed, Mar 6, 2024 at 1:22 AM Nathan Bossart
> wrote:
>> I was thinking of something more like
>>
>> typedef enum
>> {
>> NO_FORCE_SWITCH_TO_STREAMING, /* no switch
>> necessary */
>>
On Wed, Mar 6, 2024 at 1:22 AM Nathan Bossart wrote:
>
> I was thinking of something more like
>
> typedef enum
> {
> NO_FORCE_SWITCH_TO_STREAMING, /* no switch
> necessary */
> FORCE_SWITCH_TO_STREAMING_PENDING, /* exhausting pg_wal
On Tue, Mar 05, 2024 at 11:38:37PM +0530, Bharath Rupireddy wrote:
> On Tue, Mar 5, 2024 at 7:34 AM Nathan Bossart
> wrote:
>> Is there any way to simplify this? For
>> example, would it be possible to make an enum that tracks the
>> streaming_replication_retry_interval state?
>
> I guess the w
On Tue, Mar 5, 2024 at 7:34 AM Nathan Bossart wrote:
>
> cfbot claims that this one needs another rebase.
Yeah, the conflict was with the new TAP test file name in
src/test/recovery/meson.build.
> I've spent some time thinking about this one. I'll admit I'm a bit worried
> about adding more com
cfbot claims that this one needs another rebase.
I've spent some time thinking about this one. I'll admit I'm a bit worried
about adding more complexity to this state machine, but I also haven't
thought of any other viable approaches, and this still seems like a useful
feature. So, for now, I th
On Tue, Feb 20, 2024 at 11:54 AM Japin Li wrote:
>
>
> On Tue, 20 Feb 2024 at 13:40, Bharath Rupireddy
> wrote:
> > On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote:
> >> [2]
> >> +# Ensure checkpoint doesn't come in our way
> >> +$primary->append_conf('postgresql.conf', qq(
> >> +min_wal_siz
On Tue, 20 Feb 2024 at 13:40, Bharath Rupireddy
wrote:
> On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote:
>> [2]
>> +# Ensure checkpoint doesn't come in our way
>> +$primary->append_conf('postgresql.conf', qq(
>> +min_wal_size = 2MB
>> +max_wal_size = 1GB
>> +checkpoint_timeout = 1h
On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote:
>
> > Strengthened tests a bit by using recovery_min_apply_delay to mimic
> > standby spending some time fetching from archive. PSA v18 patch.
>
> Here are some minor comments:
Thanks for taking a look at it.
> [1]
> +primary). However, the
On Mon, 19 Feb 2024 at 18:36, Bharath Rupireddy
wrote:
> On Wed, Jan 31, 2024 at 6:30 PM Bharath Rupireddy
> wrote:
>>
>> Needed a rebase due to commit 776621a (conflict in
>> src/test/recovery/meson.build for new TAP test file added). Please
>> find the attached v17 patch.
>
> Strengthened te
On Wed, Jan 31, 2024 at 6:30 PM Bharath Rupireddy
wrote:
>
> Needed a rebase due to commit 776621a (conflict in
> src/test/recovery/meson.build for new TAP test file added). Please
> find the attached v17 patch.
Strengthened tests a bit by using recovery_min_apply_delay to mimic
standby spending
On Wed, Jan 3, 2024 at 4:58 PM Bharath Rupireddy
wrote:
>
> On Thu, Dec 28, 2023 at 5:26 PM Bharath Rupireddy
> wrote:
> >
> > I took a closer look at v14 and came up with the following changes:
> >
> > 1. Used advance_wal introduced by commit c161ab74f7.
> > 2. Simplified the core logic and new
On Thu, Dec 28, 2023 at 5:26 PM Bharath Rupireddy
wrote:
>
> I took a closer look at v14 and came up with the following changes:
>
> 1. Used advance_wal introduced by commit c161ab74f7.
> 2. Simplified the core logic and new TAP tests.
> 3. Reworded the comments and docs.
> 4. Simplified new DEBUG
On Sat, Oct 21, 2023 at 11:59 PM Bharath Rupireddy
wrote:
>
> On Fri, Jul 21, 2023 at 12:38 PM Bharath Rupireddy
> wrote:
> >
> > Needed a rebase. I'm attaching the v13 patch for further consideration.
>
> Needed a rebase. I'm attaching the v14 patch. It also has the following
> changes:
>
> - R
On Fri, Jul 21, 2023 at 12:38 PM Bharath Rupireddy
wrote:
>
> Needed a rebase. I'm attaching the v13 patch for further consideration.
Needed a rebase. I'm attaching the v14 patch. It also has the following changes:
- Ran pgindent on the new source code.
- Ran pgperltidy on the new TAP test.
- Im
On Tue, Apr 25, 2023 at 9:27 PM Bharath Rupireddy
wrote:
>
> > Needed a rebase. I'm attaching the v11 patch for further review.
>
> Needed a rebase, so attaching the v12 patch. I word-smithed comments
> and docs a bit.
Needed a rebase. I'm attaching the v13 patch for further consideration.
--
B
On Fri, Feb 24, 2023 at 10:26 AM Bharath Rupireddy
wrote:
>
> On Wed, Nov 16, 2022 at 11:39 AM Bharath Rupireddy
> wrote:
> >
> > I'm attaching the v10 patch for further review.
>
> Needed a rebase. I'm attaching the v11 patch for further review.
Needed a rebase, so attaching the v12 patch. I wo
On Wed, Nov 16, 2022 at 11:39 AM Bharath Rupireddy
wrote:
>
> I'm attaching the v10 patch for further review.
Needed a rebase. I'm attaching the v11 patch for further review.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
F
On Thu, Jan 19, 2023 at 6:20 AM Nathan Bossart wrote:
>
> On Tue, Jan 17, 2023 at 07:44:52PM +0530, Bharath Rupireddy wrote:
> > On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart
> > wrote:
> >> With your patch, we might replay one of these "old" files in pg_wal instead
> >> of the complete version
On Tue, Jan 17, 2023 at 07:44:52PM +0530, Bharath Rupireddy wrote:
> On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart
> wrote:
>> With your patch, we might replay one of these "old" files in pg_wal instead
>> of the complete version of the file from the archives,
>
> That's true even today, withou
On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart wrote:
>
> On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote:
> > In summary, the standby state machine in WaitForWALToBecomeAvailable()
> > exhausts all the WAL in pg_wal before switching to streaming after
> > failing to fetch from a
On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote:
> In summary, the standby state machine in WaitForWALToBecomeAvailable()
> exhausts all the WAL in pg_wal before switching to streaming after
> failing to fetch from archive. The v8 patch proposed upthread deviates
> from this behav
On Wed, Nov 16, 2022 at 9:38 AM Ian Lawrence Barwick wrote:
>
> While reviewing the patch backlog, we have determined that this patch adds
> one or more TAP tests but has not added the test to the "meson.build" file.
Thanks for pointing it out. Yeah, the test wasn't picking up on meson
builds. I
2022年10月18日(火) 11:02 Bharath Rupireddy :
>
> On Tue, Oct 11, 2022 at 8:40 AM Nathan Bossart
> wrote:
> >
> > On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote:
> > > On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart
> > > wrote:
> > >> I wonder if it would be better to simply remov
On Tue, Oct 11, 2022 at 8:40 AM Nathan Bossart wrote:
>
> On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote:
> > On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart
> > wrote:
> >> I wonder if it would be better to simply remove this extra polling of
> >> pg_wal as a prerequisite to y
On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote:
> On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart
> wrote:
>> I wonder if it would be better to simply remove this extra polling of
>> pg_wal as a prerequisite to your patch. The existing commentary leads me
>> to think there migh
On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart wrote:
>
> > Instead, the simplest would be to just pass XLOG_FROM_WAL to
> > XLogFileReadAnyTLI() when we're about to switch the source to stream
> > mode. This doesn't change the existing behaviour.
>
> It might be more consistent with existing beha
On Sun, Oct 09, 2022 at 02:39:47PM +0530, Bharath Rupireddy wrote:
> We can give it a chance to restore from pg_wal before switching to
> streaming to not change any behaviour of the state machine. But, not
> definitely by setting currentSource to XLOG_FROM_WAL, we basically
> never explicitly set
On Sun, Oct 9, 2022 at 3:22 AM Nathan Bossart wrote:
>
> As I mentioned upthread [0], I'm still a little concerned that this patch
> will cause the state machine to go straight from archive recovery to
> streaming replication, skipping recovery from pg_wal.
>
> [0] https://postgr.es/m/202209062157
On Mon, Sep 19, 2022 at 07:49:21PM +0530, Bharath Rupireddy wrote:
> SwitchFromArchiveToStreamEnabled() seemed better at this point. I'm
> attaching the v7 patch with that change. Please review it further.
As I mentioned upthread [0], I'm still a little concerned that this patch
will cause the sta
On Fri, Sep 16, 2022 at 4:58 PM Bharath Rupireddy
wrote:
>
> On Fri, Sep 16, 2022 at 12:06 PM Kyotaro Horiguchi
> wrote:
> >
> > In other words, it seems to me that the macro name doesn't manifest
> > the condition correctly.
> >
> > I don't think we don't particularly want to do that uncondition
On Fri, Sep 16, 2022 at 12:06 PM Kyotaro Horiguchi
wrote:
>
> In other words, it seems to me that the macro name doesn't manifest
> the condition correctly.
>
> I don't think we don't particularly want to do that unconditionally.
> I wanted just to get rid of the macro from the usage site. Even i
At Fri, 16 Sep 2022 09:15:58 +0530, Bharath Rupireddy
wrote in
> On Thu, Sep 15, 2022 at 1:52 PM Kyotaro Horiguchi
> wrote:
> >
> > At Thu, 15 Sep 2022 10:28:12 +0530, Bharath Rupireddy
> > wrote in
> > > I'm attaching the v6 patch that's rebased on to the latest HEAD.
> > > Please consider t
On Thu, Sep 15, 2022 at 1:52 PM Kyotaro Horiguchi
wrote:
>
> At Thu, 15 Sep 2022 10:28:12 +0530, Bharath Rupireddy
> wrote in
> > I'm attaching the v6 patch that's rebased on to the latest HEAD.
> > Please consider this for review.
>
> Thaks for the new version!
>
> +#define StreamingReplRetryEn
At Thu, 15 Sep 2022 10:28:12 +0530, Bharath Rupireddy
wrote in
> I'm attaching the v6 patch that's rebased on to the latest HEAD.
> Please consider this for review.
Thaks for the new version!
+#define StreamingReplRetryEnabled() \
+ (streaming_replication_retry_interval > 0 && \
+
On Mon, Sep 12, 2022 at 11:56 AM Bharath Rupireddy
wrote:
>
> Please review the attached v5 patch.
I'm attaching the v6 patch that's rebased on to the latest HEAD.
Please consider this for review.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: ht
On Mon, Sep 12, 2022 at 9:03 AM Bharath Rupireddy
wrote:
>
> Please review the attached v4 patch addressing above review comments.
Oops, there's a compiler warning [1] with the v4 patch, fixed it.
Please review the attached v5 patch.
[1] https://cirrus-ci.com/task/5730076611313664?logs=gcc_warni
On Sat, Sep 10, 2022 at 3:35 AM Nathan Bossart wrote:
>
> Well, for wal_keep_size, using bytes makes sense. Given you know how much
> disk space you have, you can set this parameter accordingly to avoid
> retaining too much of it for standby servers. For your proposed parameter,
> it's not so si
On Fri, Sep 09, 2022 at 11:07:00PM +0530, Bharath Rupireddy wrote:
> On Fri, Sep 9, 2022 at 10:29 PM Nathan Bossart
> wrote:
>> IMO the timeout approach would be more intuitive for users. When it comes
>> to archive recovery, "WAL segment" isn't a standard unit of measure. WAL
>> segment size c
On Fri, Sep 9, 2022 at 10:29 PM Nathan Bossart wrote:
>
> On Fri, Sep 09, 2022 at 12:14:25PM +0530, Bharath Rupireddy wrote:
> > On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi
> > wrote:
> >> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
> >> wrote in
> >> > My general point is that we s
On Fri, Sep 09, 2022 at 12:14:25PM +0530, Bharath Rupireddy wrote:
> On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi
> wrote:
>> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
>> wrote in
>> > My general point is that we should probably offer some basic preventative
>> > measure against fli
On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi
wrote:
>
> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
> wrote in
> > On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote:
> > > I'm attaching the v3 patch with the review comments addressed, please
> > > review it further.
>
At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
wrote in
> On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote:
> > I'm attaching the v3 patch with the review comments addressed, please
> > review it further.
>
> My general point is that we should probably offer some basic preve
Being late for the party.
It seems to me that the function is getting too long. I think we
might want to move the core part of the patch into another function.
I think it might be better if intentionalSourceSwitch doesn't need
lastSourceFailed set. It would look like this:
> if (lastSourceFail
On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote:
> On Wed, Sep 7, 2022 at 3:27 AM Nathan Bossart
> wrote:
>> It's not clear to me how this is expected to interact with the pg_wal phase
>> of standby recovery. As the docs note [0], standby servers loop through
>> archive recover
On Wed, Sep 7, 2022 at 3:27 AM Nathan Bossart wrote:
>
> +
> + wal_source_switch_interval configuration
> parameter
> +
>
> I don't want to bikeshed on the name too much, but I do think we need
> something more descriptive. I'm thinking of something like
> streaming_replication
+
+ wal_source_switch_interval configuration
parameter
+
I don't want to bikeshed on the name too much, but I do think we need
something more descriptive. I'm thinking of something like
streaming_replication_attempt_interval or
streaming_replication_retry_interval.
+Sp
On Fri, Jul 8, 2022 at 9:16 PM Bharath Rupireddy
wrote:
>
> On Sat, Jun 25, 2022 at 1:31 AM Cary Huang wrote:
> >
> > The following review has been posted through the commitfest application:
> > make installcheck-world: tested, passed
> > Implements feature: tested, passed
> > Spec complia
On Sat, Jun 25, 2022 at 1:31 AM Cary Huang wrote:
>
> The following review has been posted through the commitfest application:
> make installcheck-world: tested, passed
> Implements feature: tested, passed
> Spec compliant: not tested
> Documentation:not tested
>
> Hel
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation:not tested
Hello
I tested this patch in a setup where the standby is in the mid
On Sat, Apr 30, 2022 at 6:19 PM Bharath Rupireddy
wrote:
>
> On Mon, Nov 29, 2021 at 1:30 AM SATYANARAYANA NARLAPURAM
> wrote:
> >
> > Hi Hackers,
> >
> > When the standby couldn't connect to the primary it switches the XLog
> > source from streaming to archive and continues in that state until
On Mon, Nov 29, 2021 at 1:30 AM SATYANARAYANA NARLAPURAM
wrote:
>
> Hi Hackers,
>
> When the standby couldn't connect to the primary it switches the XLog source
> from streaming to archive and continues in that state until it can get the
> WAL from the archive location. On a server with high WAL
On Mon, Nov 29, 2021 at 1:30 AM SATYANARAYANA NARLAPURAM
wrote:
>
> Hi Hackers,
>
> When the standby couldn't connect to the primary it switches the XLog source
> from streaming to archive and continues in that state until it can get the
> WAL from the archive location. On a server with high WAL
60 matches
Mail list logo
