On Thu, Jan 02, 2025 at 11:12:37PM +0500, Andrey M. Borodin wrote:
> In my observation restore from archive is many orders of magnitude
> faster than streaming replication. Advanced archive tools employ
> compression (x6 to speed), download parallelism (x4), are not
> constrained be primary's netwo
> On 23 Mar 2024, at 14:22, Bharath Rupireddy
> wrote:
>
> IMHO, it makes sense to have something like replay_source_order if
> there's any use case that arises in future requiring the standby to
> intentionally switch to pg_wal or archive. But not as part of this
> feature.
IMO, it's vital
Hi,
On Thu, Aug 29, 2024 at 6:32 AM Bharath Rupireddy
wrote:
> In synchronous replication setup, until standby finishes fetching WAL
> from the archive, the commits on the primary have to wait which can
> increase the query latency. If the standby can connect to the primary
> as soon as the broke
Hi,
Thanks for looking into this.
On Fri, Aug 23, 2024 at 5:03 AM John H wrote:
>
> For a motivation aspect I can see this being useful
> synchronous_replicas if you have commit set to flush mode.
In synchronous replication setup, until standby finishes fetching WAL
from the archive, the commit
Hi,
I took a brief look at the patch.
For a motivation aspect I can see this being useful
synchronous_replicas if you have commit set to flush mode.
So +1 on feature, easier configurability, although thinking about it
more you could probably have the restore script be smarter and provide
non-zero
On Mon, Mar 18, 2024 at 11:38 AM Michael Paquier wrote:
>
> On Sun, Mar 17, 2024 at 11:37:58AM +0530, Bharath Rupireddy wrote:
> > Rebase needed after 071e3ad59d6fd2d6d1277b2bd9579397d10ded28 due to a
> > conflict in meson.build. Please see the attached v23 patch.
>
> I've been reading this patch,
On Sun, Mar 17, 2024 at 11:37:58AM +0530, Bharath Rupireddy wrote:
> Rebase needed after 071e3ad59d6fd2d6d1277b2bd9579397d10ded28 due to a
> conflict in meson.build. Please see the attached v23 patch.
I've been reading this patch, and this is a very tricky one. Please
be *very* cautious.
+#strea
On Wed, Mar 6, 2024 at 9:49 PM Nathan Bossart wrote:
>
> > I played with that idea and it came out very nice. Please see the
> > attached v22 patch. Note that personally I didn't like "FORCE" being
> > there in the names, so I've simplified them a bit.
>
> Thanks. I'd like to spend some time test
On Wed, Mar 06, 2024 at 10:02:43AM +0530, Bharath Rupireddy wrote:
> On Wed, Mar 6, 2024 at 1:22 AM Nathan Bossart
> wrote:
>> I was thinking of something more like
>>
>> typedef enum
>> {
>> NO_FORCE_SWITCH_TO_STREAMING, /* no switch
>> necessary */
>>
On Wed, Mar 6, 2024 at 1:22 AM Nathan Bossart wrote:
>
> I was thinking of something more like
>
> typedef enum
> {
> NO_FORCE_SWITCH_TO_STREAMING, /* no switch
> necessary */
> FORCE_SWITCH_TO_STREAMING_PENDING, /* exhausting pg_wal
On Tue, Mar 05, 2024 at 11:38:37PM +0530, Bharath Rupireddy wrote:
> On Tue, Mar 5, 2024 at 7:34 AM Nathan Bossart
> wrote:
>> Is there any way to simplify this? For
>> example, would it be possible to make an enum that tracks the
>> streaming_replication_retry_interval state?
>
> I guess the w
On Tue, Mar 5, 2024 at 7:34 AM Nathan Bossart wrote:
>
> cfbot claims that this one needs another rebase.
Yeah, the conflict was with the new TAP test file name in
src/test/recovery/meson.build.
> I've spent some time thinking about this one. I'll admit I'm a bit worried
> about adding more com
cfbot claims that this one needs another rebase.
I've spent some time thinking about this one. I'll admit I'm a bit worried
about adding more complexity to this state machine, but I also haven't
thought of any other viable approaches, and this still seems like a useful
feature. So, for now, I th
On Tue, Feb 20, 2024 at 11:54 AM Japin Li wrote:
>
>
> On Tue, 20 Feb 2024 at 13:40, Bharath Rupireddy
> wrote:
> > On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote:
> >> [2]
> >> +# Ensure checkpoint doesn't come in our way
> >> +$primary->append_conf('postgresql.conf', qq(
> >> +min_wal_siz
On Tue, 20 Feb 2024 at 13:40, Bharath Rupireddy
wrote:
> On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote:
>> [2]
>> +# Ensure checkpoint doesn't come in our way
>> +$primary->append_conf('postgresql.conf', qq(
>> +min_wal_size = 2MB
>> +max_wal_size = 1GB
>> +checkpoint_timeout = 1h
On Mon, Feb 19, 2024 at 8:25 PM Japin Li wrote:
>
> > Strengthened tests a bit by using recovery_min_apply_delay to mimic
> > standby spending some time fetching from archive. PSA v18 patch.
>
> Here are some minor comments:
Thanks for taking a look at it.
> [1]
> +primary). However, the
On Mon, 19 Feb 2024 at 18:36, Bharath Rupireddy
wrote:
> On Wed, Jan 31, 2024 at 6:30 PM Bharath Rupireddy
> wrote:
>>
>> Needed a rebase due to commit 776621a (conflict in
>> src/test/recovery/meson.build for new TAP test file added). Please
>> find the attached v17 patch.
>
> Strengthened te
On Wed, Jan 31, 2024 at 6:30 PM Bharath Rupireddy
wrote:
>
> Needed a rebase due to commit 776621a (conflict in
> src/test/recovery/meson.build for new TAP test file added). Please
> find the attached v17 patch.
Strengthened tests a bit by using recovery_min_apply_delay to mimic
standby spending
On Wed, Jan 3, 2024 at 4:58 PM Bharath Rupireddy
wrote:
>
> On Thu, Dec 28, 2023 at 5:26 PM Bharath Rupireddy
> wrote:
> >
> > I took a closer look at v14 and came up with the following changes:
> >
> > 1. Used advance_wal introduced by commit c161ab74f7.
> > 2. Simplified the core logic and new
On Thu, Dec 28, 2023 at 5:26 PM Bharath Rupireddy
wrote:
>
> I took a closer look at v14 and came up with the following changes:
>
> 1. Used advance_wal introduced by commit c161ab74f7.
> 2. Simplified the core logic and new TAP tests.
> 3. Reworded the comments and docs.
> 4. Simplified new DEBUG
On Sat, Oct 21, 2023 at 11:59 PM Bharath Rupireddy
wrote:
>
> On Fri, Jul 21, 2023 at 12:38 PM Bharath Rupireddy
> wrote:
> >
> > Needed a rebase. I'm attaching the v13 patch for further consideration.
>
> Needed a rebase. I'm attaching the v14 patch. It also has the following
> changes:
>
> - R
On Fri, Jul 21, 2023 at 12:38 PM Bharath Rupireddy
wrote:
>
> Needed a rebase. I'm attaching the v13 patch for further consideration.
Needed a rebase. I'm attaching the v14 patch. It also has the following changes:
- Ran pgindent on the new source code.
- Ran pgperltidy on the new TAP test.
- Im
On Tue, Apr 25, 2023 at 9:27 PM Bharath Rupireddy
wrote:
>
> > Needed a rebase. I'm attaching the v11 patch for further review.
>
> Needed a rebase, so attaching the v12 patch. I word-smithed comments
> and docs a bit.
Needed a rebase. I'm attaching the v13 patch for further consideration.
--
B
On Fri, Feb 24, 2023 at 10:26 AM Bharath Rupireddy
wrote:
>
> On Wed, Nov 16, 2022 at 11:39 AM Bharath Rupireddy
> wrote:
> >
> > I'm attaching the v10 patch for further review.
>
> Needed a rebase. I'm attaching the v11 patch for further review.
Needed a rebase, so attaching the v12 patch. I wo
On Wed, Nov 16, 2022 at 11:39 AM Bharath Rupireddy
wrote:
>
> I'm attaching the v10 patch for further review.
Needed a rebase. I'm attaching the v11 patch for further review.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
F
On Thu, Jan 19, 2023 at 6:20 AM Nathan Bossart wrote:
>
> On Tue, Jan 17, 2023 at 07:44:52PM +0530, Bharath Rupireddy wrote:
> > On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart
> > wrote:
> >> With your patch, we might replay one of these "old" files in pg_wal instead
> >> of the complete version
On Tue, Jan 17, 2023 at 07:44:52PM +0530, Bharath Rupireddy wrote:
> On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart
> wrote:
>> With your patch, we might replay one of these "old" files in pg_wal instead
>> of the complete version of the file from the archives,
>
> That's true even today, withou
On Thu, Jan 12, 2023 at 6:21 AM Nathan Bossart wrote:
>
> On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote:
> > In summary, the standby state machine in WaitForWALToBecomeAvailable()
> > exhausts all the WAL in pg_wal before switching to streaming after
> > failing to fetch from a
On Tue, Oct 18, 2022 at 07:31:33AM +0530, Bharath Rupireddy wrote:
> In summary, the standby state machine in WaitForWALToBecomeAvailable()
> exhausts all the WAL in pg_wal before switching to streaming after
> failing to fetch from archive. The v8 patch proposed upthread deviates
> from this behav
On Wed, Nov 16, 2022 at 9:38 AM Ian Lawrence Barwick wrote:
>
> While reviewing the patch backlog, we have determined that this patch adds
> one or more TAP tests but has not added the test to the "meson.build" file.
Thanks for pointing it out. Yeah, the test wasn't picking up on meson
builds. I
2022年10月18日(火) 11:02 Bharath Rupireddy :
>
> On Tue, Oct 11, 2022 at 8:40 AM Nathan Bossart
> wrote:
> >
> > On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote:
> > > On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart
> > > wrote:
> > >> I wonder if it would be better to simply remov
On Tue, Oct 11, 2022 at 8:40 AM Nathan Bossart wrote:
>
> On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote:
> > On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart
> > wrote:
> >> I wonder if it would be better to simply remove this extra polling of
> >> pg_wal as a prerequisite to y
On Mon, Oct 10, 2022 at 11:33:57AM +0530, Bharath Rupireddy wrote:
> On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart
> wrote:
>> I wonder if it would be better to simply remove this extra polling of
>> pg_wal as a prerequisite to your patch. The existing commentary leads me
>> to think there migh
On Mon, Oct 10, 2022 at 3:17 AM Nathan Bossart wrote:
>
> > Instead, the simplest would be to just pass XLOG_FROM_WAL to
> > XLogFileReadAnyTLI() when we're about to switch the source to stream
> > mode. This doesn't change the existing behaviour.
>
> It might be more consistent with existing beha
On Sun, Oct 09, 2022 at 02:39:47PM +0530, Bharath Rupireddy wrote:
> We can give it a chance to restore from pg_wal before switching to
> streaming to not change any behaviour of the state machine. But, not
> definitely by setting currentSource to XLOG_FROM_WAL, we basically
> never explicitly set
On Sun, Oct 9, 2022 at 3:22 AM Nathan Bossart wrote:
>
> As I mentioned upthread [0], I'm still a little concerned that this patch
> will cause the state machine to go straight from archive recovery to
> streaming replication, skipping recovery from pg_wal.
>
> [0] https://postgr.es/m/202209062157
On Mon, Sep 19, 2022 at 07:49:21PM +0530, Bharath Rupireddy wrote:
> SwitchFromArchiveToStreamEnabled() seemed better at this point. I'm
> attaching the v7 patch with that change. Please review it further.
As I mentioned upthread [0], I'm still a little concerned that this patch
will cause the sta
On Fri, Sep 16, 2022 at 4:58 PM Bharath Rupireddy
wrote:
>
> On Fri, Sep 16, 2022 at 12:06 PM Kyotaro Horiguchi
> wrote:
> >
> > In other words, it seems to me that the macro name doesn't manifest
> > the condition correctly.
> >
> > I don't think we don't particularly want to do that uncondition
On Fri, Sep 16, 2022 at 12:06 PM Kyotaro Horiguchi
wrote:
>
> In other words, it seems to me that the macro name doesn't manifest
> the condition correctly.
>
> I don't think we don't particularly want to do that unconditionally.
> I wanted just to get rid of the macro from the usage site. Even i
At Fri, 16 Sep 2022 09:15:58 +0530, Bharath Rupireddy
wrote in
> On Thu, Sep 15, 2022 at 1:52 PM Kyotaro Horiguchi
> wrote:
> >
> > At Thu, 15 Sep 2022 10:28:12 +0530, Bharath Rupireddy
> > wrote in
> > > I'm attaching the v6 patch that's rebased on to the latest HEAD.
> > > Please consider t
On Thu, Sep 15, 2022 at 1:52 PM Kyotaro Horiguchi
wrote:
>
> At Thu, 15 Sep 2022 10:28:12 +0530, Bharath Rupireddy
> wrote in
> > I'm attaching the v6 patch that's rebased on to the latest HEAD.
> > Please consider this for review.
>
> Thaks for the new version!
>
> +#define StreamingReplRetryEn
At Thu, 15 Sep 2022 10:28:12 +0530, Bharath Rupireddy
wrote in
> I'm attaching the v6 patch that's rebased on to the latest HEAD.
> Please consider this for review.
Thaks for the new version!
+#define StreamingReplRetryEnabled() \
+ (streaming_replication_retry_interval > 0 && \
+
On Mon, Sep 12, 2022 at 11:56 AM Bharath Rupireddy
wrote:
>
> Please review the attached v5 patch.
I'm attaching the v6 patch that's rebased on to the latest HEAD.
Please consider this for review.
--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: ht
On Mon, Sep 12, 2022 at 9:03 AM Bharath Rupireddy
wrote:
>
> Please review the attached v4 patch addressing above review comments.
Oops, there's a compiler warning [1] with the v4 patch, fixed it.
Please review the attached v5 patch.
[1] https://cirrus-ci.com/task/5730076611313664?logs=gcc_warni
On Sat, Sep 10, 2022 at 3:35 AM Nathan Bossart wrote:
>
> Well, for wal_keep_size, using bytes makes sense. Given you know how much
> disk space you have, you can set this parameter accordingly to avoid
> retaining too much of it for standby servers. For your proposed parameter,
> it's not so si
On Fri, Sep 09, 2022 at 11:07:00PM +0530, Bharath Rupireddy wrote:
> On Fri, Sep 9, 2022 at 10:29 PM Nathan Bossart
> wrote:
>> IMO the timeout approach would be more intuitive for users. When it comes
>> to archive recovery, "WAL segment" isn't a standard unit of measure. WAL
>> segment size c
On Fri, Sep 9, 2022 at 10:29 PM Nathan Bossart wrote:
>
> On Fri, Sep 09, 2022 at 12:14:25PM +0530, Bharath Rupireddy wrote:
> > On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi
> > wrote:
> >> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
> >> wrote in
> >> > My general point is that we s
On Fri, Sep 09, 2022 at 12:14:25PM +0530, Bharath Rupireddy wrote:
> On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi
> wrote:
>> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
>> wrote in
>> > My general point is that we should probably offer some basic preventative
>> > measure against fli
On Fri, Sep 9, 2022 at 10:57 AM Kyotaro Horiguchi
wrote:
>
> At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
> wrote in
> > On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote:
> > > I'm attaching the v3 patch with the review comments addressed, please
> > > review it further.
>
At Thu, 8 Sep 2022 10:53:56 -0700, Nathan Bossart
wrote in
> On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote:
> > I'm attaching the v3 patch with the review comments addressed, please
> > review it further.
>
> My general point is that we should probably offer some basic preve
Being late for the party.
It seems to me that the function is getting too long. I think we
might want to move the core part of the patch into another function.
I think it might be better if intentionalSourceSwitch doesn't need
lastSourceFailed set. It would look like this:
> if (lastSourceFail
On Thu, Sep 08, 2022 at 05:16:53PM +0530, Bharath Rupireddy wrote:
> On Wed, Sep 7, 2022 at 3:27 AM Nathan Bossart
> wrote:
>> It's not clear to me how this is expected to interact with the pg_wal phase
>> of standby recovery. As the docs note [0], standby servers loop through
>> archive recover
On Wed, Sep 7, 2022 at 3:27 AM Nathan Bossart wrote:
>
> +
> + wal_source_switch_interval configuration
> parameter
> +
>
> I don't want to bikeshed on the name too much, but I do think we need
> something more descriptive. I'm thinking of something like
> streaming_replication
+
+ wal_source_switch_interval configuration
parameter
+
I don't want to bikeshed on the name too much, but I do think we need
something more descriptive. I'm thinking of something like
streaming_replication_attempt_interval or
streaming_replication_retry_interval.
+Sp
On Fri, Jul 8, 2022 at 9:16 PM Bharath Rupireddy
wrote:
>
> On Sat, Jun 25, 2022 at 1:31 AM Cary Huang wrote:
> >
> > The following review has been posted through the commitfest application:
> > make installcheck-world: tested, passed
> > Implements feature: tested, passed
> > Spec complia
On Sat, Jun 25, 2022 at 1:31 AM Cary Huang wrote:
>
> The following review has been posted through the commitfest application:
> make installcheck-world: tested, passed
> Implements feature: tested, passed
> Spec compliant: not tested
> Documentation:not tested
>
> Hel
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation:not tested
Hello
I tested this patch in a setup where the standby is in the mid
On Sat, Apr 30, 2022 at 6:19 PM Bharath Rupireddy
wrote:
>
> On Mon, Nov 29, 2021 at 1:30 AM SATYANARAYANA NARLAPURAM
> wrote:
> >
> > Hi Hackers,
> >
> > When the standby couldn't connect to the primary it switches the XLog
> > source from streaming to archive and continues in that state until
On Mon, Nov 29, 2021 at 1:30 AM SATYANARAYANA NARLAPURAM
wrote:
>
> Hi Hackers,
>
> When the standby couldn't connect to the primary it switches the XLog source
> from streaming to archive and continues in that state until it can get the
> WAL from the archive location. On a server with high WAL
On Mon, Nov 29, 2021 at 1:30 AM SATYANARAYANA NARLAPURAM
wrote:
>
> Hi Hackers,
>
> When the standby couldn't connect to the primary it switches the XLog source
> from streaming to archive and continues in that state until it can get the
> WAL from the archive location. On a server with high WAL
Hi Hackers,
When the standby couldn't connect to the primary it switches the XLog
source from streaming to archive and continues in that state until it can
get the WAL from the archive location. On a server with high WAL activity,
typically getting the WAL from the archive is slower than streaming
61 matches
Mail list logo
