Hi Álvaro, Thanks for the info. I did find the thread you are referring to, my understanding was that the fix was back-ported to 17.8; we are on 17.10. I will also re-test on the latest 18 version just to be sure, but I think my situation is different -- the slot here is invalidated _by_ a checkpoint that recycles some WAL segments when multiple concurrent basebackups trigger checkpoints.
Cheers Nick On Fri, Aug 21, 2026 at 5:35 PM Álvaro Herrera <[email protected]> wrote: > Hello Nick > > On 2026-Aug-21, Nick Ivanov wrote: > > > We are encountering a possible race condition when executing several > > `pg_basebackup --wal-method=stream --slot=... --create-slot` concurrently > > while initialising streaming replicas. Our automation starts 3 > > pg_basebackup processes on 3 future replica servers within 1 second of > each > > other. One of them almost always fails with "requested WAL segment ... > has > > already been removed". > > I think this is related to this thread here: > > https://www.postgresql.org/message-id/flat/5e045179-236f-4f8f-84f1-0f2566ba784c.mengjuan.cmj%40alibaba-inc.com > > and to this commit > > Author: Amit Kapila <[email protected]> > Branch: master Release: REL_19_BR [006dd4b2e] 2025-12-08 05:21:22 +0000 > Branch: REL_18_STABLE Release: REL_18_2 [d3ceb2084] 2025-12-08 05:33:14 > +0000 > > Prevent invalidation of newly created replication slots. > > A race condition could cause a newly created replication slot to become > invalidated between WAL reservation and a checkpoint. > > Previously, if the required WAL was removed, we retried the reservation > process. However, the slot could still be invalidated before the retry > if > the WAL was not yet removed but the checkpoint advanced the redo > pointer > beyond the slot's intended restart LSN and computed the minimum LSN > that > needs to be preserved for the slots. > > The fix is to acquire an exclusive lock on > ReplicationSlotAllocationLock > during WAL reservation to serialize WAL reservation and checkpoint's > minimum restart_lsn computation. This ensures that, if WAL reservation > occurs first, the checkpoint waits until restart_lsn is updated before > removing WAL. If the checkpoint runs first, subsequent WAL reservations > pick a position at or after the latest checkpoint's redo pointer. > > We can't use the same fix for branch 17 and prior because commit > 2090edc6f3 changed to compute to the minimum restart_LSN among slot's > at > the beginning of checkpoint (or restart point). The fix for 17 and > prior > branches is under discussion and will be committed separately. > > Reported-by: suyu.cmj <[email protected]> > Author: Hou Zhijie <[email protected]> > Reviewed-by: Vitaly Davydov <[email protected]> > Reviewed-by: Masahiko Sawada <[email protected]> > Reviewed-by: Amit Kapila <[email protected]> > Backpatch-through: 18 > Discussion: > https://postgr.es/m/5e045179-236f-4f8f-84f1-0f2566ba784c.mengjuan....@alibaba-inc.com > > > and to this other commit > > > Author: Amit Kapila <[email protected]> > Branch: REL_17_STABLE Release: REL_17_8 [3510ebeb0] 2026-01-08 07:17:56 > +0000 > Branch: REL_16_STABLE Release: REL_16_12 [24cce33c3] 2026-01-08 07:07:23 > +0000 > Branch: REL_15_STABLE Release: REL_15_16 [aae05622a] 2026-01-08 06:54:52 > +0000 > Branch: REL_14_STABLE Release: REL_14_21 [7406df605] 2026-01-08 06:44:28 > +0000 > > Prevent invalidation of newly created replication slots. > > A race condition could cause a newly created replication slot to become > invalidated between WAL reservation and a checkpoint. > > Previously, if the required WAL was removed, we retried the reservation > process. However, the slot could still be invalidated before the retry > if > the WAL was not yet removed but the checkpoint advanced the redo > pointer > beyond the slot's intended restart LSN and computed the minimum LSN > that > needs to be preserved for the slots. > > The fix is to acquire an exclusive lock on > ReplicationSlotAllocationLock > during WAL reservation, and a shared lock during the minimum LSN > calculation at checkpoints to serialize the process. This ensures > that, if > WAL reservation occurs first, the checkpoint waits until restart_lsn is > updated before calculating the minimum LSN. If the checkpoint runs > first, > subsequent WAL reservations pick a position at or after the latest > checkpoint's redo pointer. > > We used a similar fix in HEAD (via commit 006dd4b2e5) and 18. The > difference is that in 17 and prior branches we need to additionally > handle > the race condition with slot's minimum LSN computation during > checkpoints. > > Reported-by: suyu.cmj <[email protected]> > Author: Hou Zhijie <[email protected]> > Author: vignesh C <[email protected]> > Reviewed-by: Hayato Kuroda <[email protected]> > Reviewed-by: Masahiko Sawada <[email protected]> > Reviewed-by: Amit Kapila <[email protected]> > Backpatch-through: 14 > Discussion: > https://postgr.es/m/5e045179-236f-4f8f-84f1-0f2566ba784c.mengjuan....@alibaba-inc.com > > > What version are you using? > > If you're using a version that contains these fixes, then we may have > some slightly different bug ... > > -- > Álvaro Herrera Breisgau, Deutschland — > https://www.EnterpriseDB.com/ > "No hay ausente sin culpa ni presente sin disculpa" (Prov. francés) > -- Nick Ivanov Solutions Architect www.enterprisedb.com
