Re: Replication slot WAL reservation

2025-04-04 Thread Christophe Pettus
> On Mar 25, 2025, at 13:58, Phillip Diffley wrote: > > Oh I see! I was conflating the data I see coming out of a replication slot > with the internal organization of the WAL. I think the more specific question > I am trying to answer is, as a consumer of a replication slot, how do I > reas

Re: Replication slot WAL reservation

2025-03-26 Thread Phillip Diffley
> You're looking for Standby Status Update Awesome! I completely missed that. > Transactions are always presented to the output plugin in commit order Ok great. I think that is what I needed to know. Just to confirm, it sounds like the order messages are sent from the output plugin is what matt

Re: Replication slot WAL reservation

2025-03-26 Thread Christophe Pettus
> On Mar 26, 2025, at 07:55, Phillip Diffley wrote: > Just to confirm, it sounds like the order messages are sent from the output > plugin is what matters here. When you update confirmed_flush_lsn to LSN "A", > any messages that were sent by the output plugin after the message with LSN > "A"

Re: Replication slot WAL reservation

2025-03-25 Thread Christophe Pettus
> On Mar 25, 2025, at 20:56, Phillip Diffley wrote: > > Is there a message type that is used to confirm what logs have been > successfully consumed? You're looking for Standby Status Update: https://www.postgresql.org/docs/current/protocol-replication.html#PROTOCOL-REPLICATION-STAN

Re: Replication slot WAL reservation

2025-03-25 Thread Phillip Diffley
> You shouldn't need to manually advance the replication slot. > The client is also expected to send back regular messages letting the publisher / primary know that it has successfully consumed up to a particular point I was thinking of these as the same thing, but it sounds like they are differen

Re: Replication slot WAL reservation

2025-03-25 Thread Phillip Diffley
Oh I see! I was conflating the data I see coming out of a replication slot with the internal organization of the WAL. I think the more specific question I am trying to answer is, as a consumer of a replication slot, how do I reason about what replication records will be made unavailable when I conf

Re: Replication slot WAL reservation

2025-03-25 Thread Christophe Pettus
Missed this question! > On Mar 25, 2025, at 09:56, Phillip Diffley wrote: > But when processing data from a replication slot, we confirm rows that have > been processed and can be deleted from the WAL based on the LSN (eg. with > pg_replication_slot_advance). How does postgres identify what par

Re: Replication slot WAL reservation

2025-03-25 Thread Christophe Pettus
> On Mar 25, 2025, at 09:56, Phillip Diffley wrote: > 1. Every DML operation (insert, update, delete, truncate) will have a row in > the WAL and that row will have an LSN assigned to it. > 2. The LSNs are assigned when the operation happens. > 3. Operations within a transaction are written to