Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-01 Thread Masahiko Sawada
On Mon, Jul 28, 2025 at 10:02 PM shveta malik wrote: > > On Fri, Jul 25, 2025 at 11:45 AM Masahiko Sawada > wrote: > > > > Thank you for testing the patch! > > > > I've reworked the locking part in the patch. The attached v4 patch > > should address all review comments including your previous >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-01 Thread Masahiko Sawada
On Mon, Jul 28, 2025 at 9:44 PM vignesh C wrote: > > On Fri, 25 Jul 2025 at 11:45, Masahiko Sawada wrote: > > > > On Tue, Jul 22, 2025 at 11:44 PM shveta malik > > wrote: > > > > > > On Tue, Jul 22, 2025 at 5:03 AM Masahiko Sawada > > > wrote: > > > > > > > > Yes, I agree. The main patch focu

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-08-01 Thread Masahiko Sawada
On Thu, Jul 31, 2025 at 5:00 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > > I thought we could fix this issue by checking the number of in-use > > logical slots while holding ReplicationSlotControlLock and > > LogicalDecodingControlLock, but it seems we need to deal with another > >

RE: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-31 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, > I thought we could fix this issue by checking the number of in-use > logical slots while holding ReplicationSlotControlLock and > LogicalDecodingControlLock, but it seems we need to deal with another > race condition too between backends and startup processes at the end > of rec

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-30 Thread Masahiko Sawada
On Wed, Jul 30, 2025 at 12:22 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > While reading more, I found a race condition. Thank you for reviewing the patch! > In this case the effective_wal_level > can be logical even when there is no logical slot. > UpdateLogicalDecodingStatusEndOf

RE: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-30 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, While reading more, I found a race condition. In this case the effective_wal_level can be logical even when there is no logical slot. UpdateLogicalDecodingStatusEndOfRecovery() checks the number of slots of the logical slot then release the lock once. Then startup process acquir

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-28 Thread shveta malik
On Fri, Jul 25, 2025 at 11:45 AM Masahiko Sawada wrote: > > Thank you for testing the patch! > > I've reworked the locking part in the patch. The attached v4 patch > should address all review comments including your previous > comments[1]. > Thank You for the patch. I have not reviewed fully, but

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-28 Thread vignesh C
On Fri, 25 Jul 2025 at 11:45, Masahiko Sawada wrote: > > On Tue, Jul 22, 2025 at 11:44 PM shveta malik wrote: > > > > On Tue, Jul 22, 2025 at 5:03 AM Masahiko Sawada > > wrote: > > > > > > Yes, I agree. The main patch focuses on the part where we > > > automatically change the effective WAL lev

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-28 Thread Masahiko Sawada
On Mon, Jul 28, 2025 at 5:34 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > > Thank you for testing the patch! > > > > I've reworked the locking part in the patch. The attached v4 patch > > should address all review comments including your previous > > comments[1]. > > Thanks for makin

RE: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-28 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, > Thank you for testing the patch! > > I've reworked the locking part in the patch. The attached v4 patch > should address all review comments including your previous > comments[1]. Thanks for making the patch! I resumed to spending time for the project. Here are my comments. 1

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-24 Thread Masahiko Sawada
On Tue, Jul 22, 2025 at 11:44 PM shveta malik wrote: > > On Tue, Jul 22, 2025 at 5:03 AM Masahiko Sawada wrote: > > > > Yes, I agree. The main patch focuses on the part where we > > automatically change the effective WAL level upon the logical slot > > creation and deletion (and potentially remov

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-22 Thread shveta malik
On Tue, Jul 22, 2025 at 5:03 AM Masahiko Sawada wrote: > > Yes, I agree. The main patch focuses on the part where we > automatically change the effective WAL level upon the logical slot > creation and deletion (and potentially remove 'logical' from > wal_level), and other things are implemented as

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-21 Thread Masahiko Sawada
On Sun, Jul 20, 2025 at 11:53 PM Amit Kapila wrote: > > On Mon, Jul 21, 2025 at 11:24 AM shveta malik wrote: > > > > On Mon, Jul 21, 2025 at 10:48 AM shveta malik > > wrote: > > > > > > I'm continuing to think it through and will share any further thoughts > > > if something comes to mind. > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-21 Thread shveta malik
On Mon, Jul 21, 2025 at 12:23 PM Amit Kapila wrote: > > I am slightly hesitant to introduce multiple ways to enable logical > decoding/replication unless that is the only path as giving multiple > options to achieve the same thing can confuse users as to which one is > preferable and pros/cons of

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 11:24 AM shveta malik wrote: > > On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > > > I'm continuing to think it through and will share any further thoughts > > if something comes to mind. > > > > How about a parameter named 'on_last_logical_slot' with possible > v

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > > > > > One concern I have is regarding the default setting of > > > 'force_slot_drop' . I assume the default value of this new DROP-SUB > > > argument will be 'false' to prevent cu

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila > > > wrote: > > > > > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta ma

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik > > > wrote: > > > > > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand D

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-18 Thread Amit Kapila
On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > > wrote: > > > > > > > > Hi, > > > > > > > > On Tue, Jun 10, 20

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-18 Thread shveta malik
On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Tue, Jun 10, 2025 at 02:00:55PM -0700, Masahiko Sawada wrote: > > > > > > > > > 0001 patch

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Amit Kapila
On Fri, Jul 18, 2025 at 12:26 AM Masahiko Sawada wrote: > > On Thu, Jul 17, 2025 at 2:36 AM Amit Kapila wrote: > > > > What if we make DROP SUBSCRIPTION fail if it can lead to removal of > > the last slot on publisher and allow DROP to succeed when the > > subscription's drop_slot_force (a new su

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Masahiko Sawada
On Thu, Jul 17, 2025 at 2:36 AM Amit Kapila wrote: > > What if we make DROP SUBSCRIPTION fail if it can lead to removal of > the last slot on publisher and allow DROP to succeed when the > subscription's drop_slot_force (a new subscription option) is set? > Now, users can still be allowed to Drop

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Masahiko Sawada
On Tue, Jul 15, 2025 at 10:55 PM shveta malik wrote: > > On Tue, Jul 15, 2025 at 10:37 PM Masahiko Sawada > wrote: > > > > I've attached updated patches that implement the idea we've discussed. > > The patches still need to be polished but the implemented ideas seem > > good. Feedback is very we

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Amit Kapila
On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Tue, Jun 10, 2025 at 02:00:55PM -0700, Masahiko Sawada wrote: > > > > > > > > > 0001 patch

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-17 Thread Amit Kapila
On Wed, Jun 18, 2025 at 3:23 PM shveta malik wrote: > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Tue, Jun 10, 2025 at 02:00:55PM -0700, Masahiko Sawada wrote: > > > > > > > > 0001 patch allows us to create a logical slot without WAL > > > > > > > > reserva

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-15 Thread shveta malik
On Tue, Jul 15, 2025 at 10:37 PM Masahiko Sawada wrote: > > I've attached updated patches that implement the idea we've discussed. > The patches still need to be polished but the implemented ideas seem > good. Feedback is very welcome. > Thank You for the patches. I just tried my hands on ptach00

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-15 Thread Masahiko Sawada
On Tue, Jul 8, 2025 at 7:41 PM shveta malik wrote: > > On Sun, Jul 6, 2025 at 8:33 PM Masahiko Sawada wrote: > > > > On Thu, Jul 3, 2025 at 3:32 PM shveta malik wrote: > > > > > > On Wed, Jul 2, 2025 at 9:46 PM Masahiko Sawada > > > wrote: > > > > > > > > On Wed, Jun 18, 2025 at 1:07 PM shveta

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-08 Thread shveta malik
On Sun, Jul 6, 2025 at 8:33 PM Masahiko Sawada wrote: > > On Thu, Jul 3, 2025 at 3:32 PM shveta malik wrote: > > > > On Wed, Jul 2, 2025 at 9:46 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Jun 18, 2025 at 1:07 PM shveta malik > > > wrote: > > > > > > > > On Wed, Jun 18, 2025 at 6:06 AM

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-06 Thread Masahiko Sawada
On Thu, Jul 3, 2025 at 3:32 PM shveta malik wrote: > > On Wed, Jul 2, 2025 at 9:46 PM Masahiko Sawada wrote: > > > > On Wed, Jun 18, 2025 at 1:07 PM shveta malik wrote: > > > > > > On Wed, Jun 18, 2025 at 6:06 AM Masahiko Sawada > > > wrote: > > > > > > > > Thank you for the comments! > > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-02 Thread shveta malik
On Wed, Jul 2, 2025 at 9:46 PM Masahiko Sawada wrote: > > On Wed, Jun 18, 2025 at 1:07 PM shveta malik wrote: > > > > On Wed, Jun 18, 2025 at 6:06 AM Masahiko Sawada > > wrote: > > > > > > Thank you for the comments! > > > > > > > > > > > 2) > > > > I see that when primary switches back its eff

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-02 Thread Masahiko Sawada
On Wed, Jun 18, 2025 at 1:07 PM shveta malik wrote: > > On Wed, Jun 18, 2025 at 6:06 AM Masahiko Sawada wrote: > > > > Thank you for the comments! > > > > > > > > 2) > > > I see that when primary switches back its effective wal_level to > > > replica while standby has wal_level=logical in conf fi

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-30 Thread Bertrand Drouvot
Hi, On Mon, Jun 30, 2025 at 12:21:44PM +0530, shveta malik wrote: > On Mon, Jun 30, 2025 at 11:16 AM Masahiko Sawada > wrote: > > > > I'm not sure if there are users who want to disable the new behavior > > and use only the current behavior. I think we can focus on the new API > > and automatic

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-29 Thread shveta malik
On Mon, Jun 30, 2025 at 11:16 AM Masahiko Sawada wrote: > > On Thu, Jun 26, 2025 at 5:50 PM shveta malik wrote: > > > > On Wed, Jun 25, 2025 at 12:20 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Wed, Jun 25, 2025 at 09:15:04AM +0530, shveta malik wrote: > > > > On Wed, Jun 25, 2

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-29 Thread Masahiko Sawada
On Thu, Jun 26, 2025 at 5:50 PM shveta malik wrote: > > On Wed, Jun 25, 2025 at 12:20 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Wed, Jun 25, 2025 at 09:15:04AM +0530, shveta malik wrote: > > > On Wed, Jun 25, 2025 at 9:12 AM shveta malik > > > wrote: > > > > > > > > On Tue, Jun 24, 20

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-26 Thread shveta malik
On Wed, Jun 25, 2025 at 12:20 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Jun 25, 2025 at 09:15:04AM +0530, shveta malik wrote: > > On Wed, Jun 25, 2025 at 9:12 AM shveta malik wrote: > > > > > > On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot > > > wrote: > > > > > > > > Yeah, I think that

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-25 Thread Bertrand Drouvot
Hi, On Wed, Jun 25, 2025 at 09:15:04AM +0530, shveta malik wrote: > On Wed, Jun 25, 2025 at 9:12 AM shveta malik wrote: > > > > On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot > > wrote: > > > > > > Yeah, I think that sounds reasonable and that would avoid users to use > > > the slot created wi

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-24 Thread shveta malik
On Wed, Jun 25, 2025 at 9:12 AM shveta malik wrote: > > On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Tue, Jun 24, 2025 at 12:13:32AM +0900, Masahiko Sawada wrote: > > > On Mon, Jun 23, 2025 at 7:01 PM Bertrand Drouvot > > > wrote: > > > > > > > > Hi, > > > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-24 Thread shveta malik
On Tue, Jun 24, 2025 at 2:12 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Jun 24, 2025 at 12:13:32AM +0900, Masahiko Sawada wrote: > > On Mon, Jun 23, 2025 at 7:01 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Mon, Jun 23, 2025 at 05:10:37PM +0900, Masahiko Sawada wrote: > > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-24 Thread Bertrand Drouvot
Hi, On Tue, Jun 24, 2025 at 12:13:32AM +0900, Masahiko Sawada wrote: > On Mon, Jun 23, 2025 at 7:01 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Mon, Jun 23, 2025 at 05:10:37PM +0900, Masahiko Sawada wrote: > > > On Thu, Jun 19, 2025 at 6:00 PM Bertrand Drouvot > > > wrote: > > > > > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread Masahiko Sawada
On Mon, Jun 23, 2025 at 7:01 PM Bertrand Drouvot wrote: > > Hi, > > On Mon, Jun 23, 2025 at 05:10:37PM +0900, Masahiko Sawada wrote: > > On Thu, Jun 19, 2025 at 6:00 PM Bertrand Drouvot > > wrote: > > > > > > - pg_activate_logical_decoding() is needed only if there is not already a > > > logical

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread Masahiko Sawada
On Thu, Jun 19, 2025 at 6:00 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Jun 18, 2025 at 03:22:59PM +0530, shveta malik wrote: > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > > wrote: > > > > > > IIUC the idea is to "just" increase WAL level to 'logical' so that one > > > could then >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread shveta malik
On Mon, Jun 23, 2025 at 1:41 PM Masahiko Sawada wrote: > > Given the discussion so far, it seems we might want to have a > safeguard to prevent the effective_wal_level from being dropped to > 'replica' if the last logical slot is accidentally dropped. Yes, needed for cases where standby or cascad

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread Bertrand Drouvot
Hi, On Mon, Jun 23, 2025 at 05:10:37PM +0900, Masahiko Sawada wrote: > On Thu, Jun 19, 2025 at 6:00 PM Bertrand Drouvot > wrote: > > > > - pg_activate_logical_decoding() is needed only if there is not already a > > logical > > slot on the primary > > - the drop requires the user to think twice i

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-23 Thread shveta malik
On Fri, Jun 20, 2025 at 12:15 PM Bertrand Drouvot wrote: > > Hi, > > On Fri, Jun 20, 2025 at 09:48:47AM +0530, shveta malik wrote: > > On Thu, Jun 19, 2025 at 2:30 PM Bertrand Drouvot > > wrote: > > > > > > I wonder if a way to address the concerns that we shared above is to use a > > > mixed app

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-19 Thread Bertrand Drouvot
Hi, On Fri, Jun 20, 2025 at 09:48:47AM +0530, shveta malik wrote: > On Thu, Jun 19, 2025 at 2:30 PM Bertrand Drouvot > wrote: > > > > I wonder if a way to address the concerns that we shared above is to use a > > mixed approach like: > > > > - Forget the immediately_reserve idea > > - If a user c

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-19 Thread shveta malik
On Thu, Jun 19, 2025 at 2:30 PM Bertrand Drouvot wrote: > > I wonder if a way to address the concerns that we shared above is to use a > mixed approach like: > > - Forget the immediately_reserve idea > - If a user creates a logical slot then we automatically switch to wal_level = > logical (if not

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-19 Thread Bertrand Drouvot
Hi, On Wed, Jun 18, 2025 at 03:22:59PM +0530, shveta malik wrote: > On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot > wrote: > > > > IIUC the idea is to "just" increase WAL level to 'logical' so that one > > could then > > be allowed to make use of logical decoding from the standby. The primary

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-18 Thread shveta malik
On Wed, Jun 18, 2025 at 2:39 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Jun 10, 2025 at 02:00:55PM -0700, Masahiko Sawada wrote: > > > > > > > 0001 patch allows us to create a logical slot without WAL > > > > > > > reservation. > > Thanks for the patch and sorry to be late in this conversatio

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-18 Thread Bertrand Drouvot
Hi, On Tue, Jun 10, 2025 at 02:00:55PM -0700, Masahiko Sawada wrote: > > > > > > 0001 patch allows us to create a logical slot without WAL > > > > > > reservation. Thanks for the patch and sorry to be late in this conversation. The thing that worry me a bit with this is that that could be easy

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-17 Thread shveta malik
On Wed, Jun 18, 2025 at 6:06 AM Masahiko Sawada wrote: > > Thank you for the comments! > > > > > 2) > > I see that when primary switches back its effective wal_level to > > replica while standby has wal_level=logical in conf file, then standby > > has this status: > > > > postgres=# show wal_level

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-17 Thread Masahiko Sawada
On Mon, Jun 16, 2025 at 11:48 PM shveta malik wrote: > > On Wed, Jun 11, 2025 at 2:31 AM Masahiko Sawada wrote: > > > > I think it's the user's responsibility to keep at least one logical > > slot. It seems that setting wal_level to 'logical' would be the most > > reliable solution for this case.

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-16 Thread shveta malik
On Wed, Jun 11, 2025 at 2:31 AM Masahiko Sawada wrote: > > I think it's the user's responsibility to keep at least one logical > slot. It seems that setting wal_level to 'logical' would be the most > reliable solution for this case. We might want to provide a way to > keep 'logical' WAL level some

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-10 Thread Masahiko Sawada
On Sun, Jun 8, 2025 at 8:57 PM shveta malik wrote: > > On Sat, Jun 7, 2025 at 2:44 AM Masahiko Sawada wrote: > > > > On Fri, Jun 6, 2025 at 3:02 AM shveta malik wrote: > > > > > > On Wed, Jun 4, 2025 at 3:40 PM shveta malik > > > wrote: > > > > > > > > On Wed, Jun 4, 2025 at 6:41 AM Masahiko S

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-08 Thread shveta malik
On Sat, Jun 7, 2025 at 2:44 AM Masahiko Sawada wrote: > > On Fri, Jun 6, 2025 at 3:02 AM shveta malik wrote: > > > > On Wed, Jun 4, 2025 at 3:40 PM shveta malik wrote: > > > > > > On Wed, Jun 4, 2025 at 6:41 AM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, May 20, 2025 at 9:54 PM Amit K

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-06 Thread Masahiko Sawada
On Fri, Jun 6, 2025 at 3:02 AM shveta malik wrote: > > On Wed, Jun 4, 2025 at 3:40 PM shveta malik wrote: > > > > On Wed, Jun 4, 2025 at 6:41 AM Masahiko Sawada > > wrote: > > > > > > On Tue, May 20, 2025 at 9:54 PM Amit Kapila > > > wrote: > > > > > > > > Yeah, I find the idea that the prese

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-06 Thread shveta malik
On Wed, Jun 4, 2025 at 3:40 PM shveta malik wrote: > > On Wed, Jun 4, 2025 at 6:41 AM Masahiko Sawada wrote: > > > > On Tue, May 20, 2025 at 9:54 PM Amit Kapila wrote: > > > > > > Yeah, I find the idea that the presence of a logical slot will allow > > > the user to enable logical decoding/repli

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-04 Thread shveta malik
On Wed, Jun 4, 2025 at 6:41 AM Masahiko Sawada wrote: > > On Tue, May 20, 2025 at 9:54 PM Amit Kapila wrote: > > > > Yeah, I find the idea that the presence of a logical slot will allow > > the user to enable logical decoding/replication more appealing than > > this new alternative, leaving aside

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-06-03 Thread Masahiko Sawada
On Tue, May 20, 2025 at 9:54 PM Amit Kapila wrote: > > On Wed, May 21, 2025 at 12:45 AM Masahiko Sawada > wrote: > > > > On Mon, May 19, 2025 at 2:05 AM Amit Kapila wrote: > > > > > > On Sun, May 18, 2025 at 1:09 AM Masahiko Sawada > > > wrote: > > > > > > > > On Sat, May 10, 2025 at 7:08 AM

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-20 Thread Amit Kapila
On Wed, May 21, 2025 at 12:45 AM Masahiko Sawada wrote: > > On Mon, May 19, 2025 at 2:05 AM Amit Kapila wrote: > > > > On Sun, May 18, 2025 at 1:09 AM Masahiko Sawada > > wrote: > > > > > > On Sat, May 10, 2025 at 7:08 AM Amit Kapila > > > wrote: > > > > > > > > > > > > Can we have a paramete

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-20 Thread Masahiko Sawada
On Mon, May 19, 2025 at 2:05 AM Amit Kapila wrote: > > On Sun, May 18, 2025 at 1:09 AM Masahiko Sawada wrote: > > > > On Sat, May 10, 2025 at 7:08 AM Amit Kapila wrote: > > > > > > > > > Can we have a parameter like immediately_reserve in > > > create_logical_slot API, similar to what we have fo

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-19 Thread Amit Kapila
On Sun, May 18, 2025 at 1:09 AM Masahiko Sawada wrote: > > On Sat, May 10, 2025 at 7:08 AM Amit Kapila wrote: > > > > > > Can we have a parameter like immediately_reserve in > > create_logical_slot API, similar to what we have for physical slots? > > We need to work out the details, but that shou

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-19 Thread Amit Kapila
On Sun, May 18, 2025 at 4:06 PM Dilip Kumar wrote: > > Thanks for proposing the idea of making wal_level configurable at > runtime. But why isn't making the relevant GUCs SIGHUP-reloadable > sufficient? > > For enabling logical replication, users are already familiar with the > wal_level and max_w

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-18 Thread Dilip Kumar
On Sun, May 18, 2025 at 1:09 AM Masahiko Sawada wrote: > > On Sat, May 10, 2025 at 7:08 AM Amit Kapila wrote: > > > > On Sat, May 10, 2025 at 1:05 PM Masahiko Sawada > > wrote: > > > > > > On Sat, May 10, 2025 at 12:00 AM Amit Kapila > > > wrote: > > > > > > > > > > > > Right, but to an exten

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-17 Thread Masahiko Sawada
On Sat, May 10, 2025 at 7:08 AM Amit Kapila wrote: > > On Sat, May 10, 2025 at 1:05 PM Masahiko Sawada wrote: > > > > On Sat, May 10, 2025 at 12:00 AM Amit Kapila > > wrote: > > > > > > > > > Right, but to an extent, this is also similar to having a requirement > > > of a logical slot on the pr

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-10 Thread Amit Kapila
On Sat, May 10, 2025 at 1:05 PM Masahiko Sawada wrote: > > On Sat, May 10, 2025 at 12:00 AM Amit Kapila wrote: > > > > > > Right, but to an extent, this is also similar to having a requirement > > of a logical slot on the primary. Now, it seems to me that the point > > you are trying to make is t

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-10 Thread Masahiko Sawada
On Sat, May 10, 2025 at 12:00 AM Amit Kapila wrote: > > On Thu, May 8, 2025 at 1:06 AM Masahiko Sawada wrote: > > > > On Tue, May 6, 2025 at 11:59 PM Amit Kapila wrote: > > > > > > On Thu, Apr 24, 2025 at 11:14 PM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Apr 24, 2025 at 5:30 AM Ami

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-10 Thread Amit Kapila
On Thu, May 8, 2025 at 1:06 AM Masahiko Sawada wrote: > > On Tue, May 6, 2025 at 11:59 PM Amit Kapila wrote: > > > > On Thu, Apr 24, 2025 at 11:14 PM Masahiko Sawada > > wrote: > > > > > > On Thu, Apr 24, 2025 at 5:30 AM Amit Kapila > > > wrote: > > > > > > > > On Wed, Apr 23, 2025 at 9:35 PM

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-07 Thread Masahiko Sawada
On Tue, May 6, 2025 at 11:59 PM Amit Kapila wrote: > > On Thu, Apr 24, 2025 at 11:14 PM Masahiko Sawada > wrote: > > > > On Thu, Apr 24, 2025 at 5:30 AM Amit Kapila wrote: > > > > > > On Wed, Apr 23, 2025 at 9:35 PM Masahiko Sawada > > > wrote: > > > > > > > > On Wed, Apr 23, 2025 at 5:46 AM

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-07 Thread Masahiko Sawada
On Tue, May 6, 2025 at 12:19 AM Bertrand Drouvot wrote: > > Hi, > > On Mon, Apr 21, 2025 at 10:31:03AM -0700, Masahiko Sawada wrote: > > I would like to discuss behavioral and user interface considerations. > > > > Upon further analysis of this patch regarding the conversion of > > wal_level to a

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-06 Thread Amit Kapila
On Thu, Apr 24, 2025 at 11:14 PM Masahiko Sawada wrote: > > On Thu, Apr 24, 2025 at 5:30 AM Amit Kapila wrote: > > > > On Wed, Apr 23, 2025 at 9:35 PM Masahiko Sawada > > wrote: > > > > > > On Wed, Apr 23, 2025 at 5:46 AM Amit Kapila > > > wrote: > > > > > > > > BTW, did we consider the idea

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-05-06 Thread Bertrand Drouvot
Hi, On Mon, Apr 21, 2025 at 10:31:03AM -0700, Masahiko Sawada wrote: > I would like to discuss behavioral and user interface considerations. > > Upon further analysis of this patch regarding the conversion of > wal_level to a SIGHUP parameter, I find that supporting all > combinations of wal_leve

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-04-24 Thread Masahiko Sawada
On Thu, Apr 24, 2025 at 5:30 AM Amit Kapila wrote: > > On Wed, Apr 23, 2025 at 9:35 PM Masahiko Sawada wrote: > > > > On Wed, Apr 23, 2025 at 5:46 AM Amit Kapila wrote: > > > > > > BTW, did we consider the idea to automatically transition to 'logical' > > > when the first logical slot is created

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-04-24 Thread Amit Kapila
On Wed, Apr 23, 2025 at 9:35 PM Masahiko Sawada wrote: > > On Wed, Apr 23, 2025 at 5:46 AM Amit Kapila wrote: > > > > BTW, did we consider the idea to automatically transition to 'logical' > > when the first logical slot is created and transition back to > > 'replica' when last logical slot gets

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-04-23 Thread Masahiko Sawada
On Wed, Apr 23, 2025 at 5:46 AM Amit Kapila wrote: > > On Mon, Apr 21, 2025 at 11:01 PM Masahiko Sawada > wrote: > > > > I would like to discuss behavioral and user interface considerations. > > > > Upon further analysis of this patch regarding the conversion of > > wal_level to a SIGHUP paramet

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-04-23 Thread Amit Kapila
On Mon, Apr 21, 2025 at 11:01 PM Masahiko Sawada wrote: > > I would like to discuss behavioral and user interface considerations. > > Upon further analysis of this patch regarding the conversion of > wal_level to a SIGHUP parameter, I find that supporting all > combinations of wal_level value chan

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-04-21 Thread Masahiko Sawada
On Thu, Feb 20, 2025 at 10:05 AM Masahiko Sawada wrote: > > On Wed, Feb 19, 2025 at 1:56 AM Bertrand Drouvot > wrote: > > > > Hi, > > Thank you for looking at the patches. > > > > > On Mon, Feb 17, 2025 at 12:07:56PM -0800, Masahiko Sawada wrote: > > > On Fri, Feb 14, 2025 at 2:35 AM Bertrand Dro

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-20 Thread Masahiko Sawada
On Wed, Feb 19, 2025 at 1:56 AM Bertrand Drouvot wrote: > > Hi, Thank you for looking at the patches. > > On Mon, Feb 17, 2025 at 12:07:56PM -0800, Masahiko Sawada wrote: > > On Fri, Feb 14, 2025 at 2:35 AM Bertrand Drouvot > > wrote: > > > Yeap, that was exactly my point when I mentioned the c

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-19 Thread Bertrand Drouvot
Hi, On Mon, Feb 17, 2025 at 12:07:56PM -0800, Masahiko Sawada wrote: > On Fri, Feb 14, 2025 at 2:35 AM Bertrand Drouvot > wrote: > > Yeap, that was exactly my point when I mentioned the custodian thread > > (taking > > into account Tom's comment quoted above). > > > > I've written PoC patches t

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-17 Thread Masahiko Sawada
On Fri, Feb 14, 2025 at 2:35 AM Bertrand Drouvot wrote: > > Hi, > > On Fri, Feb 14, 2025 at 12:17:48AM -0800, Masahiko Sawada wrote: > > On Tue, Feb 11, 2025 at 11:44 PM Bertrand Drouvot > > wrote: > > > Looking at the latest custodian worker patch, the basic architecture > > is to have a single

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-14 Thread Bertrand Drouvot
Hi, On Fri, Feb 14, 2025 at 12:17:48AM -0800, Masahiko Sawada wrote: > On Tue, Feb 11, 2025 at 11:44 PM Bertrand Drouvot > wrote: > Looking at the latest custodian worker patch, the basic architecture > is to have a single custodian worker and processes can ask it for some > work such as removin

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-14 Thread Masahiko Sawada
On Tue, Feb 11, 2025 at 11:44 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Feb 11, 2025 at 02:11:10PM -0800, Masahiko Sawada wrote: > > I've updated the patch that includes comment updates and bug fixes. > > Thanks! > > > The main idea of changing WAL level online is to decouple two aspects: > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-11 Thread Bertrand Drouvot
Hi, On Tue, Feb 11, 2025 at 02:11:10PM -0800, Masahiko Sawada wrote: > I've updated the patch that includes comment updates and bug fixes. Thanks! > The main idea of changing WAL level online is to decouple two aspects: > (1) the information included in WAL records and (2) the > functionalities

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-11 Thread Masahiko Sawada
On Fri, Jan 24, 2025 at 11:16 AM Masahiko Sawada wrote: > > On Thu, Jan 23, 2025 at 3:24 AM Ashutosh Bapat > wrote: > > > > On Thu, Jan 23, 2025 at 6:16 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Jan 10, 2025 at 12:33 AM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Jan 9, 2025

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-04 Thread Masahiko Sawada
On Mon, Feb 3, 2025 at 3:40 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > > I'm concerned that users could be confused if two different names > > refer to substantially the same thing. > > > > Having said that, I guess that we need to drastically change the > > messages. For example,

RE: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-03 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, > I'm concerned that users could be confused if two different names > refer to substantially the same thing. > > Having said that, I guess that we need to drastically change the > messages. For example, I think that the wal_level worker should say > something like "successfully m

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-28 Thread Masahiko Sawada
On Tue, Jan 28, 2025 at 1:39 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Sawada-san, > > I love the idea. I've roughly tested the patch and worked on my env. > Here are initial comments... Thank you for looking at the patch! > > 1. xloglevelworker.c > ``` > +#include "replication/logicalxlog.h" >

RE: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-28 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, I love the idea. I've roughly tested the patch and worked on my env. Here are initial comments... 1. xloglevelworker.c ``` +#include "replication/logicalxlog.h" ``` xloglevelworker.c includes replication/logicalxlog.h, but it does not exist. The line had to be removed to build a

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-24 Thread Masahiko Sawada
On Wed, Jan 22, 2025 at 11:42 PM Bertrand Drouvot wrote: > > Hi, > > On Wed, Jan 22, 2025 at 04:46:00PM -0800, Masahiko Sawada wrote: > > I would like to summarize the proposed approaches thus far: > > Thanks! > > > Regarding the user interface, there are three approaches: > > > > 1. Implementing

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-24 Thread Masahiko Sawada
On Thu, Jan 23, 2025 at 3:24 AM Ashutosh Bapat wrote: > > On Thu, Jan 23, 2025 at 6:16 AM Masahiko Sawada wrote: > > > > On Fri, Jan 10, 2025 at 12:33 AM Masahiko Sawada > > wrote: > > > > > > On Thu, Jan 9, 2025 at 3:29 AM Ashutosh Bapat > > > wrote: > > > > > > > > On Tue, Dec 31, 2024 at 10

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-23 Thread Ashutosh Bapat
On Thu, Jan 23, 2025 at 6:16 AM Masahiko Sawada wrote: > > On Fri, Jan 10, 2025 at 12:33 AM Masahiko Sawada > wrote: > > > > On Thu, Jan 9, 2025 at 3:29 AM Ashutosh Bapat > > wrote: > > > > > > On Tue, Dec 31, 2024 at 10:15 AM Masahiko Sawada > > > wrote: > > > > > > > > Hi all, > > > > > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-22 Thread Bertrand Drouvot
Hi, On Wed, Jan 22, 2025 at 04:46:00PM -0800, Masahiko Sawada wrote: > I would like to summarize the proposed approaches thus far: Thanks! > Regarding the user interface, there are three approaches: > > 1. Implementing SQL function controls (e.g., > pg_activate_logical_decoding() and pg_deactiv

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-22 Thread Masahiko Sawada
On Fri, Jan 10, 2025 at 12:33 AM Masahiko Sawada wrote: > > On Thu, Jan 9, 2025 at 3:29 AM Ashutosh Bapat > wrote: > > > > On Tue, Dec 31, 2024 at 10:15 AM Masahiko Sawada > > wrote: > > > > > > Hi all, > > > > > > Logical decoding (and logical replication) are available only when > > > wal_lev

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-22 Thread Masahiko Sawada
On Wed, Jan 22, 2025 at 2:04 AM Bertrand Drouvot wrote: > > Hi, > > On Mon, Jan 06, 2025 at 10:32:52PM -0800, Masahiko Sawada wrote: > > On Mon, Jan 6, 2025 at 3:20 AM Ashutosh Bapat > > wrote: > > > > > > On Sat, Jan 4, 2025 at 6:03 AM Masahiko Sawada > > > wrote: > > > > > > > > On Fri, Jan 3

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-22 Thread Bertrand Drouvot
Hi, On Mon, Jan 06, 2025 at 10:32:52PM -0800, Masahiko Sawada wrote: > On Mon, Jan 6, 2025 at 3:20 AM Ashutosh Bapat > wrote: > > > > On Sat, Jan 4, 2025 at 6:03 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Jan 3, 2025 at 6:31 AM Euler Taveira wrote: > > > > > > > > On Fri, Jan 3, 2025, a

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-15 Thread Peter Smith
On Thu, Jan 16, 2025 at 4:28 AM Masahiko Sawada wrote: > > On Sun, Jan 12, 2025 at 10:52 PM Peter Smith wrote: > > > > On Fri, Jan 10, 2025 at 8:28 PM Masahiko Sawada > > wrote: > > > > > > Hi, > > > > > > On Tue, Jan 7, 2025 at 11:30 PM Peter Smith wrote: > > > > > > > > Hi Sawada-San. > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-15 Thread Masahiko Sawada
On Tue, Jan 14, 2025 at 9:46 PM Ashutosh Bapat wrote: > > On Tue, Jan 14, 2025 at 3:15 AM Masahiko Sawada wrote: > > > > On Mon, Jan 13, 2025 at 1:31 AM Ashutosh Bapat > > wrote: > > > > > > On Mon, Jan 13, 2025 at 2:52 PM vignesh C wrote: > > > > > > > > I felt that the only disadvantage with

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-15 Thread Masahiko Sawada
On Sun, Jan 12, 2025 at 10:52 PM Peter Smith wrote: > > On Fri, Jan 10, 2025 at 8:28 PM Masahiko Sawada wrote: > > > > Hi, > > > > On Tue, Jan 7, 2025 at 11:30 PM Peter Smith wrote: > > > > > > Hi Sawada-San. > > > > > > FWIW, I also thought it was a good idea suggested by Bertrand [1] to > > >

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-14 Thread Ashutosh Bapat
On Tue, Jan 14, 2025 at 3:15 AM Masahiko Sawada wrote: > > On Mon, Jan 13, 2025 at 1:31 AM Ashutosh Bapat > wrote: > > > > On Mon, Jan 13, 2025 at 2:52 PM vignesh C wrote: > > > > > > I felt that the only disadvantage with this approach is that we > > > currently wait for all in-progress transac

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-14 Thread vignesh C
On Tue, 14 Jan 2025 at 03:03, Masahiko Sawada wrote: > > Yeah, delegating the activation to the background process such as the > checkpointer would also be one solution. This would work with the > approach that we enable the logical decoding via > pg_activate_logical_decoding(). On the other hand,

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-13 Thread Masahiko Sawada
On Mon, Jan 13, 2025 at 1:31 AM Ashutosh Bapat wrote: > > On Mon, Jan 13, 2025 at 2:52 PM vignesh C wrote: > > > > I felt that the only disadvantage with this approach is that we > > currently wait for all in-progress transactions to complete before > > enabling logical decoding. If a long-runnin

  1   2   >