Re: Fix slot synchronization with two_phase decoding enabled

2025-06-12 Thread Nisha Moond
On Thu, Jun 12, 2025 at 4:00 AM Peter Smith wrote: > > On Wed, Jun 11, 2025 at 8:16 PM Ajin Cherian wrote: > > > > On Fri, Jun 6, 2025 at 5:07 PM Nisha Moond wrote: > > > > > > > > > Attached v18 patch. > > > - patch-001: modified error messages as suggested above. > > > - patch-002: improved

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-12 Thread Nisha Moond
On Tue, Jun 10, 2025 at 2:29 PM shveta malik wrote: > > On Fri, Jun 6, 2025 at 12:37 PM Nisha Moond wrote: > > > > Attached v18 patch. > > - patch-001: modified error messages as suggested above. > > - patch-002: improved pg_dump docs as per Shveta's off-list suggestions. > > > > Thanks for the

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-11 Thread shveta malik
On Wed, Jun 11, 2025 at 11:31 PM Masahiko Sawada wrote: > > BTW have we addressed the point Amit mentioned before[1]? > > > The one more combination to consider is when someone takes a dump of > > an older version and loads it into a newer version. For example, where > > users dump from 17.5 and

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-11 Thread Peter Smith
On Wed, Jun 11, 2025 at 8:16 PM Ajin Cherian wrote: > > On Fri, Jun 6, 2025 at 5:07 PM Nisha Moond wrote: > > > > > > Attached v18 patch. > > - patch-001: modified error messages as suggested above. > > - patch-002: improved pg_dump docs as per Shveta's off-list suggestions. > > > > [1] > > ht

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-11 Thread Masahiko Sawada
On Fri, Jun 6, 2025 at 12:07 AM Nisha Moond wrote: > > On Thu, Jun 5, 2025 at 3:26 PM Dilip Kumar wrote: > > > > On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > > > > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond > > > wrote: > > > > > > > > > > > > Attached v17 patches. Added a top-up p

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-11 Thread Ajin Cherian
On Fri, Jun 6, 2025 at 5:07 PM Nisha Moond wrote: > > > Attached v18 patch. > - patch-001: modified error messages as suggested above. > - patch-002: improved pg_dump docs as per Shveta's off-list suggestions. > > [1] > https://www.postgresql.org/message-id/CAA4eK1%2BB067G8mUJzKUEjc5KSkYq6z0utT

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-10 Thread shveta malik
On Fri, Jun 6, 2025 at 12:37 PM Nisha Moond wrote: > > Attached v18 patch. > - patch-001: modified error messages as suggested above. > - patch-002: improved pg_dump docs as per Shveta's off-list suggestions. > Thanks for the patches. Please find few comments: 1) + * However, we allow this com

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-06 Thread Dilip Kumar
On Thu, Jun 5, 2025 at 3:59 PM Amit Kapila wrote: > > On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond > > wrote: > > > > > > > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > > > suggested by Amit above. > > > > I

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-06 Thread Nisha Moond
On Thu, Jun 5, 2025 at 3:26 PM Dilip Kumar wrote: > > On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond > > wrote: > > > > > > > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > > > suggested by Amit above. > > > > I

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Nisha Moond
On Wed, Jun 4, 2025 at 11:16 PM Masahiko Sawada wrote: > > On Sun, Jun 1, 2025 at 10:25 PM Amit Kapila wrote: > > > > The one more combination to consider is when someone takes a dump of > > an older version and loads it into a newer version. For example, where > > users dump from 17.5 and then r

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Dilip Kumar
On Thu, Jun 5, 2025 at 3:59 PM Amit Kapila wrote: > > On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond > > wrote: > > > > > > > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > > > suggested by Amit above. > > > > I

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Amit Kapila
On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond wrote: > > > > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > > suggested by Amit above. > > I have started reviewing this, although I haven't done a complete > review ye

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Dilip Kumar
On Thu, Jun 5, 2025 at 2:53 PM Dilip Kumar wrote: > > On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond wrote: > > > > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > > suggested by Amit above. > > I have started reviewing this, although I haven't done a complete > review ye

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Dilip Kumar
On Tue, Jun 3, 2025 at 11:05 AM Nisha Moond wrote: > > > Attached v17 patches. Added a top-up patch 0002 implementing the idea > suggested by Amit above. I have started reviewing this, although I haven't done a complete review yet, but I have a question on the fix we are trying to do, IIUC we are

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-05 Thread Masahiko Sawada
On Wed, Jun 4, 2025 at 10:46 AM Masahiko Sawada wrote: > > On Sun, Jun 1, 2025 at 10:25 PM Amit Kapila wrote: > > > > > > Yet another idea is to dump the subscription with two_phase = on and > > failover = false. We should do this when both options are 'true' > > during the dump. As we are docum

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-04 Thread Masahiko Sawada
On Sun, Jun 1, 2025 at 10:25 PM Amit Kapila wrote: > > > Yet another idea is to dump the subscription with two_phase = on and > failover = false. We should do this when both options are 'true' > during the dump. As we are documenting that we always dump > createsubscription with connect as false

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-02 Thread Nisha Moond
On Mon, Jun 2, 2025 at 10:55 AM Amit Kapila wrote: > > On Fri, May 30, 2025 at 3:00 PM Nisha Moond wrote: > > > > Agree that we need to cover the simple pg_dump and pg_restore with the > > patch. > > > > When pg_dump and pg_restore are used outside of pg_upgrade, there's no > > guarantee that th

Re: Fix slot synchronization with two_phase decoding enabled

2025-06-01 Thread Amit Kapila
On Fri, May 30, 2025 at 3:00 PM Nisha Moond wrote: > > Agree that we need to cover the simple pg_dump and pg_restore with the patch. > > When pg_dump and pg_restore are used outside of pg_upgrade, there's no > guarantee that the target system does not have any prepared > transactions. In such case

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-30 Thread Nisha Moond
On Fri, May 30, 2025 at 12:55 AM Masahiko Sawada wrote: > > On Thu, May 29, 2025 at 2:00 AM Nisha Moond wrote: > > > > On Wed, May 28, 2025 at 6:10 AM Masahiko Sawada > > wrote: > > > > > > On Sun, May 25, 2025 at 11:34 PM Nisha Moond > > > wrote: > > > > > > > > to > > > > On Fri, May 23, 2

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-29 Thread Masahiko Sawada
On Thu, May 29, 2025 at 2:00 AM Nisha Moond wrote: > > On Wed, May 28, 2025 at 6:10 AM Masahiko Sawada wrote: > > > > On Sun, May 25, 2025 at 11:34 PM Nisha Moond > > wrote: > > > > > > to > > > On Fri, May 23, 2025 at 10:06 PM Masahiko Sawada > > > wrote: > > > > > > > > > > > > Here are re

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-29 Thread Nisha Moond
On Wed, May 28, 2025 at 6:10 AM Masahiko Sawada wrote: > > On Sun, May 25, 2025 at 11:34 PM Nisha Moond wrote: > > > > to > > On Fri, May 23, 2025 at 10:06 PM Masahiko Sawada > > wrote: > > > > > > > > > Here are review comments for v14 patch: > > > > > > > Thank you for the review. > > > > >

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-27 Thread Masahiko Sawada
On Sun, May 25, 2025 at 11:34 PM Nisha Moond wrote: > > to > On Fri, May 23, 2025 at 10:06 PM Masahiko Sawada > wrote: > > > > > > Here are review comments for v14 patch: > > > > Thank you for the review. > > > I think we need to include a basic test case where we simply create a > > subscripti

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-25 Thread Nisha Moond
to On Fri, May 23, 2025 at 10:06 PM Masahiko Sawada wrote: > > > Here are review comments for v14 patch: > Thank you for the review. > I think we need to include a basic test case where we simply create a > subscription with two_phase=true and then enable the failover via > ALTER SUBSCRIPTION a

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-23 Thread Masahiko Sawada
On Wed, May 21, 2025 at 4:02 PM Masahiko Sawada wrote: > > On Fri, May 9, 2025 at 5:09 AM Nisha Moond wrote: > > > > On Thu, May 8, 2025 at 11:35 AM shveta malik wrote: > > > > > > On Wed, May 7, 2025 at 4:36 PM Nisha Moond > > > wrote: > > > > > > > > > > > > Attached is the v13 patch with th

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-21 Thread Masahiko Sawada
On Fri, May 9, 2025 at 5:09 AM Nisha Moond wrote: > > On Thu, May 8, 2025 at 11:35 AM shveta malik wrote: > > > > On Wed, May 7, 2025 at 4:36 PM Nisha Moond wrote: > > > > > > > > > Attached is the v13 patch with the above comments addressed. > > > > > > -- > > > > Thanks for the patch. > > > >

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-20 Thread Nisha Moond
On Tue, May 6, 2025 at 4:52 PM Zhijie Hou (Fujitsu) wrote: > > On Mon, May 5, 2025 at 6:59 PM Amit Kapila wrote: > > > > On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada > > wrote: > > > > > > While I cannot be entirely certain of my analysis, I believe the root > > > cause might be related to the

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-12 Thread shveta malik
On Sat, May 10, 2025 at 4:59 PM Amit Kapila wrote: > > On Tue, May 6, 2025 at 4:52 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Mon, May 5, 2025 at 6:59 PM Amit Kapila wrote: > > > > > > > > > Yes, this is possible. Here is my theory as to how it can happen in the > > > current > > > case. In the

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-10 Thread Amit Kapila
On Tue, May 6, 2025 at 4:52 PM Zhijie Hou (Fujitsu) wrote: > > On Mon, May 5, 2025 at 6:59 PM Amit Kapila wrote: > > > > > > Yes, this is possible. Here is my theory as to how it can happen in the > > current > > case. In the failed test, after the primary has prepared a transaction, the > > tran

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-09 Thread Nisha Moond
On Thu, May 8, 2025 at 11:35 AM shveta malik wrote: > > On Wed, May 7, 2025 at 4:36 PM Nisha Moond wrote: > > > > > > Attached is the v13 patch with the above comments addressed. > > > > -- > > Thanks for the patch. > > I think we can have the restriction mentioned under the 'two_phase' > section

RE: Fix slot synchronization with two_phase decoding enabled

2025-05-08 Thread Zhijie Hou (Fujitsu)
On Thu, May 8, 2025 at 6:04 PM Zhijie Hou (Fujitsu) wrote: > > On Tue, May 6, 2025 at 7:22 PM Zhijie Hou (Fujitsu) wrote: > > > > > On Mon, May 5, 2025 at 6:59 PM Amit Kapila wrote: > > > > > > On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada > > > > > wrote: > > > > > > > > While I cannot be enti

RE: Fix slot synchronization with two_phase decoding enabled

2025-05-08 Thread Zhijie Hou (Fujitsu)
On Tue, May 6, 2025 at 7:22 PM Zhijie Hou (Fujitsu) wrote: > > On Mon, May 5, 2025 at 6:59 PM Amit Kapila wrote: > > > > On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada > > > wrote: > > > > > > While I cannot be entirely certain of my analysis, I believe the > > > root cause might be related to

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-07 Thread shveta malik
On Wed, May 7, 2025 at 4:36 PM Nisha Moond wrote: > > > Attached is the v13 patch with the above comments addressed. > > -- Thanks for the patch. I think we can have the restriction mentioned under the 'two_phase' section as well along with the 'failover' section in the CREATE SUBSCRIPTION doc,

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-07 Thread Nisha Moond
On Wed, May 7, 2025 at 10:32 AM shveta malik wrote: > > On Mon, May 5, 2025 at 3:29 PM Nisha Moond wrote: > > > > Please find the v12 patch with above suggested changes. > > > > Thanks for the patch. Few comments for doc changes: > > 1) > func.sgml - pg_create_logical_replication_slot: > > +

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-06 Thread shveta malik
On Mon, May 5, 2025 at 3:29 PM Nisha Moond wrote: > > Please find the v12 patch with above suggested changes. > Thanks for the patch. Few comments for doc changes: 1) func.sgml - pg_create_logical_replication_slot: +failover. The parameters twophase and +failover cannot be enabl

RE: Fix slot synchronization with two_phase decoding enabled

2025-05-06 Thread Zhijie Hou (Fujitsu)
On Mon, May 5, 2025 at 6:59 PM Amit Kapila wrote: > > On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada > wrote: > > > > While I cannot be entirely certain of my analysis, I believe the root > > cause might be related to the backward movement of the confirmed_flush > > LSN. The following scenario se

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-05 Thread Masahiko Sawada
On Mon, May 5, 2025 at 2:59 AM Nisha Moond wrote: > > On Fri, May 2, 2025 at 3:05 PM shveta malik wrote: > > > > On Fri, May 2, 2025 at 12:57 PM Nisha Moond > > wrote: > > > > > > > > > Please find the v11 patch addressing the above points and all other > > > comments. I have also optimized the

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-05 Thread Amit Kapila
On Tue, May 6, 2025 at 12:26 AM Masahiko Sawada wrote: > > On Mon, May 5, 2025 at 3:59 AM Amit Kapila wrote: > > > > On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada > > wrote: > > > > > > While I cannot be entirely certain of my analysis, I believe the root > > > cause might be related to the ba

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-05 Thread Masahiko Sawada
On Mon, May 5, 2025 at 3:59 AM Amit Kapila wrote: > > On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada wrote: > > > > While I cannot be entirely certain of my analysis, I believe the root > > cause might be related to the backward movement of the confirmed_flush > > LSN. The following scenario seem

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-05 Thread Amit Kapila
On Sun, May 4, 2025 at 2:33 PM Masahiko Sawada wrote: > > While I cannot be entirely certain of my analysis, I believe the root > cause might be related to the backward movement of the confirmed_flush > LSN. The following scenario seems possible: > > 1. The walsender enables the two_phase and sets

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-05 Thread Nisha Moond
On Fri, May 2, 2025 at 3:05 PM shveta malik wrote: > > On Fri, May 2, 2025 at 12:57 PM Nisha Moond wrote: > > > > > > Please find the v11 patch addressing the above points and all other > > comments. I have also optimized the test by reducing the number of > > subscriptions and slots. > > > > Tha

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-04 Thread Masahiko Sawada
On Sat, May 3, 2025 at 10:23 AM Tom Lane wrote: > > BF member mule just showed what seems an identical failure [1]: > > 2025-05-03 17:46:20.088 CEST [24308:3] LOG: database system is ready to > accept read-only connections > 2025-05-03 17:46:20.091 CEST [24321:1] LOG: slot sync worker started >

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-03 Thread Tom Lane
BF member mule just showed what seems an identical failure [1]: 2025-05-03 17:46:20.088 CEST [24308:3] LOG: database system is ready to accept read-only connections 2025-05-03 17:46:20.091 CEST [24321:1] LOG: slot sync worker started 2025-05-03 17:46:20.100 CEST [24322:1] LOG: started streamin

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-02 Thread shveta malik
On Fri, May 2, 2025 at 12:57 PM Nisha Moond wrote: > > > Please find the v11 patch addressing the above points and all other > comments. I have also optimized the test by reducing the number of > subscriptions and slots. > Thanks for the patch. Few comments: 1) pg_upgrade/t/003_logical_slots.pl

Re: Fix slot synchronization with two_phase decoding enabled

2025-05-02 Thread Nisha Moond
On Wed, Apr 30, 2025 at 5:45 AM Masahiko Sawada wrote: > > On Tue, Apr 29, 2025 at 5:00 AM Nisha Moond wrote: > > Thank you for updating the patch! Here are some comments on v10. > Thanks for reviewing the patch! > --- > + > +# Also wait for two-phase to be enabled > +$subscriber1->poll_query_u

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-30 Thread Masahiko Sawada
On Wed, Apr 30, 2025 at 2:38 AM Zhijie Hou (Fujitsu) wrote: > > On Tue, Apr 29, 2025 at 6:57 PM Amit Kapila wrote: > > > > On Tue, Apr 29, 2025 at 3:01 AM Masahiko Sawada > > wrote: > > > > > > Your fix looks good to me. Is it worth considering putting an > > > assertion to verify if new two_phas

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-30 Thread Zhijie Hou (Fujitsu)
On Tue, Apr 29, 2025 at 6:57 PM Amit Kapila wrote: > > On Tue, Apr 29, 2025 at 3:01 AM Masahiko Sawada > wrote: > > > > Your fix looks good to me. Is it worth considering putting an > > assertion to verify if new two_phase_at is equal to or greater than > > confirmed_lsn (or at least it doesn't g

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread Masahiko Sawada
On Tue, Apr 29, 2025 at 5:00 AM Nisha Moond wrote: > > On Tue, Apr 29, 2025 at 2:51 PM shveta malik wrote: > > > > On Mon, Apr 28, 2025 at 4:33 PM Nisha Moond > > wrote: > > > > > > Please find the v9 patch, addressing the above and all other comments as > > > well. > > > > > > > Thanks for th

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread Nisha Moond
On Tue, Apr 29, 2025 at 2:51 PM shveta malik wrote: > > On Mon, Apr 28, 2025 at 4:33 PM Nisha Moond wrote: > > > > Please find the v9 patch, addressing the above and all other comments as > > well. > > > > Thanks for the patch. > > 1) > > + The default is false. This option cannot be se

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread Amit Kapila
On Tue, Apr 29, 2025 at 3:01 AM Masahiko Sawada wrote: > > Your fix looks good to me. Is it worth considering putting an > assertion to verify if new two_phase_at is equal to or greater than > confirmed_lsn (or at least it doesn't go backward), when syncing > two_phase_at? > Yeah, it makes sense.

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread shveta malik
On Mon, Apr 28, 2025 at 4:33 PM Nisha Moond wrote: > > Please find the v9 patch, addressing the above and all other comments as well. > Thanks for the patch. 1) + The default is false. This option cannot be set together with + two_phase when creating the slot. However, once +

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Zhijie Hou (Fujitsu)
On Mon, Apr 28, 2025 at 7:33 PM Zhijie Hou (Fujitsu) wrote: > > Thanks for reviewing. Here is V3 patch that addressed it. > > BTW, I also did some tests to confirm the catalog_xmin could still be > ahead in some case, and here is an example: > > 1. Create a failover replication slot named 'logi

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Amit Kapila
On Mon, Apr 28, 2025 at 11:54 PM Masahiko Sawada wrote: > > On Sat, Apr 26, 2025 at 5:07 AM Amit Kapila wrote: > > > > On Fri, Apr 25, 2025 at 9:57 PM Masahiko Sawada > > wrote: > > > > > > On Fri, Apr 25, 2025 at 3:43 AM Amit Kapila > > > wrote: > > > > > > > > The second can mislead the use

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Masahiko Sawada
On Mon, Apr 28, 2025 at 4:33 AM Zhijie Hou (Fujitsu) wrote: > > On Mon, Apr 28, 2025 at 5:33 PM shveta malik wrote: > > > > On Mon, Apr 28, 2025 at 2:27 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Fri, Apr 18, 2025 at 12:29 PM Amit Kapila wrote: > > > > > > > > On Thu, Apr 17, 2025 at 6:14 

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Masahiko Sawada
On Sat, Apr 26, 2025 at 5:07 AM Amit Kapila wrote: > > On Fri, Apr 25, 2025 at 9:57 PM Masahiko Sawada wrote: > > > > On Fri, Apr 25, 2025 at 3:43 AM Amit Kapila wrote: > > > > > > The second can mislead the user > > > for a long period in cases where prepare and commit have a large time > > > g

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Zhijie Hou (Fujitsu)
On Mon, Apr 28, 2025 at 5:33 PM shveta malik wrote: > > On Mon, Apr 28, 2025 at 2:27 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Fri, Apr 18, 2025 at 12:29 PM Amit Kapila wrote: > > > > > > On Thu, Apr 17, 2025 at 6:14 PM Zhijie Hou (Fujitsu) > > > > > > > > > > > - > > > > Fix > > > > -

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Nisha Moond
On Mon, Apr 28, 2025 at 12:04 PM shveta malik wrote: > > On Fri, Apr 25, 2025 at 5:53 PM Nisha Moond wrote: > > > > Please find the attached v8 patch with above comments addressed. > > This version includes the documentation updates suggested by > > Sawada-san at [1], and incorporates his feedbac

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread shveta malik
On Mon, Apr 28, 2025 at 2:27 PM Zhijie Hou (Fujitsu) wrote: > > On Fri, Apr 18, 2025 at 12:29 PM Amit Kapila wrote: > > > > On Thu, Apr 17, 2025 at 6:14 PM Zhijie Hou (Fujitsu) > > > > > > > > - > > > Fix > > > - > > > > > > I think we should keep the confirmed_flush even if the previous

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Zhijie Hou (Fujitsu)
On Fri, Apr 18, 2025 at 12:29 PM Amit Kapila wrote: > > On Thu, Apr 17, 2025 at 6:14 PM Zhijie Hou (Fujitsu) > > > > - > > Fix > > - > > > > I think we should keep the confirmed_flush even if the previous synced > > restart_lsn/catalog_xmin is newer. Attachments include a patch for the >

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-27 Thread shveta malik
On Fri, Apr 25, 2025 at 5:53 PM Nisha Moond wrote: > > Please find the attached v8 patch with above comments addressed. > This version includes the documentation updates suggested by > Sawada-san at [1], and incorporates his feedback from [2] as well. > Thanks for the patches. 1) Regarding docu

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-26 Thread Amit Kapila
On Fri, Apr 25, 2025 at 9:57 PM Masahiko Sawada wrote: > > On Fri, Apr 25, 2025 at 3:43 AM Amit Kapila wrote: > > > > On Fri, Apr 25, 2025 at 6:02 AM Masahiko Sawada > > wrote: > > > > > > I realized that users who create a logical slot using > > > pg_create_logical_replication_slot() would not

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-25 Thread Masahiko Sawada
On Fri, Apr 25, 2025 at 3:43 AM Amit Kapila wrote: > > On Fri, Apr 25, 2025 at 6:02 AM Masahiko Sawada wrote: > > > > I realized that users who create a logical slot using > > pg_create_logical_replication_slot() would not be able to enable both > > options at slot creation, and there is no easy

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-25 Thread Nisha Moond
On Thu, Apr 24, 2025 at 3:57 PM shveta malik wrote: > > On Thu, Apr 24, 2025 at 2:54 PM Nisha Moond wrote: > > > > Please find the updated patch for Approach 3, which implements the > > idea suggested by Swada-san above. > > > > Thank You for the patch. > > 1) > > CreateDecodingContext: > > if

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-25 Thread Amit Kapila
On Fri, Apr 25, 2025 at 6:02 AM Masahiko Sawada wrote: > > I realized that users who create a logical slot using > pg_create_logical_replication_slot() would not be able to enable both > options at slot creation, and there is no easy way to enable the > failover after two_phase-enabled-slot creati

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-24 Thread Masahiko Sawada
On Thu, Apr 24, 2025 at 10:48 AM Masahiko Sawada wrote: > > On Thu, Apr 24, 2025 at 2:24 AM Nisha Moond wrote: > > > > On Thu, Apr 24, 2025 at 12:28 PM Amit Kapila > > wrote: > > > > > > On Wed, Apr 23, 2025 at 11:04 PM Masahiko Sawada > > > wrote: > > > > > > > > On Tue, Apr 22, 2025 at 3:00

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-24 Thread Masahiko Sawada
On Thu, Apr 24, 2025 at 2:24 AM Nisha Moond wrote: > > On Thu, Apr 24, 2025 at 12:28 PM Amit Kapila wrote: > > > > On Wed, Apr 23, 2025 at 11:04 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Apr 22, 2025 at 3:00 AM Amit Kapila > > > wrote: > > > > > > > > On Mon, Apr 21, 2025 at 8:44 AM Z

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-24 Thread shveta malik
On Thu, Apr 24, 2025 at 2:54 PM Nisha Moond wrote: > > Please find the updated patch for Approach 3, which implements the > idea suggested by Swada-san above. > Thank You for the patch. 1) CreateDecodingContext: if (ctx->twophase && !slot->data.two_phase) { + /* + * Do not allow two-phase

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-24 Thread Nisha Moond
On Thu, Apr 24, 2025 at 12:28 PM Amit Kapila wrote: > > On Wed, Apr 23, 2025 at 11:04 PM Masahiko Sawada > wrote: > > > > On Tue, Apr 22, 2025 at 3:00 AM Amit Kapila wrote: > > > > > > On Mon, Apr 21, 2025 at 8:44 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > On Sat, Apr 19, 2025 at 2:1

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-23 Thread Amit Kapila
On Wed, Apr 23, 2025 at 11:04 PM Masahiko Sawada wrote: > > On Tue, Apr 22, 2025 at 3:00 AM Amit Kapila wrote: > > > > On Mon, Apr 21, 2025 at 8:44 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Sat, Apr 19, 2025 at 2:19 AM Masahiko Sawada wrote: > > > > > > > > On Tue, Apr 8, 2025 at 10:14 P

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-23 Thread Masahiko Sawada
On Tue, Apr 22, 2025 at 3:00 AM Amit Kapila wrote: > > On Mon, Apr 21, 2025 at 8:44 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Sat, Apr 19, 2025 at 2:19 AM Masahiko Sawada wrote: > > > > > > On Tue, Apr 8, 2025 at 10:14 PM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > > > > > -- > >

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-23 Thread Nisha Moond
On Tue, Apr 22, 2025 at 3:23 PM shveta malik wrote: > > On Fri, Apr 11, 2025 at 1:03 PM Nisha Moond wrote: > > > > > > Patch "v5_aprch_3-0001" implements the above Approach 3. > > > > Thanks Hou-san for implementing approach-2 and providing the patch. > > > > I find Approach 2 better than Approac

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-22 Thread Amit Kapila
On Mon, Apr 21, 2025 at 8:44 AM Zhijie Hou (Fujitsu) wrote: > > On Sat, Apr 19, 2025 at 2:19 AM Masahiko Sawada wrote: > > > > On Tue, Apr 8, 2025 at 10:14 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > > > -- > > > Approach 2 > > > -- > > > > > > Instead of disallowing the use

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-22 Thread shveta malik
On Fri, Apr 11, 2025 at 1:03 PM Nisha Moond wrote: > > > Patch "v5_aprch_3-0001" implements the above Approach 3. > > Thanks Hou-san for implementing approach-2 and providing the patch. > I find Approach 2 better than Approach1. Yet to review Approach3. Please find my initial comments: Approach

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-21 Thread Zhijie Hou (Fujitsu)
On Tue, Apr 22, 2025 at 11:23 AM Amit Kapila wrote: > > On Fri, Apr 18, 2025 at 9:58 AM Amit Kapila wrote: > > > > On Thu, Apr 17, 2025 at 6:14 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > - > > > Fix > > > - > > > > > > I think we should keep the confirmed_flush even if the previous

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-21 Thread Amit Kapila
On Fri, Apr 18, 2025 at 9:58 AM Amit Kapila wrote: > > On Thu, Apr 17, 2025 at 6:14 PM Zhijie Hou (Fujitsu) > wrote: > > > > - > > Fix > > - > > > > I think we should keep the confirmed_flush even if the previous synced > > restart_lsn/catalog_xmin is newer. Attachments include a patch fo

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-20 Thread Zhijie Hou (Fujitsu)
On Sat, Apr 19, 2025 at 2:19 AM Masahiko Sawada wrote: > > On Tue, Apr 8, 2025 at 10:14 PM Zhijie Hou (Fujitsu) > wrote: > > > > > > -- > > Approach 2 > > -- > > > > Instead of disallowing the use of two-phase and failover together, a more > > flexible strategy could be only restr

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-18 Thread Masahiko Sawada
On Tue, Apr 8, 2025 at 10:14 PM Zhijie Hou (Fujitsu) wrote: > > > -- > Approach 2 > -- > > Instead of disallowing the use of two-phase and failover together, a more > flexible strategy could be only restrict failover for slots with two-phase > enabled when there's a possibility of

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-17 Thread Amit Kapila
On Thu, Apr 17, 2025 at 6:14 PM Zhijie Hou (Fujitsu) wrote: > > Hi, > > The recently added tests for slotsync on two-phase enabled slots failed[1] due > to a broken assertion triggered while decoding the COMMIT PREPARED record on > the promoted standby. > > - > Analysis > - > > if

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-17 Thread Zhijie Hou (Fujitsu)
On Thu, Apr 17, 2025 at 8:44 PM Zhijie Hou (Fujitsu) wrote: > > Hi, > > The recently added tests for slotsync on two-phase enabled slots > failed[1] due to a broken assertion triggered while decoding the > COMMIT PREPARED record on the promoted standby. Here is the missing link: [1] https:/

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-17 Thread Zhijie Hou (Fujitsu)
Hi, The recently added tests for slotsync on two-phase enabled slots failed[1] due to a broken assertion triggered while decoding the COMMIT PREPARED record on the promoted standby. - Analysis - if ((txn->final_lsn < two_phase_at) && is_commit) { /*

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-11 Thread Nisha Moond
HI, Please find the patches attached for all three approaches. On Wed, Apr 9, 2025 at 10:45 AM Zhijie Hou (Fujitsu) wrote: > > On Thu, Apr 3, 2025 at 3:16 PM Zhijie Hou (Fujitsu) wrote: > > > > On Thu, Apr 3, 2025 at 1:38 PM Masahiko Sawada wrote: > > > > > > > > On Wed, Apr 2, 2025 at 7:58 PM A

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-08 Thread Zhijie Hou (Fujitsu)
On Thu, Apr 3, 2025 at 3:16 PM Zhijie Hou (Fujitsu) wrote: > > On Thu, Apr 3, 2025 at 1:38 PM Masahiko Sawada wrote: > > > > > On Wed, Apr 2, 2025 at 7:58 PM Amit Kapila > > wrote: > > > > > > On Thu, Apr 3, 2025 at 7:50 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > On Thu, Apr 3, 2025 a

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-05 Thread Zhijie Hou (Fujitsu)
On Thu, Apr 3, 2025 at 1:38 PM Masahiko Sawada wrote: > > On Wed, Apr 2, 2025 at 7:58 PM Amit Kapila > wrote: > > > > On Thu, Apr 3, 2025 at 7:50 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Thu, Apr 3, 2025 at 3:30 AM Masahiko Sawada wrote: > > > > > > > > > > > On Wed, Apr 2, 2025 at 6:3

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-05 Thread Zhijie Hou (Fujitsu)
On Thu, Mar 27, 2025 at 2:29 PM Amit Kapila wrote: > > On Tue, Mar 25, 2025 at 12:1 PM Amit Kapila > wrote: > > > > On Tue, Mar 25, 2025 at 11:05 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Hi, > > > > > > When testing the slot synchronization with logical replication slots that > > > enable

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Amit Kapila
On Thu, Apr 3, 2025 at 11:08 AM Masahiko Sawada wrote: > > On Wed, Apr 2, 2025 at 7:58 PM Amit Kapila wrote: > > > > On Thu, Apr 3, 2025 at 7:50 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Thu, Apr 3, 2025 at 3:30 AM Masahiko Sawada wrote: > > > > > > > > > > > On Wed, Apr 2, 2025 at 6:33 

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Masahiko Sawada
On Wed, Apr 2, 2025 at 7:58 PM Amit Kapila wrote: > > On Thu, Apr 3, 2025 at 7:50 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Thu, Apr 3, 2025 at 3:30 AM Masahiko Sawada wrote: > > > > > > > > On Wed, Apr 2, 2025 at 6:33 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > Thank you for the explanat

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Amit Kapila
On Thu, Apr 3, 2025 at 7:50 AM Zhijie Hou (Fujitsu) wrote: > > On Thu, Apr 3, 2025 at 3:30 AM Masahiko Sawada wrote: > > > > > On Wed, Apr 2, 2025 at 6:33 AM Zhijie Hou (Fujitsu) > > wrote: > > > > Thank you for the explanation! I agree that the issue happens in these > > cases. > > > > As anoth

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Zhijie Hou (Fujitsu)
On Thu, Apr 3, 2025 at 3:30 AM Masahiko Sawada wrote: > > On Wed, Apr 2, 2025 at 6:33 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Wed, Apr 2, 2025 at 3:45 PM Masahiko Sawada wrote: > > > > Hi, > > > > > > > > On Mon, Mar 31, 2025 at 4:3 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > After further

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Masahiko Sawada
On Wed, Apr 2, 2025 at 6:33 AM Zhijie Hou (Fujitsu) wrote: > > On Wed, Apr 2, 2025 at 3:45 PM Masahiko Sawada wrote: > > Hi, > > > > > On Mon, Mar 31, 2025 at 4:3 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Thu, Mar 27, 2025 at 2:29 PM Amit Kapila wrote: > > > > > > > > > > > On Tue, Mar 25

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Zhijie Hou (Fujitsu)
On Wed, Apr 2, 2025 at 3:45 PM Masahiko Sawada wrote: Hi, > > On Mon, Mar 31, 2025 at 4:3 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Thu, Mar 27, 2025 at 2:29 PM Amit Kapila wrote: > > > > > > > > On Tue, Mar 25, 2025 at 12:1 PM Amit Kapila > > > > > > wrote: > > > > > > > > On Tue, Mar 25, 2

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-02 Thread Masahiko Sawada
On Mon, Mar 31, 2025 at 4:34 AM Zhijie Hou (Fujitsu) wrote: > > On Thu, Mar 27, 2025 at 2:29 PM Amit Kapila wrote: > > > > > On Tue, Mar 25, 2025 at 12:1 PM Amit Kapila > > wrote: > > > > > > On Tue, Mar 25, 2025 at 11:05 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > Hi, > > > > > > > > W

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-01 Thread Zhijie Hou (Fujitsu)
On Wed, Apr 2, 2025 at 12:41 PM Amit Kapila wrote: > > On Tue, Apr 1, 2025 at 4:28 PM Zhijie Hou (Fujitsu) > > wrote: > > > > Here is the V3 patch set which addressed all the comments. > > > > Comment 0n 0001 > NULL for logical slots where > + two_phase is false and physical slots. > +

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-01 Thread Amit Kapila
On Tue, Apr 1, 2025 at 4:28 PM Zhijie Hou (Fujitsu) wrote: > > Here is the V3 patch set which addressed all the comments. > Comment 0n 0001 NULL for logical slots where + two_phase is false and physical slots. + change above to: NULL for logical slots where two_phase is false and for

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-01 Thread Amit Kapila
On Mon, Mar 31, 2025 at 5:04 PM Zhijie Hou (Fujitsu) wrote: > > Thanks for the comments, they have been addressed in V2. > In the PG17 patch, we only have a check preventing failover and two_phase in CreateSubscription. Don't we need a similar check for AlterSubscription? Apart from the above, I

RE: Fix slot synchronization with two_phase decoding enabled

2025-04-01 Thread Zhijie Hou (Fujitsu)
On Tue, Apr 1, 2025 at 2:09 PM Amit Kapila wrote: > > On Mon, Mar 31, 2025 at 5:0 PM Zhijie Hou (Fujitsu) > wrote: > > > > Thanks for the comments, they have been addressed in V2. > > > > In the PG17 patch, we only have a check preventing failover and > two_phase in CreateSubscription. Don't

Re: Fix slot synchronization with two_phase decoding enabled

2025-03-31 Thread Amit Kapila
On Mon, Mar 31, 2025 at 5:04 PM Zhijie Hou (Fujitsu) wrote: > > On Thu, Mar 27, 2025 at 2:29 PM Amit Kapila wrote: > > > > > I suspect that this can happen in PG17 as well, but I need to think > > more about it to make a reproducible test case. > > After further analysis, I was able to reproduce t

Re: Fix slot synchronization with two_phase decoding enabled

2025-03-26 Thread Amit Kapila
On Tue, Mar 25, 2025 at 12:14 PM Amit Kapila wrote: > > On Tue, Mar 25, 2025 at 11:05 AM Zhijie Hou (Fujitsu) > wrote: > > > > Hi, > > > > When testing the slot synchronization with logical replication slots that > > enabled two_phase decoding, I found that transactions prepared before > > two-p

Re: Fix slot synchronization with two_phase decoding enabled

2025-03-24 Thread Amit Kapila
On Tue, Mar 25, 2025 at 11:05 AM Zhijie Hou (Fujitsu) wrote: > > Hi, > > When testing the slot synchronization with logical replication slots that > enabled two_phase decoding, I found that transactions prepared before > two-phase > decoding is enabled may fail to replicate to the subscriber afte