Re: Fix slot synchronization with two_phase decoding enabled

2025-04-28 Thread Nisha Moond
On Mon, Apr 28, 2025 at 12:04 PM shveta malik wrote: > > On Fri, Apr 25, 2025 at 5:53 PM Nisha Moond wrote: > > > > Please find the attached v8 patch with above comments addressed. > > This version includes the documentation updates suggested by > > Sawada-sa

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-25 Thread Nisha Moond
On Thu, Apr 24, 2025 at 3:57 PM shveta malik wrote: > > On Thu, Apr 24, 2025 at 2:54 PM Nisha Moond wrote: > > > > Please find the updated patch for Approach 3, which implements the > > idea suggested by Swada-san above. > > > > Thank You for the patch. > &

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-24 Thread Nisha Moond
On Thu, Apr 24, 2025 at 12:28 PM Amit Kapila wrote: > > On Wed, Apr 23, 2025 at 11:04 PM Masahiko Sawada > wrote: > > > > On Tue, Apr 22, 2025 at 3:00 AM Amit Kapila wrote: > > > > > > On Mon, Apr 21, 2025 at 8:44 AM Zhijie Hou (Fujitsu) > > > wrote: > > > > > > > > On Sat, Apr 19, 2025 at 2:1

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-23 Thread Nisha Moond
On Tue, Apr 22, 2025 at 3:23 PM shveta malik wrote: > > On Fri, Apr 11, 2025 at 1:03 PM Nisha Moond wrote: > > > > > > Patch "v5_aprch_3-0001" implements the above Approach 3. > > > > Thanks Hou-san for implementing approach-2 and providing th

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-22 Thread Nisha Moond
On Mon, Apr 21, 2025 at 9:26 PM Nathan Bossart wrote: > > Apparently replication origins in tests are supposed to be prefixed with > "regress_". Here is a new patch with that fixed. > Hi Nathan, Thanks for the patch! I tested it for functionality and here are a few comments: 1) While testing, I

Re: Conflict detection for update_deleted in logical replication

2025-04-16 Thread Nisha Moond
On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu) wrote: > > Here's a rebased version of the patch series. > Thanks for the patches. While testing the GUC "max_conflict_retention_duration", I noticed a behavior that seems to bypass its intended purpose. On Pub, if a txn is stuck in the COMMI

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-11 Thread Nisha Moond
HI, Please find the patches attached for all three approaches. On Wed, Apr 9, 2025 at 10:45 AM Zhijie Hou (Fujitsu) wrote: > > On Thu, Apr 3, 2025 at 3:16 PM Zhijie Hou (Fujitsu) wrote: > > > > On Thu, Apr 3, 2025 at 1:38 PM Masahiko Sawada wrote: > > > > > > > > On Wed, Apr 2, 2025 at 7:58 PM A

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-04-05 Thread Nisha Moond
On Thu, Mar 20, 2025 at 5:23 PM Zhijie Hou (Fujitsu) wrote: > > On Thu, Mar 20, 2025 at 3:06 PM Nisha Moond wrote: > > > > Attached is v6 patch with above comments addressed. > > Thanks updating the patch. I have some comments: > > 1. > > The naming style o

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-04-04 Thread Nisha Moond
On Fri, Mar 21, 2025 at 3:38 PM Amit Kapila wrote: > > On Fri, Mar 21, 2025 at 1:48 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Fri, Mar 21, 2025 at 12:50 PM Nisha Moond wrote: > > > > > Thanks, Hou-san, for the review and fix patches. I’ve incorporated > &

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-20 Thread Nisha Moond
On Wed, Mar 19, 2025 at 4:18 PM Amit Kapila wrote: > > On Wed, Mar 19, 2025 at 11:11 AM Nisha Moond wrote: > > > > Please find the attached v5-0001 patch without the stats part. > > > > Review: > === > 1. > + foreach_ptr(TupleTableSlot, slot, conflic

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-18 Thread Nisha Moond
On Mon, Mar 17, 2025 at 3:20 PM Amit Kapila wrote: > > On Thu, Mar 13, 2025 at 4:30 PM Nisha Moond wrote: > > > > Attached is the v4 patch (test case changes only). > > > > Comments: > = > 1. > + /* > + * Report an INSERT_EXISTS or U

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-14 Thread Nisha Moond
Hi Shubham, Here are a few comments for the v12 patch. doc/src/sgml/ref/pg_createsubscriber.sgml : 1. + + For all source server non-template databases create subscriptions for + create subscriptions for databases with the same names on the + target server. is “create sub

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-13 Thread Nisha Moond
false positives. Note: I have broken down the full error detail message check into multiple checks to avoid very long lines in the file. I'll see if there's a better way to compare the full error detail in a single check. Attached is the v4 patch (test case changes only). -- Thanks, N

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-12 Thread Nisha Moond
how that pans out. > Meanwhile, I had a couple more replies below. > > On Tue, Feb 25, 2025 at 8:37 PM Nisha Moond wrote: > > > > On Mon, Feb 24, 2025 at 7:39 AM Peter Smith wrote: > > > > > > Hi Nisha. > > > > > > Some review comments for pat

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-11 Thread Nisha Moond
On Mon, Mar 10, 2025 at 4:31 PM Shubham Khanna wrote: > > > 2) This part of code has another bug: > > "dbinfos.dbinfo->made_publication = false;" incorrectly modifies > > dbinfo[0], even if the failure occurs in other databases (dbinfo[1], > > etc.). Since cleanup_objects_atexit() checks made_pub

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-11 Thread Nisha Moond
On Tue, Mar 11, 2025 at 11:10 AM Dilip Kumar wrote: > > On Thu, Feb 20, 2025 at 5:01 PM Nisha Moond wrote: > > > > Hi Hackers, > > (CCing people involved in related discussions) > > > > I am starting this thread to propose a new conflict detection type

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-10 Thread Nisha Moond
On Mon, Mar 10, 2025 at 12:11 PM Shubham Khanna wrote: > > On Thu, Mar 6, 2025 at 9:27 AM Peter Smith wrote: > > > > 6. > > - dbinfo->pubname, dbinfo->dbname, PQresultErrorMessage(res)); > > - dbinfo->made_publication = false; /* don't try again. */ > > + pubname, dbname, PQresultErrorMessage(res

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-05 Thread Nisha Moond
On Tue, Mar 4, 2025 at 8:05 PM Shubham Khanna wrote: > > The attached Patch contains the suggested changes. > Hi Shubham, Here are few comments for 040_pg_createsubscriber.pl 1) +# Run pg_createsubscriber on node S using '--cleanup-existing-publications'. +# --verbose is used twice to show more

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-04 Thread Nisha Moond
On Tue, Mar 4, 2025 at 10:30 AM Shubham Khanna wrote: > > Fixed the suggested changes. The attached patch contains the required changes. > Hi Shubham, Thanks for the patch! I tested its functionality and didn't find any issues so far. I'll continue with further testing. Here are some review comm

Race condition in replication slot usage introduced by commit f41d846

2025-02-26 Thread Nisha Moond
Hi, Commit f41d846 [1] introduced a race condition that allows a process to acquire and continue using an invalidated replication slot, leading to unexpected behavior. Example Scenario: Suppose there is an orphaned logical slot that will be invalidated on the next checkpoint. Consider the followi

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-02-25 Thread Nisha Moond
On Mon, Feb 24, 2025 at 7:39 AM Peter Smith wrote: > > Hi Nisha. > > Some review comments for patch v1-0001 > > == > GENERAL > > 1. > This may be a basic/naive question, but it is unclear to me why we > care about the stats of confl_multiple_unique_conflicts? > > I can understand it would be u

Conflict detection for multiple_unique_conflicts in logical replication

2025-02-20 Thread Nisha Moond
Hi Hackers, (CCing people involved in related discussions) I am starting this thread to propose a new conflict detection type "multiple_unique_conflicts" that identifies when an incoming row during logical replication violates more than one UNIQUE constraint. If multiple constraints (such as the p

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-16 Thread Nisha Moond
On Mon, Feb 17, 2025 at 11:29 AM Amit Kapila wrote: > > On Fri, Feb 14, 2025 at 5:30 PM Nisha Moond wrote: > > > > Here is a summary of changes in v78: > > > > A few minor comments: > 1. > Slots that appear idle due to a disrupted connection between > +

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-14 Thread Nisha Moond
Please find the updated v78 patches after a few off-list review rounds. Here is a summary of changes in v78: patch-001: - Fixed bugs reported by Hou-san and Peter in [1] and [2]. - Fixed a race condition reported by Hou-san off-list, which could lead to an assert failure. This failure happens when

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-11 Thread Nisha Moond
On Tue, Feb 11, 2025 at 11:42 AM Zhijie Hou (Fujitsu) wrote: > > On Monday, February 10, 2025 8:03 PM Nisha Moond > wrote: > > > > On Sat, Feb 8, 2025 at 12:28 PM Zhijie Hou (Fujitsu) > > > > wrote: > > > > > > > > 3. > &g

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-11 Thread Nisha Moond
On Tue, Feb 11, 2025 at 8:49 AM Peter Smith wrote: > > Hi Nisha. > > Some review comments about v74-0001 > > == > src/backend/replication/slot.c > > 1. > /* Maximum number of invalidation causes */ > -#define RS_INVAL_MAX_CAUSES RS_INVAL_WAL_LEVEL > - > -StaticAssertDecl(lengthof(SlotInvalida

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Nisha Moond
On Mon, Feb 10, 2025 at 6:12 PM vignesh C wrote: > > On Mon, 10 Feb 2025 at 17:33, Nisha Moond wrote: > > > > Here are the v73 patches incorporating the comments above and the > > subsequent comments from [1]. > > - patch 002 is rebased on 001 with no new chang

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Nisha Moond
On Sat, Feb 8, 2025 at 12:28 PM Zhijie Hou (Fujitsu) wrote: > > On Friday, February 7, 2025 9:06 PM Nisha Moond > wrote: > > > > Attached v72 patches, addressed the above comments as well as Vignesh's > > comments in [2]. > > - There are no new changes in

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-07 Thread Nisha Moond
On Fri, Feb 7, 2025 at 8:00 AM Peter Smith wrote: > > Hi Nisha, > > Some review comments for v71-0001. > > == > src/backend/replication/slot.c > > SlotInvalidationCauses[] > > 2. > [RS_INVAL_WAL_REMOVED] = "wal_removed", > [RS_INVAL_HORIZON] = "rows_removed", > [RS_INVAL_WAL_LEVEL] = "wa

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-06 Thread Nisha Moond
On Thu, Feb 6, 2025 at 10:17 AM Amit Kapila wrote: > > On Thu, Feb 6, 2025 at 8:02 AM Nisha Moond wrote: > > > > On Wed, Feb 5, 2025 at 2:42 PM Amit Kapila wrote: > > > > > > Would it address your concern if we write the actual idle duration > > &g

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-05 Thread Nisha Moond
On Wed, Feb 5, 2025 at 2:42 PM Amit Kapila wrote: > > On Wed, Feb 5, 2025 at 10:30 AM vignesh C wrote: > > > > On Tue, 4 Feb 2025 at 19:56, Nisha Moond wrote: > > > > > > Here is v69 patch set addressing above and Kuroda-san's comments in [1]. > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-05 Thread Nisha Moond
On Wed, Feb 5, 2025 at 12:58 PM Peter Smith wrote: > > Hi Nisha, > > Some review comments for the patch v69-0002. > > == > .../t/044_invalidate_inactive_slots.pl > > 2. > +if ($ENV{enable_injection_points} ne 'yes') > +{ > + plan skip_all => 'Injection points not supported by this build'; > +}

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-05 Thread Nisha Moond
On Wed, Feb 5, 2025 at 10:30 AM vignesh C wrote: > > On Tue, 4 Feb 2025 at 19:56, Nisha Moond wrote: > > > > Here is v69 patch set addressing above and Kuroda-san's comments in [1]. > > 2) Here we have mentioned about invalidation happens only for a) >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-04 Thread Nisha Moond
On Tue, Feb 4, 2025 at 4:42 PM vignesh C wrote: > > On Tue, 4 Feb 2025 at 15:58, Nisha Moond wrote: > > > > Here are the v68 patches, incorporating above as well as comments from [1]. > > > Few comments: > 1) Let's call Timest

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-04 Thread Nisha Moond
On Tue, Feb 4, 2025 at 10:45 AM Amit Kapila wrote: > > On Mon, Feb 3, 2025 at 6:35 PM Shlok Kyal wrote: > > > > I reviewed the v66 patch. I have few comments: > > > > 1. I also feel the default value should be set to '0' as suggested by > > Vignesh in 1st point of [1]. > > > > +1. This will ensur

Re: Avoid updating inactive_since for invalid replication slots

2025-02-03 Thread Nisha Moond
On Tue, Feb 4, 2025 at 9:33 AM Zhijie Hou (Fujitsu) wrote: > > On Monday, February 3, 2025 8:03 PM Nisha Moond > wrote: > > > > Hi Hackers, > > (CC people involved in the earlier discussion) > > > > Right now, it is possible for the 'inactive_since&#x

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-03 Thread Nisha Moond
from [1] and [2]. - No change in patch-003 since the last version. [1] https://www.postgresql.org/message-id/CALDaNm0FS%2BFqQk2dadiJFCMM_MhKROMsJUb%3Db8wtRH6isScQsQ%40mail.gmail.com [2] https://www.postgresql.org/message-id/CAHut%2BPs_6%2BNBOt%2BKpQQaBG2R3T-FLS93TbUC27uzyDMu%3D37n-Q%40mail.gmail.com -- Thanks, Ni

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-03 Thread Nisha Moond
On Mon, Feb 3, 2025 at 2:55 PM Amit Kapila wrote: > > On Fri, Jan 31, 2025 at 5:50 PM Nisha Moond wrote: > > > > Please find the attached v66 patch set. The base patch(v65-001) is > > committed now, so I have rebased the patches. > > > > * > &

Avoid updating inactive_since for invalid replication slots

2025-02-03 Thread Nisha Moond
Hi Hackers, (CC people involved in the earlier discussion) Right now, it is possible for the 'inactive_since' value of an invalid replication slot to be updated multiple times, which is unexpected behavior. As suggested in the ongoing thread [1], this patch introduces a new dedicated function to u

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-31 Thread Nisha Moond
On Fri, Jan 31, 2025 at 2:32 PM Amit Kapila wrote: > > On Fri, Jan 31, 2025 at 10:40 AM Peter Smith wrote: > > > > == > > src/backend/replication/slot.c > > > > ReportSlotInvalidation: > > > > 1. > > + > > + case RS_INVAL_IDLE_TIMEOUT: > > + Assert(inactive_since > 0); > > + /* translator: se

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-30 Thread Nisha Moond
On Tue, Jan 28, 2025 at 5:28 PM Nisha Moond wrote: > > On Tue, Jan 28, 2025 at 3:26 PM Amit Kapila wrote: > > > > On Mon, Dec 30, 2024 at 11:05 AM Peter Smith wrote: > > > > > > I think we are often too quick to throw out perfectly good tests. > >

Re: Improve error handling for invalid slots and ensure a same 'inactive_since' time for inactive slots

2025-01-29 Thread Nisha Moond
On Thu, Jan 30, 2025 at 9:56 AM Amit Kapila wrote: > > On Thu, Jan 30, 2025 at 5:23 AM Peter Smith wrote: > > > > My understanding was that the purpose of this patch was not anything > > to do with "optimisations" per se, but rather it was (like the > > $SUBJECT says) to ensure the *same* 'active

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-28 Thread Nisha Moond
On Tue, Jan 28, 2025 at 3:26 PM Amit Kapila wrote: > > On Mon, Dec 30, 2024 at 11:05 AM Peter Smith wrote: > > > > I think we are often too quick to throw out perfectly good tests. > > Citing that some similar GUCs don't do testing as a reason to skip > > them just seems to me like an example of

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-28 Thread Nisha Moond
On Mon, Jan 27, 2025 at 4:20 PM Amit Kapila wrote: > > On Mon, Jan 27, 2025 at 11:00 AM Nisha Moond wrote: > > > > I discussed the above comments further with Peter off-list, and here > > are the v63 patches with the following changes: > > patch-001: The Assert

Improve error handling for invalid slots and ensure a same 'inactive_since' time for inactive slots

2025-01-28 Thread Nisha Moond
Hello Hackers, (CC people involved in the earlier discussion) While implementing slot invalidation based on inactive(idle) timeout (see [1]), several general optimizations and improvements were identified. This thread is a spin-off from [1], intended to address these optimizations separately from

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-26 Thread Nisha Moond
On Wed, Jan 22, 2025 at 10:49 AM Nisha Moond wrote: > > On Tue, Jan 21, 2025 at 8:22 AM Peter Smith wrote: > > > > Some review comments for patch v61-0001. > > > > == > > src/backend/replication/slot.c > > > > 2. > > + /* > > + * T

Re: Conflict detection for update_deleted in logical replication

2025-01-22 Thread Nisha Moond
On Sat, Jan 18, 2025 at 9:15 AM Zhijie Hou (Fujitsu) wrote: > > > Here is the V24 patch set. I modified 0004 patch to implement the slot > Invalidation part. Since the automatic recovery could be an optimization and > the discussion is in progress, I didn't implement that part. Few comments for p

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-21 Thread Nisha Moond
On Tue, Jan 21, 2025 at 8:22 AM Peter Smith wrote: > > Some review comments for patch v61-0001. > > == > src/backend/replication/slot.c > > InvalidatePossiblyObsoleteSlot: > > 1. > /* > * Check if the slot needs to be invalidated. If it needs to be > - * invalidated, and is not currently a

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-21 Thread Nisha Moond
On Tue, Jan 21, 2025 at 8:26 AM Peter Smith wrote: > > Some review comments for patch v61-0002 > > == > src/backend/replication/slot.c > > 1. > * Whether a slot needs to be invalidated depends on the cause. A slot is > - * removed if it: > + * invalidated if it: > * - RS_INVAL_WAL_REMOVED:

Re: Conflict detection for update_deleted in logical replication

2025-01-20 Thread Nisha Moond
On Wed, Jan 8, 2025 at 4:03 PM Masahiko Sawada wrote: > > On Wed, Jan 8, 2025 at 1:53 AM Amit Kapila wrote: > > On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada > > wrote: > > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond > > > wrote: > > > > He

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-20 Thread Nisha Moond
On Fri, Jan 17, 2025 at 6:50 PM Shlok Kyal wrote: > > Hi Nisha, > > Thanks for providing an updated patch. I have tested the patch and ran > some tests. The patch works fine. I have few comments: > Thanks for your review. Attached are the v61 patches. I've addressed the comments and rebased patc

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-15 Thread Nisha Moond
On Mon, Jan 13, 2025 at 12:22 PM vignesh C wrote: > > On Thu, 2 Jan 2025 at 15:57, Nisha Moond wrote: > > > > Thank you for your feedback! Please find the v59 patch set addressing > > all the comments. > > Note: There are no new changes in patch-0001. > > Few

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-15 Thread Nisha Moond
On Wed, Jan 15, 2025 at 11:37 AM Shlok Kyal wrote: > > On Thu, 2 Jan 2025 at 15:57, Nisha Moond wrote: > > > > On Thu, Jan 2, 2025 at 8:16 AM Peter Smith wrote: > > > > > > Hi Nisha, > > > > > > Here are some minor review comments for pat

Re: Conflict detection for update_deleted in logical replication

2025-01-13 Thread Nisha Moond
Here are the performance test results and analysis with the recent patches. Test Setup: - Created Pub and Sub nodes with logical replication and below configurations. autovacuum_naptime = '30s' shared_buffers = '30GB' max_wal_size = 20GB min_wal_size = 10GB track_commit_timestamp =

Re: Conflict detection for update_deleted in logical replication

2025-01-10 Thread Nisha Moond
On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote: > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote: > > [3] Test with pgbench run on both publisher and subscriber. > > > > Test setup: > > - Tests performed on pgHead + v16 patches > > -

Re: Conflict detection for update_deleted in logical replication

2025-01-07 Thread Nisha Moond
On Tue, Jan 7, 2025 at 6:04 PM Zhijie Hou (Fujitsu) wrote: > > > Attached the V19 patch which addressed comments in [1][2][3][4][5][6][7]. > Here are a couple of initial review comments on v19 patch set: 1) The subscription option 'retain_conflict_info' remains set to "true" for a subscription e

Re: Conflict detection for update_deleted in logical replication

2025-01-06 Thread Nisha Moond
On Mon, Jan 6, 2025 at 4:52 PM Zhijie Hou (Fujitsu) wrote: > > On Friday, January 3, 2025 2:36 PM Masahiko Sawada > wrote: > > Hi, > > > > > I have one comment on the 0001 patch: > > Thanks for the comments! > > > > > + /* > > +* The changes made by this and later transactions are

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-02 Thread Nisha Moond
nks, Nisha From 8154e2baee6fcf348524899c1f8b643a1e3564fc Mon Sep 17 00:00:00 2001 From: Nisha Moond Date: Mon, 18 Nov 2024 16:13:26 +0530 Subject: [PATCH v59 1/2] Enhance replication slot error handling, slot invalidation, and inactive_since setting logic In ReplicationSlotAcquire(), raise an error for inval

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-01-02 Thread Nisha Moond
On Thu, Jan 2, 2025 at 5:44 AM Peter Smith wrote: > > Hi Nisha. > > My review comments for patch v58-0001. > > == > src/backend/replication/slot.c > > InvalidatePossiblyObsoleteSlot: > > 1. > /* > - * If the slot can be acquired, do so and mark it invalidated > - * immediately. Otherwise we

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-30 Thread Nisha Moond
On Fri, Dec 27, 2024 at 9:22 AM vignesh C wrote: > > > Few comments: > 1) We have disabled the similar configuration max_slot_wal_keep_size > by setting to -1, as this GUC also is in similar lines, should we > disable this and let the user configure it? > +/* > + * Invalidate replication slots tha

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-30 Thread Nisha Moond
On Mon, Dec 30, 2024 at 11:05 AM Peter Smith wrote: > > On Tue, Dec 24, 2024 at 10:37 PM Nisha Moond wrote: > > > > On Fri, Dec 20, 2024 at 3:12 PM Amit Kapila wrote: > >> > >> On Mon, Dec 16, 2024 at 4:10 PM Nisha Moond > >> wrote: > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-24 Thread Nisha Moond
On Fri, Dec 20, 2024 at 3:12 PM Amit Kapila wrote: > On Mon, Dec 16, 2024 at 4:10 PM Nisha Moond > wrote: > > > > Here is the v56 patch set with the above comments incorporated. > > > > Review comments: > === > 1. > + { > + {&

Re: Conflict detection for update_deleted in logical replication

2024-12-19 Thread Nisha Moond
Here is further performance test analysis with v16 patch-set. In the test scenarios already shared on -hackers [1], where pgbench was run only on the publisher node in a pub-sub setup, no performance degradation was observed on either node. In contrast, when pgbench was run only on the subscri

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-16 Thread Nisha Moond
On Mon, Dec 16, 2024 at 9:58 AM Peter Smith wrote: > > Hi Nisha. > > Thanks for the v55* patches. > > I have no comments for patch v55-0001. > > I have only 1 comment for patch v55-0002 regarding some remaining > nitpicks (below) about the consistency of phrases. > > == > > SUGGESTIONS: > > Do

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-13 Thread Nisha Moond
On Wed, Dec 11, 2024 at 8:14 AM Peter Smith wrote: > > Hi Nisha. > > Here are some review comments for patch v54-0002. > == > src/test/recovery/t/043_invalidate_inactive_slots.pl > > 5. > +# Wait for slot to first become idle and then get invalidated > +sub wait_for_slot_invalidation > +{ > +

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-13 Thread Nisha Moond
On Thu, Dec 12, 2024 at 9:42 AM vignesh C wrote: > > > Now that we support idle_replication_slot_timeout in milliseconds, we > can set this value from 1s to 1ms or 10millseconds and change sleep to > usleep, this will bring down the test execution time significantly: +1 v55 implements the test us

Re: Conflict detection for update_deleted in logical replication

2024-12-10 Thread Nisha Moond
Here are the test steps and analysis for epoch-related handling (Tested with v15 Patch-Set). In the 'update_deleted' detection design, the launcher process compares XIDs to track minimum XIDs, and the apply workers maintain the oldest running XIDs. The launcher also requests publisher status at re

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-10 Thread Nisha Moond
2Ct39yg%40mail.gmail.com [3] https://www.postgresql.org/message-id/CAHut%2BPtHbYNxPvtMfs7jARbsVcFXL1%3DC9SO3Q93NgVDgbKN7LQ%40mail.gmail.com -- Thanks, Nisha From 713871d8cda02f2b70c63983fc49dede3097f016 Mon Sep 17 00:00:00 2001 From: Nisha Moond Date: Mon, 18 Nov 2024 16:13:26 +0530 Subject: [PATCH v54

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-04 Thread Nisha Moond
M79f34LLBGq4UeRuZ1URWP6JNZtdN2khYPrLc1YqrQ%40mail.gmail.com [2] https://www.postgresql.org/message-id/TYAPR01MB5692B7687EE7981AA91BA5B9F5362%40TYAPR01MB5692.jpnprd01.prod.outlook.com -- Thanks, Nisha From b7e55950ae7577dfc8ae8893157d6b6124fdba01 Mon Sep 17 00:00:00 2001 From: Nisha Moond Date: Mon, 18 N

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-29 Thread Nisha Moond
On Thu, Nov 28, 2024 at 5:20 AM Peter Smith wrote: > > Hi Nisha. Here are some review comments for patch v51-0002. > > == > src/backend/replication/slot.c > > ReplicationSlotAcquire: > > 2. > GENERAL. > > This just is a question/idea. It may not be feasible to change. It > seems like there is

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-29 Thread Nisha Moond
On Thu, Nov 28, 2024 at 1:29 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Nisha, > > > > > Attached v51 patch-set addressing all comments in [1] and [2]. > > > > Thanks for working on the feature! I've stated to review the patch. > Here are my comments - sorry if there are something which have alrea

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-29 Thread Nisha Moond
On Tue, Nov 19, 2024 at 12:47 PM Nisha Moond wrote: > > On Thu, Nov 14, 2024 at 5:29 AM Peter Smith wrote: > > > > > > 12. > > /* > > - * If the slot can be acquired, do so and mark it invalidated > > - * immediately. Otherwise we'll sign

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-29 Thread Nisha Moond
On Thu, Nov 28, 2024 at 2:44 PM vignesh C wrote: > > > > > We are setting inactive_since when the replication slot is released. > > We are marking the slot as inactive only if it has been released. > > However, there's a scenario where the network connection between the > > publisher and subscribe

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-27 Thread Nisha Moond
On Wed, Nov 27, 2024 at 8:39 AM Peter Smith wrote: > > Hi Nisha, > > Here are some review comments for the patch v50-0002. > > == > src/backend/replication/slot.c > > InvalidatePossiblyObsoleteSlot: > > 1. > + if (now && > + TimestampDifferenceExceeds(s->inactive_since, now, > +replication

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-21 Thread Nisha Moond
On Wed, Nov 20, 2024 at 1:29 PM vignesh C wrote: > > On Tue, 19 Nov 2024 at 12:43, Nisha Moond wrote: > > > > Attached is the v49 patch set: > > - Fixed the bug reported in [1]. > > - Addressed comments in [2] and [3]. > > > > I've split the pat

Re: Conflict detection for update_deleted in logical replication

2024-11-20 Thread Nisha Moond
On Thu, Nov 14, 2024 at 8:24 AM Zhijie Hou (Fujitsu) wrote: > > Attach the V9 patch set which addressed above comments. > Reviewed v9 patch-set and here are my comments for below changes: @@ -1175,10 +1189,29 @@ ApplyLauncherMain(Datum main_arg) long elapsed; if (!sub->enabled) + { + can_ad

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-18 Thread Nisha Moond
On Thu, Nov 14, 2024 at 5:29 AM Peter Smith wrote: > > Hi Nisha. > > Thanks for the recent patch updates. Here are my review comments for > the latest patch v48-0001. > Thank you for the review. Comments are addressed in v49 version. Below is my response to comments that may require further discu

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-18 Thread Nisha Moond
On Thu, Nov 14, 2024 at 9:14 AM vignesh C wrote: > > On Wed, 13 Nov 2024 at 15:00, Nisha Moond wrote: > > > > Please find the v48 patch attached. > > > > On Thu, Sep 19, 2024 at 9:40 AM shveta malik wrote: > > > > > > When we promote hot

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-18 Thread Nisha Moond
Attached is the v49 patch set: - Fixed the bug reported in [1]. - Addressed comments in [2] and [3]. I've split the patch into two, implementing the suggested idea in comment #5 of [2] separately in 001: Patch-001: Adds additional error reports (for all invalidation types) in ReplicationSlotAcqui

Re: DOCS - pg_replication_slot . Fix the 'inactive_since' description

2024-11-18 Thread Nisha Moond
On Fri, Nov 15, 2024 at 3:01 PM Amit Kapila wrote: > > On Thu, Nov 14, 2024 at 12:12 PM Nisha Moond wrote: > > > > On Thu, Oct 31, 2024 at 11:05 PM Bruce Momjian wrote: > > > > > > > > > Yes, all good suggestions, updated patch attached. >

Re: DOCS - pg_replication_slot . Fix the 'inactive_since' description

2024-11-13 Thread Nisha Moond
On Thu, Oct 31, 2024 at 11:05 PM Bruce Momjian wrote: > > > Yes, all good suggestions, updated patch attached. > Few comments for the changes under "inactive_since" description: +The time when slot synchronization (see ) +was most recently stopped. NULL if the slot +has

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-13 Thread Nisha Moond
Please find the v48 patch attached. On Thu, Sep 19, 2024 at 9:40 AM shveta malik wrote: > > When we promote hot standby with synced logical slots to become new > primary, the logical slots are never invalidated with > 'inactive_timeout' on new primary. It seems the check in > SlotInactiveTimeout

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-13 Thread Nisha Moond
On Wed, Sep 18, 2024 at 12:22 PM shveta malik wrote: > > On Mon, Sep 16, 2024 at 3:31 PM Bharath Rupireddy > wrote: > > > > Hi, > > > > > > Please find the attached v46 patch having changes for the above review > > comments and your test review comments and Shveta's review comments. > > > > Thank

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-11 Thread Nisha Moond
On Wed, Sep 18, 2024 at 3:31 PM shveta malik wrote: > > On Wed, Sep 18, 2024 at 2:49 PM shveta malik wrote: > > > > > > Please find the attached v46 patch having changes for the above review > > > > comments and your test review comments and Shveta's review comments. > > > > > > When the synced s

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-11-07 Thread Nisha Moond
On Mon, Sep 16, 2024 at 3:31 PM Bharath Rupireddy wrote: > > Please find the attached v46 patch having changes for the above review > comments and your test review comments and Shveta's review comments. > Hi, I’ve reviewed this thread and am interested in working on the remaining tasks and commen

Re: Conflict detection for update_deleted in logical replication

2024-10-25 Thread Nisha Moond
> Here is the V5 patch set which addressed above comments. > Here are a couple of comments on v5 patch-set - 1) In FindMostRecentlyDeletedTupleInfo(), + /* Try to find the tuple */ + while (index_getnext_slot(scan, ForwardScanDirection, scanslot)) + { + Assert(tuples_equal(scanslot, searchslot, e

Re: Conflict Detection and Resolution

2024-10-08 Thread Nisha Moond
On Tue, Oct 1, 2024 at 9:48 AM shveta malik wrote: > > On Mon, Sep 30, 2024 at 2:55 PM Peter Smith wrote: > > > > On Mon, Sep 30, 2024 at 4:29 PM shveta malik wrote: > > > > > > On Mon, Sep 30, 2024 at 11:04 AM Peter Smith > > > wrote: > > > > > > > > On Mon, Sep 30, 2024 at 2:27 PM shveta mal

Re: Conflict Detection and Resolution

2024-10-08 Thread Nisha Moond
On Mon, Sep 30, 2024 at 11:59 AM shveta malik wrote: > > On Mon, Sep 30, 2024 at 11:04 AM Peter Smith wrote: > > > > On Mon, Sep 30, 2024 at 2:27 PM shveta malik wrote: > > > > > > On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > > ... > > > > > > > > 13. General - ordering of conflict_type

Re: Conflict Detection and Resolution

2024-10-08 Thread Nisha Moond
On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > > Here are some review comments for v14-0001. > ~~~ > 7. > +ALTER SUBSCRIPTION name > RESET CONFLICT RESOLVER FOR ( class="parameter">conflict_type) > > I can see that this matches the implementation, but I was wondering > why don't you permit r

Re: Clock-skew management in logical replication

2024-09-24 Thread Nisha Moond
On Mon, Sep 23, 2024 at 4:00 PM Nisha Moond wrote: > > On Fri, Sep 20, 2024 at 7:51 PM Tom Lane wrote: > > > > Nisha Moond writes: > > > While considering the implementation of timestamp-based conflict > > > resolution (last_update_wins) in logical repl

Re: Clock-skew management in logical replication

2024-09-23 Thread Nisha Moond
On Fri, Sep 20, 2024 at 7:51 PM Tom Lane wrote: > > Nisha Moond writes: > > While considering the implementation of timestamp-based conflict > > resolution (last_update_wins) in logical replication (see [1]), there > > was a feedback at [2] and the discussion on

Clock-skew management in logical replication

2024-09-20 Thread Nisha Moond
Hello Hackers, (CC people involved in the earlier discussion) While considering the implementation of timestamp-based conflict resolution (last_update_wins) in logical replication (see [1]), there was a feedback at [2] and the discussion on whether or not to manage clock-skew at database level. We

Re: Conflict Detection and Resolution

2024-09-19 Thread Nisha Moond
On Fri, Sep 20, 2024 at 8:40 AM Nisha Moond wrote: > > On Wed, Sep 18, 2024 at 10:46 AM vignesh C wrote: > > > > On Thu, 12 Sept 2024 at 14:03, Ajin Cherian wrote: > > > > > > On Tue, Sep 3, 2024 at 7:42 PM vignesh C wrote: > > > > &

Re: Conflict Detection and Resolution

2024-09-19 Thread Nisha Moond
On Fri, Sep 20, 2024 at 8:40 AM Nisha Moond wrote: > > On Wed, Sep 18, 2024 at 10:46 AM vignesh C wrote: > > > > On Thu, 12 Sept 2024 at 14:03, Ajin Cherian wrote: > > > > > > On Tue, Sep 3, 2024 at 7:42 PM vignesh C wrote: > > > > &

Re: Conflict Detection and Resolution

2024-09-09 Thread Nisha Moond
On Fri, Sep 6, 2024 at 2:05 PM Ajin Cherian wrote: > > > > On Thu, Aug 29, 2024 at 2:50 PM shveta malik wrote: >> >> On Wed, Aug 28, 2024 at 4:07 PM shveta malik wrote: >> > >> > > On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: >> > > > >> > >> > The review is WIP. Please find a few comme

Re: Commit Timestamp and LSN Inversion issue

2024-09-08 Thread Nisha Moond
On Wed, Sep 4, 2024 at 12:23 PM shveta malik wrote: > > Hello hackers, > (Cc people involved in the earlier discussion) > > I would like to discuss the $Subject. > > While discussing Logical Replication's Conflict Detection and > Resolution (CDR) design in [1] , it came to our notice that the > c

Re: Conflict Detection and Resolution

2024-09-06 Thread Nisha Moond
On Thu, Aug 29, 2024 at 4:43 PM Amit Kapila wrote: > > On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > > > Please find issues which need some thoughts and approval for > > time-based resolution and clock-skew. > > > > 1) > > Time based conflict resolution and two phase transactions: > >

Re: Conflict Detection and Resolution

2024-08-29 Thread Nisha Moond
On Mon, Aug 26, 2024 at 2:23 PM Amit Kapila wrote: > > On Thu, Aug 22, 2024 at 3:45 PM shveta malik wrote: > > > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond > > wrote: > > > > > > The patches have been rebased on the latest pgHead following the merg

Re: Conflict Detection and Resolution

2024-08-29 Thread Nisha Moond
On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > On Thu, Aug 22, 2024 at 3:44 PM shveta malik wrote: > > > > > > For clock-skew and timestamp based resolution, if needed, I will post > > another email for the design items where suggestions are needed. > > > > Please find issues which need

Re: Conflict Detection and Resolution

2024-08-27 Thread Nisha Moond
On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > On Thu, Aug 22, 2024 at 3:44 PM shveta malik wrote: > > > > > > For clock-skew and timestamp based resolution, if needed, I will post > > another email for the design items where suggestions are needed. > > > > Please find issues which need

  1   2   >