Re: Conflict Detection and Resolution

2025-01-13 Thread Amit Kapila
On Mon, Jan 13, 2025 at 11:01 PM Michail Nikolaev wrote: > > Just curious - are any plans to continue to work on this project? > Yes, but our intention is to reach a consensus/conclusion about how to deal with update_delete conflicts. The topic is discussed in the CF entry [1]. If you want to he

Re: Conflict Detection and Resolution

2025-01-13 Thread Michail Nikolaev
Hello, everyone! Just curious - are any plans to continue to work on this project? Best regards, Mikhail.

Re: Conflict Detection and Resolution

2024-10-28 Thread Diego Fronza
Hello hackers, Hey I'm Diego and I do work for Percona and started to work on PostgreSQL and I would like to contribute to the project moving forward. I have been following this thread since the beginning, but due to my limited knowledge of the overall code structure, my first review of the provi

Re: Conflict Detection and Resolution

2024-10-20 Thread shveta malik
On Fri, Oct 18, 2024 at 4:30 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, October 9, 2024 2:34 PM shveta malik > wrote: > > > > On Wed, Oct 9, 2024 at 8:58 AM shveta malik > > wrote: > > > > > > On Tue, Oct 8, 2024 at 3:12 PM Nisha Moond > > wrote: > > > > > > > > > > > Please find few com

Re: Conflict Detection and Resolution

2024-10-20 Thread Michail Nikolaev
l', + 't/035_delete_missing_race.pl', + 't/036_update_missing_race.pl', 't/100_bugs.pl', ], }, diff --git a/src/test/subscription/t/035_delete_missing_race.pl b/src/test/subscription/t/035_delete_missing_race.pl new file mode 100644 ind

RE: Conflict Detection and Resolution

2024-10-18 Thread Zhijie Hou (Fujitsu)
On Wednesday, October 9, 2024 2:34 PM shveta malik wrote: > > On Wed, Oct 9, 2024 at 8:58 AM shveta malik > wrote: > > > > On Tue, Oct 8, 2024 at 3:12 PM Nisha Moond > wrote: > > > > > > > Please find few comments on v14-patch004: > > patch004: > 1) > GetConflictResolver currently errors out

Re: Conflict Detection and Resolution

2024-10-15 Thread Peter Smith
ou are planning to keep this implementation of the Map, then that -1 end of List marker is critical to the logic for the scanner part to know when to stop looking for valid resolvers. So, I think you should add another comment about that. ~~~ 4. +/* + * Report a warning about incomplete conf

Re: Conflict Detection and Resolution

2024-10-08 Thread shveta malik
On Wed, Oct 9, 2024 at 8:58 AM shveta malik wrote: > > On Tue, Oct 8, 2024 at 3:12 PM Nisha Moond wrote: > > > Please find few comments on v14-patch004: patch004: 1) GetConflictResolver currently errors out when the resolver is last_update_wins and track_commit_timestamp is disabled. It means e

Re: Conflict Detection and Resolution

2024-10-08 Thread shveta malik
On Tue, Oct 8, 2024 at 3:12 PM Nisha Moond wrote: > I have not started reviewing v15 yet, but here are few comments for v14-patch003: 1) In apply_handle_update_internal(), I see that FindReplTupleInLocalRel() used to lock the row to be updated in exclusive mode, but now since we are avoiding thi

Re: Conflict Detection and Resolution

2024-10-08 Thread Nisha Moond
On Tue, Oct 1, 2024 at 9:48 AM shveta malik wrote: > > On Mon, Sep 30, 2024 at 2:55 PM Peter Smith wrote: > > > > On Mon, Sep 30, 2024 at 4:29 PM shveta malik wrote: > > > > > > On Mon, Sep 30, 2024 at 11:04 AM Peter Smith > > > wrote: > > > > > > > > On Mon, Sep 30, 2024 at 2:27 PM shveta mal

Re: Conflict Detection and Resolution

2024-10-08 Thread Nisha Moond
On Mon, Sep 30, 2024 at 11:59 AM shveta malik wrote: > > On Mon, Sep 30, 2024 at 11:04 AM Peter Smith wrote: > > > > On Mon, Sep 30, 2024 at 2:27 PM shveta malik wrote: > > > > > > On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > > ... > > > > > > > > 13. General - ordering of conflict_type

Re: Conflict Detection and Resolution

2024-10-08 Thread Nisha Moond
On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > > Here are some review comments for v14-0001. > ~~~ > 7. > +ALTER SUBSCRIPTION name > RESET CONFLICT RESOLVER FOR ( class="parameter">conflict_type) > > I can see that this matches the implementation, but I was wondering > why don't you permit r

Re: Conflict Detection and Resolution

2024-10-02 Thread Peter Smith
onflict resolvers should create defaults CREATE SUBSCRIPTION regress_testsub CONNECTION 'dbname=regress_doesnotexist' PUBLICATION testpub WITH (connect = false); WARNING: subscription was created, but is not connected HINT: To initiate replication, you must manually create the re

Re: Conflict Detection and Resolution

2024-10-01 Thread shveta malik
On Tue, Oct 1, 2024 at 9:54 AM shveta malik wrote: > > On Tue, Oct 1, 2024 at 9:48 AM shveta malik wrote: > > > > I have started reviewing patch002, it is WIP, but please find initial > set of comments: > Please find second set of comments for patch002: 9) can_create_full_tuple(): + for (i = 0;

Re: Conflict Detection and Resolution

2024-09-30 Thread shveta malik
On Tue, Oct 1, 2024 at 9:48 AM shveta malik wrote: > I have started reviewing patch002, it is WIP, but please find initial set of comments: 1. ExecSimpleRelationInsert(): + /* Check for conflict and return to caller for resolution if found */ + if (resolver != CR_ERROR && + has_conflicting_tuple

Re: Conflict Detection and Resolution

2024-09-30 Thread shveta malik
On Mon, Sep 30, 2024 at 2:55 PM Peter Smith wrote: > > On Mon, Sep 30, 2024 at 4:29 PM shveta malik wrote: > > > > On Mon, Sep 30, 2024 at 11:04 AM Peter Smith wrote: > > > > > > On Mon, Sep 30, 2024 at 2:27 PM shveta malik > > > wrote: > > > > > > > > On Fri, Sep 27, 2024 at 1:00 PM Peter Smi

Re: Conflict Detection and Resolution

2024-09-30 Thread Peter Smith
On Mon, Sep 30, 2024 at 4:29 PM shveta malik wrote: > > On Mon, Sep 30, 2024 at 11:04 AM Peter Smith wrote: > > > > On Mon, Sep 30, 2024 at 2:27 PM shveta malik wrote: > > > > > > On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > > > > > > > ~~~ > > > > > > > > 14. > > > > 99. General - orde

Re: Conflict Detection and Resolution

2024-09-30 Thread shveta malik
On Fri, Sep 27, 2024 at 2:33 PM shveta malik wrote: > > On Fri, Sep 27, 2024 at 10:44 AM shveta malik wrote: > > > > > > > > > > Thanks for the review. > > > > Here is the v14 patch-set fixing review comments in [1] and [2]. > > > > > > > > > > Thanks for the patches. I am reviewing patch001, it

Re: Conflict Detection and Resolution

2024-09-29 Thread shveta malik
On Mon, Sep 30, 2024 at 11:04 AM Peter Smith wrote: > > On Mon, Sep 30, 2024 at 2:27 PM shveta malik wrote: > > > > On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > ... > > > > > > 13. General - ordering of conflict_type. > > > > > > nit - Instead of just some apparent random order, let's pu

Re: Conflict Detection and Resolution

2024-09-29 Thread Peter Smith
On Mon, Sep 30, 2024 at 2:27 PM shveta malik wrote: > > On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: ... > > > > 13. General - ordering of conflict_type. > > > > nit - Instead of just some apparent random order, let's put each > > insert/update/delete conflict type in alphabetical order, so

Re: Conflict Detection and Resolution

2024-09-29 Thread shveta malik
On Fri, Sep 27, 2024 at 1:00 PM Peter Smith wrote: > > Here are some review comments for v14-0001. > > This is a WIP, but here are my comments for all the SGML parts. > > (There will be some overlap here with comments already posted by Shveta) > > == > 1. file modes after applying the patch >

Re: Conflict Detection and Resolution

2024-09-27 Thread shveta malik
On Fri, Sep 27, 2024 at 10:44 AM shveta malik wrote: > > > > > > > Thanks for the review. > > > Here is the v14 patch-set fixing review comments in [1] and [2]. > > > > > > > Thanks for the patches. I am reviewing patch001, it is WIP, but please > > find initial set of comments: > > > Please find

Re: Conflict Detection and Resolution

2024-09-27 Thread Peter Smith
Here are some review comments for v14-0001. This is a WIP, but here are my comments for all the SGML parts. (There will be some overlap here with comments already posted by Shveta) == 1. file modes after applying the patch mode change 100644 => 100755 doc/src/sgml/ref/alter_subscription.sg

Re: Conflict Detection and Resolution

2024-09-26 Thread shveta malik
On Thu, Sep 26, 2024 at 2:57 PM shveta malik wrote: > > On Fri, Sep 20, 2024 at 8:40 AM Nisha Moond wrote: > > > > Thanks for the review. > > Here is the v14 patch-set fixing review comments in [1] and [2]. > > > > Thanks for the patches. I am reviewing patch001, it is WIP, but please > find init

Re: Conflict Detection and Resolution

2024-09-26 Thread shveta malik
On Fri, Sep 20, 2024 at 8:40 AM Nisha Moond wrote: > > Thanks for the review. > Here is the v14 patch-set fixing review comments in [1] and [2]. > Thanks for the patches. I am reviewing patch001, it is WIP, but please find initial set of comments: 1) Please see these 2 errors: postgres=# create

Re: Conflict Detection and Resolution

2024-09-19 Thread Nisha Moond
ROM pg_catalog.pg_subscription_conflict " > > + "WHERE confsubid = > > %u;", subinfo[i].dobj.catId.oid); > > + confRes = ExecuteSqlQuery(fout, confQuery->data, > > PGRES_TUPLES_OK); > > + > &

Re: Conflict Detection and Resolution

2024-09-19 Thread Ajin Cherian
On Fri, Sep 13, 2024 at 10:20 PM vignesh C wrote: > > Few comments: > 1) Tab completion missing for: > a) ALTER SUBSCRIPTION name CONFLICT RESOLVER > b) ALTER SUBSCRIPTION name RESET CONFLICT RESOLVER ALL > c) ALTER SUBSCRIPTION name RESET CONFLICT RESOLVER FOR > > Added. > 2) Documentation mis

Re: Conflict Detection and Resolution

2024-09-19 Thread Nisha Moond
ROM pg_catalog.pg_subscription_conflict " > > + "WHERE confsubid = > > %u;", subinfo[i].dobj.catId.oid); > > + confRes = ExecuteSqlQuery(fout, confQuery->data, > > PGRES_TUPLES_OK); > > + > &

Re: Conflict Detection and Resolution

2024-09-19 Thread shveta malik
On Thu, Sep 19, 2024 at 5:43 PM vignesh C wrote: > > > > > I was reviewing the CONFLICT RESOLVER (insert_exists='apply_remote') > and found that one conflict remains unresolved in the following > scenario: Thanks for the review and testing. > Pub: > CREATE TABLE circles(c1 CIRCLE, c2 text, EXCLU

Re: Conflict Detection and Resolution

2024-09-19 Thread Michail Nikolaev
Hello! Sorry for being noisy, just for the case, want to notice that [1] needs to be addressed before any real usage of conflict resolution. [1]: https://www.postgresql.org/message-id/flat/OS0PR01MB5716E30952F542E256DD72E294802%40OS0PR01MB5716.jpnprd01.prod.outlook.com#8aa2083efa76e6a65f51b8a7fd5

Re: Conflict Detection and Resolution

2024-09-19 Thread vignesh C
On Thu, 12 Sept 2024 at 14:03, Ajin Cherian wrote: > > On Tue, Sep 3, 2024 at 7:42 PM vignesh C wrote: > > On Fri, 30 Aug 2024 at 11:01, Nisha Moond > wrote: > > > > Here is the v11 patch-set. Changes are: > I was reviewing the CONFLICT RESOLVER (insert_exists='apply_remote') and f

Re: Conflict Detection and Resolution

2024-09-17 Thread vignesh C
g is logged in this case as it seems strange to throw a warning first and then an error for the same track_commit_timestamp configuration: postgres=# create subscription sub1 connection ... publication pub1 conflict resolver (insert_exists = 'last_update_wins'); WARNING: conflict detection and r

Re: Conflict Detection and Resolution

2024-09-13 Thread vignesh C
On Thu, 12 Sept 2024 at 14:03, Ajin Cherian wrote: > > On Tue, Sep 3, 2024 at 7:42 PM vignesh C wrote: > > On Fri, 30 Aug 2024 at 11:01, Nisha Moond > wrote: > > > > Here is the v11 patch-set. Changes are: > > 1) This command crashes: > ALTER SUBSCRIPTION name RESET CONFLICT

Re: Conflict Detection and Resolution

2024-09-09 Thread shveta malik
On Mon, Sep 9, 2024 at 2:58 PM shveta malik wrote: > > On Fri, Sep 6, 2024 at 2:05 PM Ajin Cherian wrote: > > > > > > Thank you for your feedback, Shveta. I've addressed both sets of comments > > you provided. > > Thanks for the patches. I am reviewing v12-patch001, it is WIP. But > please find

Re: Conflict Detection and Resolution

2024-09-09 Thread Nisha Moond
On Fri, Sep 6, 2024 at 2:05 PM Ajin Cherian wrote: > > > > On Thu, Aug 29, 2024 at 2:50 PM shveta malik wrote: >> >> On Wed, Aug 28, 2024 at 4:07 PM shveta malik wrote: >> > >> > > On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: >> > > > >> > >> > The review is WIP. Please find a few comme

Re: Conflict Detection and Resolution

2024-09-09 Thread shveta malik
On Fri, Sep 6, 2024 at 2:05 PM Ajin Cherian wrote: > > > Thank you for your feedback, Shveta. I've addressed both sets of comments you > provided. Thanks for the patches. I am reviewing v12-patch001, it is WIP. But please find first set of comments: 1) src/sgml/logical-replication.sgml: + Users

Re: Conflict Detection and Resolution

2024-09-06 Thread Nisha Moond
On Thu, Aug 29, 2024 at 4:43 PM Amit Kapila wrote: > > On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > > > Please find issues which need some thoughts and approval for > > time-based resolution and clock-skew. > > > > 1) > > Time based conflict resolution and two phase transactions: > >

Re: Conflict Detection and Resolution

2024-09-03 Thread vignesh C
n commands: +check_conflict_detection(void) +{ + if (!track_commit_timestamp) + ereport(WARNING, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("conflict detection and resolution could be incomplete due to disabled track_commit_t

Re: Conflict Detection and Resolution

2024-09-03 Thread vignesh C
On Fri, 30 Aug 2024 at 11:01, Nisha Moond wrote: > > Here is the v11 patch-set. Changes are: 1) This command crashes: ALTER SUBSCRIPTION name RESET CONFLICT RESOLVER FOR NULL; #0 __strcmp_avx2 () at ../sysdeps/x86_64/multiarch/strcmp-avx2.S:116 #1 0x55c67270600a in ResetConflictResolver (su

Re: Conflict Detection and Resolution

2024-08-29 Thread Amit Kapila
On Wed, Aug 28, 2024 at 4:07 PM shveta malik wrote: > > > On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: > > > > > The review is WIP. Please find a few comments on patch001. > > 1) > logical-repliction.sgmlL > > + Additional logging is triggered for specific conflict_resolvers. > Users can

Re: Conflict Detection and Resolution

2024-08-29 Thread shveta malik
On Fri, Aug 30, 2024 at 12:13 PM Amit Kapila wrote: > > On Wed, Aug 28, 2024 at 10:58 AM shveta malik wrote: > > > > On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: > > > > > >> 2) > > >> Currently pg_dump is dumping even the default resolvers configuration. > > >> As an example if I have n

Re: Conflict Detection and Resolution

2024-08-29 Thread Amit Kapila
On Wed, Aug 28, 2024 at 10:58 AM shveta malik wrote: > > On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: > > > >> 2) > >> Currently pg_dump is dumping even the default resolvers configuration. > >> As an example if I have not changed default configuration for say > >> sub1, it still dumps al

Re: Conflict Detection and Resolution

2024-08-29 Thread Nisha Moond
On Mon, Aug 26, 2024 at 2:23 PM Amit Kapila wrote: > > On Thu, Aug 22, 2024 at 3:45 PM shveta malik wrote: > > > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond > > wrote: > > > > > > The patches have been rebased on the latest pgHead following the merge > > > of the conflict detection patch [1].

Re: Conflict Detection and Resolution

2024-08-29 Thread Nisha Moond
On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > On Thu, Aug 22, 2024 at 3:44 PM shveta malik wrote: > > > > > > For clock-skew and timestamp based resolution, if needed, I will post > > another email for the design items where suggestions are needed. > > > > Please find issues which need

Re: Conflict Detection and Resolution

2024-08-29 Thread Amit Kapila
On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > Please find issues which need some thoughts and approval for > time-based resolution and clock-skew. > > 1) > Time based conflict resolution and two phase transactions: > > Time based conflict resolution (last_update_wins) is the one > resol

Re: Conflict Detection and Resolution

2024-08-28 Thread shveta malik
On Wed, Aug 28, 2024 at 4:07 PM shveta malik wrote: > > > On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: > > > > > The review is WIP. Please find a few comments on patch001. > More comments on ptach001 in continuation of previous comments: 6) SetDefaultResolvers() can be called from parse

Re: Conflict Detection and Resolution

2024-08-28 Thread shveta malik
> On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: > > The review is WIP. Please find a few comments on patch001. 1) logical-repliction.sgmlL + Additional logging is triggered for specific conflict_resolvers. Users can also configure conflict_types while creating the subscription. Refer to

Re: Conflict Detection and Resolution

2024-08-27 Thread shveta malik
On Wed, Aug 28, 2024 at 10:30 AM Ajin Cherian wrote: > >> 2) >> Currently pg_dump is dumping even the default resolvers configuration. >> As an example if I have not changed default configuration for say >> sub1, it still dumps all: >> >> CREATE SUBSCRIPTION sub1 CONNECTION '..' PUBLICATION pub1 W

Re: Conflict Detection and Resolution

2024-08-27 Thread Ajin Cherian
On Wed, Aug 28, 2024 at 2:27 PM shveta malik wrote: > > 2) > Currently pg_dump is dumping even the default resolvers configuration. > As an example if I have not changed default configuration for say > sub1, it still dumps all: > > CREATE SUBSCRIPTION sub1 CONNECTION '..' PUBLICATION pub1 WITH (.

Re: Conflict Detection and Resolution

2024-08-27 Thread shveta malik
On Tue, Aug 27, 2024 at 1:51 PM Nisha Moond wrote: > > Please find v10 patch-set. Changes are: > > 1) patch-001: > - Corrected a patch application warning. > - Added support for pg_dump. > - As suggested in pt.4 of [1]: added a warning during CREATE and > ALTER subscription when track_commit_ti

Re: Conflict Detection and Resolution

2024-08-27 Thread Nisha Moond
On Fri, Aug 23, 2024 at 10:39 AM shveta malik wrote: > > On Thu, Aug 22, 2024 at 3:44 PM shveta malik wrote: > > > > > > For clock-skew and timestamp based resolution, if needed, I will post > > another email for the design items where suggestions are needed. > > > > Please find issues which need

Re: Conflict Detection and Resolution

2024-08-27 Thread Nisha Moond
On Mon, Aug 26, 2024 at 2:23 PM Amit Kapila wrote: > > On Thu, Aug 22, 2024 at 3:45 PM shveta malik wrote: > > > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond > > wrote: > > > > > > The patches have been rebased on the latest pgHead following the merge > > > of the conflict detection patch [1].

Re: Conflict Detection and Resolution

2024-08-27 Thread Nisha Moond
On Mon, Aug 26, 2024 at 9:05 AM shveta malik wrote: > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond wrote: > > > > The patches have been rebased on the latest pgHead following the merge > > of the conflict detection patch [1]. The detect_conflict option has > > been removed, and conflict detectio

Re: Conflict Detection and Resolution

2024-08-26 Thread shveta malik
On Mon, Aug 26, 2024 at 2:23 PM Amit Kapila wrote: > > On Thu, Aug 22, 2024 at 3:45 PM shveta malik wrote: > > > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond > > wrote: > > > > > > The patches have been rebased on the latest pgHead following the merge > > > of the conflict detection patch [1].

Re: Conflict Detection and Resolution

2024-08-26 Thread Amit Kapila
On Mon, Aug 26, 2024 at 7:28 AM Peter Smith wrote: > > On Thu, Aug 22, 2024 at 8:15 PM shveta malik wrote: > > > > Hi Shveta, > > I felt it would be better to keep the syntax similar to the existing > INSERT ... ON CONFLICT [1]. > > I'd suggest a syntax like this: > > ... ON CONFLICT ['conflict_t

Re: Conflict Detection and Resolution

2024-08-26 Thread Amit Kapila
On Thu, Aug 22, 2024 at 3:45 PM shveta malik wrote: > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond wrote: > > > > The patches have been rebased on the latest pgHead following the merge > > of the conflict detection patch [1]. > > Thanks for working on patches. > > Summarizing the issues which ne

Re: Conflict Detection and Resolution

2024-08-25 Thread Nisha Moond
On Thu, Aug 22, 2024 at 3:45 PM shveta malik wrote: > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond wrote: > > > > The patches have been rebased on the latest pgHead following the merge > > of the conflict detection patch [1]. > > Thanks for working on patches. > > Summarizing the issues which ne

Re: Conflict Detection and Resolution

2024-08-25 Thread shveta malik
On Mon, Aug 26, 2024 at 7:28 AM Peter Smith wrote: > > On Thu, Aug 22, 2024 at 8:15 PM shveta malik wrote: > > > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond > > wrote: > > > > > > The patches have been rebased on the latest pgHead following the merge > > > of the conflict detection patch [1].

Re: Conflict Detection and Resolution

2024-08-25 Thread shveta malik
On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond wrote: > > The patches have been rebased on the latest pgHead following the merge > of the conflict detection patch [1]. The detect_conflict option has > been removed, and conflict detection is now enabled by default. This > change required the following

Re: Conflict Detection and Resolution

2024-08-25 Thread Peter Smith
On Thu, Aug 22, 2024 at 8:15 PM shveta malik wrote: > > On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond wrote: > > > > The patches have been rebased on the latest pgHead following the merge > > of the conflict detection patch [1]. > > Thanks for working on patches. > > Summarizing the issues which ne

Re: Conflict Detection and Resolution

2024-08-22 Thread shveta malik
On Thu, Aug 22, 2024 at 3:44 PM shveta malik wrote: > > > For clock-skew and timestamp based resolution, if needed, I will post > another email for the design items where suggestions are needed. > Please find issues which need some thoughts and approval for time-based resolution and clock-skew.

Re: Conflict Detection and Resolution

2024-08-22 Thread shveta malik
On Wed, Aug 21, 2024 at 4:08 PM Nisha Moond wrote: > > The patches have been rebased on the latest pgHead following the merge > of the conflict detection patch [1]. Thanks for working on patches. Summarizing the issues which need some suggestions/thoughts. 1) For subscription based resolvers, c

Re: Conflict Detection and Resolution

2024-07-30 Thread Dilip Kumar
On Tue, Jul 30, 2024 at 4:56 PM shveta malik wrote: > > On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar wrote: > > > > On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: > > > > Comment in 0002, > > > > 1) I do not see any test case that set a proper conflict type and > > conflict resolver, all tes

Re: Conflict Detection and Resolution

2024-07-30 Thread shveta malik
On Tue, Jul 30, 2024 at 4:04 PM Dilip Kumar wrote: > > On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: > > Comment in 0002, > > 1) I do not see any test case that set a proper conflict type and > conflict resolver, all tests either give incorrect conflict > type/conflict resolver or the confl

Re: Conflict Detection and Resolution

2024-07-30 Thread Dilip Kumar
On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: Comment in 0002, 1) I do not see any test case that set a proper conflict type and conflict resolver, all tests either give incorrect conflict type/conflict resolver or the conflict resolver is ignored 0003 2) I was trying to think about this

Re: Conflict Detection and Resolution

2024-07-29 Thread shveta malik
On Fri, Jul 26, 2024 at 9:50 AM Ajin Cherian wrote: >> > Please find v7 patch-set, the changes are: > Thanks Ajin for working on this. Please find few comments: 1) parse_subscription_conflict_resolvers(): Here we loop in this function to find the given conflict type in the supported list and err

RE: Conflict Detection and Resolution

2024-07-08 Thread Zhijie Hou (Fujitsu)
On Monday, July 8, 2024 12:32 PM Zhijie Hou (Fujitsu) wrote: > > I researched about how to detect the resolve update_deleted and thought > about one idea: which is to maintain the xmin in logical slot to preserve > the dead row and support latest_timestamp_xmin resolution for > update_deleted to

RE: Conflict Detection and Resolution

2024-07-07 Thread Zhijie Hou (Fujitsu)
Hi, I researched about how to detect the resolve update_deleted and thought about one idea: which is to maintain the xmin in logical slot to preserve the dead row and support latest_timestamp_xmin resolution for update_deleted to maintain data consistency. Here are details of the xmin idea and re

Re: Conflict Detection and Resolution

2024-07-05 Thread Nisha Moond
On Mon, Jul 1, 2024 at 1:17 PM Ajin Cherian wrote: > > > > On Thu, Jun 27, 2024 at 1:14 PM Nisha Moond wrote: >> >> Please find the attached 'patch0003', which implements conflict >> resolutions according to the global resolver settings. >> >> Summary of Conflict Resolutions Implemented in 'patc

Re: Conflict Detection and Resolution

2024-07-05 Thread Dilip Kumar
On Fri, Jul 5, 2024 at 2:23 PM Amit Kapila wrote: > > On Fri, Jul 5, 2024 at 11:58 AM Dilip Kumar wrote: > > > > On Thu, Jul 4, 2024 at 5:37 PM Amit Kapila wrote: > > > > So, the situation will be the same. We can even > > > > > decide to spill the data to files if the decision is that we need

Re: Conflict Detection and Resolution

2024-07-05 Thread Amit Kapila
On Fri, Jul 5, 2024 at 11:58 AM Dilip Kumar wrote: > > On Thu, Jul 4, 2024 at 5:37 PM Amit Kapila wrote: > > > So, the situation will be the same. We can even > > > > decide to spill the data to files if the decision is that we need to > > > > wait to avoid network buffer-fill situations. But no

Re: Conflict Detection and Resolution

2024-07-04 Thread Dilip Kumar
On Thu, Jul 4, 2024 at 5:37 PM Amit Kapila wrote: > > So, the situation will be the same. We can even > > > decide to spill the data to files if the decision is that we need to > > > wait to avoid network buffer-fill situations. But note that the wait > > > in apply worker has consequences that t

Re: Conflict Detection and Resolution

2024-07-04 Thread Amit Kapila
On Wed, Jul 3, 2024 at 5:06 PM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 4:48 PM Amit Kapila wrote: > > > > On Wed, Jul 3, 2024 at 4:04 PM Dilip Kumar wrote: > > > > > > > > > What I am saying is that if we are not applying the whole transaction, > > > it means we are not receiving it either

Re: Conflict Detection and Resolution

2024-07-03 Thread Masahiko Sawada
On Mon, Jul 1, 2024 at 6:54 PM Amit Kapila wrote: > > On Mon, Jul 1, 2024 at 1:35 PM Masahiko Sawada wrote: > > > > Setting resolvers at table-level and subscription-level sounds good to > > me. DDLs for setting resolvers at subscription-level would need the > > subscription name to be specified?

Re: Conflict Detection and Resolution

2024-07-03 Thread Dilip Kumar
On Wed, Jul 3, 2024 at 5:08 PM shveta malik wrote: > > On Wed, Jul 3, 2024 at 4:12 PM Dilip Kumar wrote: > > > > On Wed, Jul 3, 2024 at 4:02 PM shveta malik wrote: > > > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > > > > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik > > >

Re: Conflict Detection and Resolution

2024-07-03 Thread shveta malik
On Wed, Jul 3, 2024 at 4:12 PM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 4:02 PM shveta malik wrote: > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik > > > wrote: > > > > > > > > > Yes, I also think it should be independent

Re: Conflict Detection and Resolution

2024-07-03 Thread Dilip Kumar
On Wed, Jul 3, 2024 at 4:48 PM Amit Kapila wrote: > > On Wed, Jul 3, 2024 at 4:04 PM Dilip Kumar wrote: > > > > On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila wrote: > > > > > > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote: > > > > > > > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila > > > > w

Re: Conflict Detection and Resolution

2024-07-03 Thread Amit Kapila
On Wed, Jul 3, 2024 at 4:04 PM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila wrote: > > > > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote: > > > > > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila > > > wrote: > > > > > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar >

Re: Conflict Detection and Resolution

2024-07-03 Thread Dilip Kumar
On Wed, Jul 3, 2024 at 4:02 PM shveta malik wrote: > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik wrote: > > > > > > > Yes, I also think it should be independent of CDR. IMHO, it should be > > > > based on the user-configured maximum

Re: Conflict Detection and Resolution

2024-07-03 Thread shveta malik
On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila wrote: > > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote: > > > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila wrote: > > > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > > But waiting after applying the operations and before applyi

Re: Conflict Detection and Resolution

2024-07-03 Thread Dilip Kumar
On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila wrote: > > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote: > > > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila wrote: > > > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > > But waiting after applying the operations and before applyi

Re: Conflict Detection and Resolution

2024-07-03 Thread shveta malik
On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik wrote: > > > > > Yes, I also think it should be independent of CDR. IMHO, it should be > > > based on the user-configured maximum clock skew tolerance and can be > > > independent of CDR. > > > >

Re: Conflict Detection and Resolution

2024-07-03 Thread Amit Kapila
On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila wrote: > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > But waiting after applying the operations and before applying the > > commit would mean that we need to wait with the locks he

Re: Conflict Detection and Resolution

2024-07-03 Thread Dilip Kumar
On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila wrote: > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > But waiting after applying the operations and before applying the > commit would mean that we need to wait with the locks held. That could > be a recipe for deadlocks in the system. I see

Re: Conflict Detection and Resolution

2024-07-03 Thread Amit Kapila
On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik wrote: > > > > > Yes, I also think it should be independent of CDR. IMHO, it should be > > > based on the user-configured maximum clock skew tolerance and can be > > > independent of CDR. > > > >

Re: Conflict Detection and Resolution

2024-07-02 Thread Dilip Kumar
On Wed, Jul 3, 2024 at 11:00 AM shveta malik wrote: > > > Yes, I also think it should be independent of CDR. IMHO, it should be > > based on the user-configured maximum clock skew tolerance and can be > > independent of CDR. > > +1 > > > IIUC we would make the remote apply wait just > > before co

Re: Conflict Detection and Resolution

2024-07-02 Thread shveta malik
On Wed, Jul 3, 2024 at 10:47 AM Dilip Kumar wrote: > > On Tue, Jul 2, 2024 at 2:40 PM shveta malik wrote: > > > > On Wed, Jun 19, 2024 at 1:52 PM Dilip Kumar wrote: > > > > > > On Tue, Jun 18, 2024 at 3:29 PM shveta malik > > > wrote: > > > > On Tue, Jun 18, 2024 at 11:34 AM Dilip Kumar > >

Re: Conflict Detection and Resolution

2024-07-02 Thread Dilip Kumar
On Tue, Jul 2, 2024 at 2:40 PM shveta malik wrote: > > On Wed, Jun 19, 2024 at 1:52 PM Dilip Kumar wrote: > > > > On Tue, Jun 18, 2024 at 3:29 PM shveta malik wrote: > > > On Tue, Jun 18, 2024 at 11:34 AM Dilip Kumar > > > wrote: > > > > > > I tried to work out a few scenarios with this, where

Re: Conflict Detection and Resolution

2024-07-02 Thread shveta malik
On Wed, Jun 19, 2024 at 1:52 PM Dilip Kumar wrote: > > On Tue, Jun 18, 2024 at 3:29 PM shveta malik wrote: > > On Tue, Jun 18, 2024 at 11:34 AM Dilip Kumar wrote: > > > > I tried to work out a few scenarios with this, where the apply worker > > will wait until its local clock hits 'remote_commit

Re: Conflict Detection and Resolution

2024-07-01 Thread shveta malik
On Mon, Jul 1, 2024 at 11:47 AM Masahiko Sawada wrote: > > Hi, > > On Thu, May 23, 2024 at 3:37 PM shveta malik wrote: > > > > DELETE > > > > Conflict Type: > > > > delete_missing: An incoming delete is trying to delete a row on a > > target node which does not e

Re: Conflict Detection and Resolution

2024-07-01 Thread Amit Kapila
On Mon, Jul 1, 2024 at 1:35 PM Masahiko Sawada wrote: > > Setting resolvers at table-level and subscription-level sounds good to > me. DDLs for setting resolvers at subscription-level would need the > subscription name to be specified? > Yes, it should be part of the ALTER/CREATE SUBSCRIPTION com

Re: Conflict Detection and Resolution

2024-07-01 Thread Amit Kapila
On Mon, Jul 1, 2024 at 11:47 AM Masahiko Sawada wrote: > > On Thu, May 23, 2024 at 3:37 PM shveta malik wrote: > > > > DELETE > > > > Conflict Type: > > > > delete_missing: An incoming delete is trying to delete a row on a > > target node which does not exist. >

Re: Conflict Detection and Resolution

2024-07-01 Thread Masahiko Sawada
On Thu, Jun 27, 2024 at 1:50 PM shveta malik wrote: > > On Wed, Jun 26, 2024 at 2:33 PM Amit Kapila wrote: > > > > On Tue, Jun 25, 2024 at 3:39 PM shveta malik wrote: > > > > > > On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, Jun 24, 2024 at 1:47 PM shveta mal

Re: Conflict Detection and Resolution

2024-07-01 Thread Ajin Cherian
On Thu, Jun 27, 2024 at 1:14 PM Nisha Moond wrote: > Please find the attached 'patch0003', which implements conflict > resolutions according to the global resolver settings. > > Summary of Conflict Resolutions Implemented in 'patch0003': > > INSERT Conflicts: > > 1) Conf

Re: Conflict Detection and Resolution

2024-06-30 Thread Masahiko Sawada
Hi, On Thu, May 23, 2024 at 3:37 PM shveta malik wrote: > > DELETE > > Conflict Type: > > delete_missing: An incoming delete is trying to delete a row on a > target node which does not exist. IIUC the 'delete_missing' conflict doesn't cover the case where an inc

Re: Conflict Detection and Resolution

2024-06-26 Thread shveta malik
On Wed, Jun 26, 2024 at 2:33 PM Amit Kapila wrote: > > On Tue, Jun 25, 2024 at 3:39 PM shveta malik wrote: > > > > On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila wrote: > > > > > > On Mon, Jun 24, 2024 at 1:47 PM shveta malik > > > wrote: > > > > > > > > On Thu, Jun 20, 2024 at 6:41 PM Amit Kapil

Re: Conflict Detection and Resolution

2024-06-26 Thread Amit Kapila
On Tue, Jun 25, 2024 at 3:39 PM shveta malik wrote: > > On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila wrote: > > > > On Mon, Jun 24, 2024 at 1:47 PM shveta malik wrote: > > > > > > On Thu, Jun 20, 2024 at 6:41 PM Amit Kapila > > > wrote: > > > > > > > > >> In the second patch, we can implement s

Re: Conflict Detection and Resolution

2024-06-25 Thread shveta malik
On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila wrote: > > On Mon, Jun 24, 2024 at 1:47 PM shveta malik wrote: > > > > On Thu, Jun 20, 2024 at 6:41 PM Amit Kapila wrote: > > > > > > >> In the second patch, we can implement simple built-in resolution > > > >> strategies like apply and skip (which can

Re: Conflict Detection and Resolution

2024-06-25 Thread Amit Kapila
On Mon, Jun 24, 2024 at 1:47 PM shveta malik wrote: > > On Thu, Jun 20, 2024 at 6:41 PM Amit Kapila wrote: > > > > >> In the second patch, we can implement simple built-in resolution > > >> strategies like apply and skip (which can be named as remote_apply and > > >> keep_local, see [3][4] for de

Re: Conflict Detection and Resolution

2024-06-20 Thread Amit Kapila
and >> 'delete_missing' (the definition of each could be found in the initial >> email [1]) and throw an ERROR or write them in LOG. Various people >> agreed to have this as a separate committable work [2]. This can help >> users to detect and monitor the conflicts

  1   2   >