Re: Synchronizing slots from primary to standby

2023-08-29 Thread shveta malik
On Fri, Aug 25, 2023 at 11:09 AM shveta malik wrote: > > On Wed, Aug 23, 2023 at 4:21 PM Dilip Kumar wrote: > > > > On Wed, Aug 23, 2023 at 3:38 PM shveta malik wrote: > > > > > I have reviewed the v12-0002 patch and I have some comments. I see the > > lat

Re: Synchronizing slots from primary to standby

2023-08-29 Thread shveta malik
On Wed, Aug 23, 2023 at 4:21 PM Dilip Kumar wrote: > > On Wed, Aug 23, 2023 at 3:38 PM shveta malik wrote: > > > I have reviewed the v12-0002 patch and I have some comments. I see the > latest version posted sometime back and if any of this comment is > already fixed in t

Re: Synchronizing slots from primary to standby

2023-09-06 Thread shveta malik
On Fri, Sep 1, 2023 at 1:59 PM Peter Smith wrote: > > Hi Shveta. Here are some comments for patch v14-0002 > > The patch is large, so my code review is a WIP... more later next week... > Thanks Peter for the feedback. I have tried to address most of these in v15. Please find my response inline fo

Re: Synchronizing slots from primary to standby

2023-09-06 Thread shveta malik
On Fri, Aug 25, 2023 at 2:15 PM Alvaro Herrera wrote: > > Wait a minute ... > > From bac0fbef8b203c530e5117b0b7cfee13cfab78b9 Mon Sep 17 00:00:00 2001 > From: Bharath Rupireddy > Date: Sat, 22 Jul 2023 10:17:48 + > Subject: [PATCH v13 1/2] Allow logical walsenders to wait for physical > stan

Re: Synchronizing slots from primary to standby

2023-09-10 Thread shveta malik
On Fri, Sep 8, 2023 at 4:40 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shveta, > > I resumed to check the thread. Here are my high-level comments. > Sorry if you have been already discussed. Thanks Kuroda-san for the feedback. > > 01. General > > I think the documentation can be added, not only G

Re: Synchronizing slots from primary to standby

2023-09-13 Thread shveta malik
On Wed, Sep 6, 2023 at 2:18 PM shveta malik wrote: > > On Fri, Sep 1, 2023 at 1:59 PM Peter Smith wrote: > > > > Hi Shveta. Here are some comments for patch v14-0002 > > > > The patch is large, so my code review is a WIP... more later next week... > > >

Re: Synchronizing slots from primary to standby

2023-09-18 Thread shveta malik
On Wed, Sep 13, 2023 at 4:48 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shveta, > > Sorry for the late response. > > > Thanks Kuroda-san for the feedback. > > > > > > 01. General > > > > > > I think the documentation can be added, not only GUCs. How about adding > > examples > > > for combinations

Re: Synchronizing slots from primary to standby

2023-09-18 Thread shveta malik
On Fri, Sep 15, 2023 at 2:22 PM Peter Smith wrote: > > Hi. Here are some review comments for v17-0002. > Thanks Peter for the feedback. I have addressed most of these in v18 except 2. Please find my comments for the ones not addressed. > This is a WIP and a long way from complete, but I wanted

Re: Synchronizing slots from primary to standby

2023-09-20 Thread shveta malik
On Tue, Sep 19, 2023 at 10:29 AM shveta malik wrote: > > On Fri, Sep 15, 2023 at 2:22 PM Peter Smith wrote: > > > > Hi. Here are some review comments for v17-0002. > > > > Thanks Peter for the feedback. I have addressed most of these in v18 > except 2. Please

Re: Synchronizing slots from primary to standby

2023-09-24 Thread shveta malik
On Fri, Sep 22, 2023 at 6:01 PM Drouvot, Bertrand wrote: > > Hi, > > Thanks for all the work that has been done on this feature, and sorry > to have been quiet on it for so long. Thanks for looking into this. > > On 9/18/23 12:22 PM, shveta malik wrote: > > On W

Re: Synchronizing slots from primary to standby

2023-09-24 Thread shveta malik
On Fri, Sep 22, 2023 at 3:48 PM Amit Kapila wrote: > > On Thu, Sep 21, 2023 at 9:16 AM shveta malik wrote: > > > > On Tue, Sep 19, 2023 at 10:29 AM shveta malik > > wrote: > > > > Currently in patch001, synchronize_slot_names is a GUC on both primary >

Re: Synchronizing slots from primary to standby

2023-09-27 Thread shveta malik
On Mon, Sep 25, 2023 at 7:46 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Ajin, Shveta, > > Thank you for rebasing the patch set! Here are new comments for v19_2-0001. > Thank You Kuroda-san for the feedback. Most of these are addressed in v20. Please find my response inline. > 01. WalSndWaitForSt

Re: Synchronizing slots from primary to standby

2023-10-03 Thread shveta malik
On Tue, Oct 3, 2023 at 7:56 PM Drouvot, Bertrand wrote: > > Hi, > > On 10/3/23 12:54 PM, Amit Kapila wrote: > > On Mon, Oct 2, 2023 at 11:39 AM Drouvot, Bertrand > > wrote: > >> > >> On 9/29/23 1:33 PM, Amit Kapila wrote: > >>> On Thu, Sep 28, 2023 at 6:31 PM Drouvot, Bertrand > >>> wrote: > >>>

Re: Synchronizing slots from primary to standby

2023-10-03 Thread shveta malik
On Wed, Oct 4, 2023 at 5:36 AM Amit Kapila wrote: > > On Tue, Oct 3, 2023 at 9:27 PM shveta malik wrote: > > > > On Tue, Oct 3, 2023 at 7:56 PM Drouvot, Bertrand > > wrote: > > > > > > Hi, > > > > > > On 10/3/23 12:54 PM, Amit K

Re: Synchronizing slots from primary to standby

2023-10-03 Thread shveta malik
On Wed, Oct 4, 2023 at 9:56 AM shveta malik wrote: > > On Wed, Oct 4, 2023 at 5:36 AM Amit Kapila wrote: > > > > On Tue, Oct 3, 2023 at 9:27 PM shveta malik wrote: > > > > > > On Tue, Oct 3, 2023 at 7:56 PM Drouvot, Bertrand > > > wrote: > >

Re: Synchronizing slots from primary to standby

2023-10-03 Thread shveta malik
On Mon, Oct 2, 2023 at 4:29 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shveta, > > Thank you for updating the patch! > > I found another ERROR due to the slot removal. Is this a real issue? > > 1. applied add_sleep.txt, which emulated the case the tablesync worker stucked >and the primary cras

Re: Synchronizing slots from primary to standby

2023-10-04 Thread shveta malik
On Wed, Oct 4, 2023 at 5:00 PM Amit Kapila wrote: > > On Wed, Oct 4, 2023 at 11:55 AM Drouvot, Bertrand > wrote: > > > > On 10/4/23 6:26 AM, shveta malik wrote: > > > On Wed, Oct 4, 2023 at 5:36 AM Amit Kapila > > > wrote: > > >> > >

Re: Synchronizing slots from primary to standby

2023-10-04 Thread shveta malik
On Wed, Oct 4, 2023 at 12:08 PM Drouvot, Bertrand wrote: > > Hi, > > On 10/4/23 7:00 AM, shveta malik wrote: > > On Wed, Oct 4, 2023 at 9:56 AM shveta malik wrote: > > > The most simplistic approach would be: > > > > 1) maintain standby_sl

Re: Synchronizing slots from primary to standby

2023-10-06 Thread shveta malik
On Fri, Oct 6, 2023 at 2:07 PM Alvaro Herrera wrote: > > On 2023-Sep-27, Peter Smith wrote: > > > 3. get_local_synced_slot_names > > > > + for (int i = 0; i < max_replication_slots; i++) > > + { > > + ReplicationSlot *s = &ReplicationSlotCtl->replication_slots[i]; > > + > > + /* Check if it is log

Re: Synchronizing slots from primary to standby

2023-10-09 Thread shveta malik
On Wed, Oct 4, 2023 at 8:53 AM Peter Smith wrote: > > Here are some review comments for v20-0002. > Thanks Peter for the feedback. Comments from 31 till end are addressed in v22. First 30 comments will be addressed in the next version. > == > 1. GENERAL - errmsg/elog messages > > There are a

Re: Synchronizing slots from primary to standby

2023-10-09 Thread shveta malik
On Mon, Oct 9, 2023 at 3:24 AM Peter Smith wrote: > > On Fri, Oct 6, 2023 at 7:37 PM Alvaro Herrera wrote: > > > > On 2023-Sep-27, Peter Smith wrote: > > > > > 3. get_local_synced_slot_names > > > > > > + for (int i = 0; i < max_replication_slots; i++) > > > + { > > > + ReplicationSlot *s = &Repl

Re: Synchronizing slots from primary to standby

2023-10-09 Thread shveta malik
On Mon, Oct 2, 2023 at 4:29 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shveta, > > Thank you for updating the patch! Thanks for the feedback Kuroda-san. I have addressed most of these in v22. Please find my comments inline. > > I found another ERROR due to the slot removal. Is this a real issue?

Re: Synchronizing slots from primary to standby

2023-10-11 Thread shveta malik
On Tue, Oct 10, 2023 at 12:52 PM Peter Smith wrote: > > On Mon, Oct 9, 2023 at 9:34 PM shveta malik wrote: > > > > On Wed, Oct 4, 2023 at 8:53 AM Peter Smith wrote: > > > > > > Here are some review comments for v20-0002. > > > > > > > Tha

Re: Synchronizing slots from primary to standby

2023-10-11 Thread shveta malik
On Tue, Oct 10, 2023 at 12:52 PM Peter Smith wrote: > > On Mon, Oct 9, 2023 at 9:34 PM shveta malik wrote: > > > > On Wed, Oct 4, 2023 at 8:53 AM Peter Smith wrote: > > > > > > Here are some review comments for v20-0002. > > > > > > > Tha

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-19 Thread shveta malik
On Thu, Jan 19, 2023 at 12:42 PM Takamichi Osumi (Fujitsu) wrote: > > On Wednesday, January 18, 2023 4:06 PM Peter Smith > wrote: > > Here are my review comments for the latest patch v16-0001. (excluding the > > test code) > Hi, thank you for your review ! > > > == > > > > General > > > > 1

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-20 Thread shveta malik
On Fri, Jan 20, 2023 at 1:08 PM Peter Smith wrote: > a) the message should say that this is the *remaining* time to left to wait. > > b) it might be convenient to know from the log what was the original > min_apply_delay value in the 1st place. > > For example, the logs might look something like

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-20 Thread shveta malik
On Fri, Jan 20, 2023 at 2:23 PM shveta malik wrote: > > On Fri, Jan 20, 2023 at 1:08 PM Peter Smith wrote: > > > a) the message should say that this is the *remaining* time to left to wait. > > > > b) it might be convenient to know from the log what was the original &

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-24 Thread shveta malik
On Tue, Jan 24, 2023 at 5:49 PM Takamichi Osumi (Fujitsu) wrote: > > > Attached the patch v20 that has incorporated all comments so far. > Kindly have a look at the attached patch. > > > Best Regards, > Takamichi Osumi > Thank You for patch. My previous comments are addressed. Tested it a

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-25 Thread shveta malik
On Mon, Jan 23, 2023 at 6:30 PM Melih Mutlu wrote: > > Hi, > > Thanks for your review. > Attached updated versions of the patches. > Hello, I am still in the process of reviewing the patch, before that I tried to run below test: --publisher create table tab1(id int , name varchar); create table

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-27 Thread shveta malik
On Thu, Jan 26, 2023 at 7:53 PM Melih Mutlu wrote: > > If a relation is currently being synced by a tablesync worker and uses a > replication slot/origin for that operation, then srrelslotname and > srreloriginname fields will have values. > When a relation is done with its replication slot/orig

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-31 Thread shveta malik
On Fri, Jan 27, 2023 at 3:41 PM shveta malik wrote: > > > I am reviewing further... > thanks > Shveta Few more comments: v4-0001: 1) REPLICATION_SLOT_SNAPSHOT --Do we need 'CREATE' prefix with it i.e. CREATE_REPLICATION_SNAPSHOT (or some ot

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-01-31 Thread shveta malik
On Tue, Jan 31, 2023 at 3:57 PM wangw.f...@fujitsu.com wrote: > > On Mon, Jan 23, 2023 21:00 PM Melih Mutlu wrote: > > Hi, > > > > Thanks for your review. > > Attached updated versions of the patches. > > Thanks for updating the patch set. > > > > 5. New member "created_slot" in structure Logical

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread shveta malik
On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu wrote: > > Hi, > > Please see attached patches for the below changes. > > Thanks for reviewing, > -- > Melih Mutlu > Microsoft Hello Melih, Thank you for making the changes. I have few more comments: 1) src/backend/replication/logical/worker.c: (errmsg

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread shveta malik
On Wed, Feb 1, 2023 at 5:42 PM Melih Mutlu wrote: > > > Thanks for investigating this error. I think it's the same error as the one > Shi reported earlier. [1] > I couldn't reproduce it yet but will apply your tweaks and try again. > Looking into this. > > [1] > https://www.postgresql.org/messa

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread shveta malik
On Wed, Feb 1, 2023 at 5:42 PM Melih Mutlu wrote: > > Hi Shveta, > > shveta malik , 1 Şub 2023 Çar, 15:01 tarihinde şunu > yazdı: >> >> On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu wrote: >> 2) I found a crash in the previous patch (v9), but have not tested it &g

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Thu, Feb 2, 2023 at 9:18 AM shveta malik wrote: > > > Hi Melih, > I think I am able to identify the root cause. It is not memory > corruption, but the way origin-names are stored in system-catalog > mapped to a particular relation-id before even those are created. > A

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Wed, Feb 1, 2023 at 5:37 PM Melih Mutlu wrote: > > Hi, > Please see attached patches. > > Thanks, > -- > Melih Mutlu > Microsoft Hi Melih, Few suggestions on v10-0002-Reuse patch 1) for (int64 i = 1; i <= lastusedid; i++) { charoriginname_to_drop[N

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Thu, Feb 2, 2023 at 5:01 PM shveta malik wrote: > > Reviewing further > Few more comments for v10-0002 and v7-0001: 1) + * need_full_snapshot + * if true, create a snapshot able to read all tables, + * otherwise do not create any snapshot. + * CreateDecodi

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-02 Thread shveta malik
On Fri, Feb 3, 2023 at 11:50 AM shveta malik wrote: > > On Thu, Feb 2, 2023 at 5:01 PM shveta malik wrote: > > > > 2e) > When it goes to reuse flow (i.e. before walrcv_slot_snapshot), if > needed we can dump newly obtained origin_startpos also: > > erepor

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-03 Thread shveta malik
On Thu, Feb 2, 2023 at 5:01 PM shveta malik wrote: > > Reviewing further > Hi Melih, int64 rep_slot_id; int64 lastusedid; int64 sublastusedid --Should all of the above be unsigned integers? thanks Shveta

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-08 Thread shveta malik
On Tue, Feb 7, 2023 at 8:18 AM shiy.f...@fujitsu.com wrote: > > > On Thu, Feb 2, 2023 11:48 AM shveta malik wrote: > > > > > > So to fix this, I think either we update origin and slot entries in > > the system catalog after the creation has passed or we clean-up

Re: Conflict Detection and Resolution

2024-06-10 Thread shveta malik
On Fri, Jun 7, 2024 at 6:08 PM Tomas Vondra wrote: > > >>> > >>> UPDATE > >>> > >>> > >>> Conflict Detection Method: > >>> > >>> Origin conflict detection: The ‘origin’ info is used to detect > >>> conflict which can be obtained from commit-timest

Re: Conflict Detection and Resolution

2024-06-10 Thread shveta malik
On Fri, Jun 7, 2024 at 6:10 PM Tomas Vondra wrote: > > >>> I don't understand the why should update_missing or update_deleted be > >>> different, especially considering it's not detected reliably. And also > >>> that even if we happen to find the row the associated TOAST data may > >>> have alread

Re: Conflict Detection and Resolution

2024-06-11 Thread shveta malik
On Mon, Jun 10, 2024 at 5:24 PM Tomas Vondra wrote: > > > > On 6/10/24 12:56, shveta malik wrote: > > On Fri, Jun 7, 2024 at 6:08 PM Tomas Vondra > > wrote: > >> > >>>>> > >>>>> UPDA

Re: Conflict Detection and Resolution

2024-06-12 Thread shveta malik
On Tue, Jun 11, 2024 at 7:44 PM Tomas Vondra wrote: > > > > On 6/11/24 10:35, shveta malik wrote: > > On Mon, Jun 10, 2024 at 5:24 PM Tomas Vondra > > wrote: > >> > >> > >> > >> On 6/10/24 12:56, shveta malik wrote: &g

Re: Conflict Detection and Resolution

2024-06-18 Thread shveta malik
On Tue, Jun 18, 2024 at 11:34 AM Dilip Kumar wrote: > > On Tue, Jun 18, 2024 at 10:17 AM Dilip Kumar wrote: > > > > On Mon, Jun 17, 2024 at 3:23 PM Amit Kapila wrote: > > > > > > On Wed, Jun 12, 2024 at 10:03 AM Dilip Kumar > > > wrote: > > > > > > > > On Tue, Jun 11, 2024 at 7:44 PM Tomas Von

Re: Conflict Detection and Resolution

2024-06-19 Thread shveta malik
On Wed, Jun 19, 2024 at 1:52 PM Dilip Kumar wrote: > > On Tue, Jun 18, 2024 at 3:29 PM shveta malik wrote: > > On Tue, Jun 18, 2024 at 11:34 AM Dilip Kumar wrote: > > > > I tried to work out a few scenarios with this, where the apply worker > > will w

Re: Conflict detection and logging in logical replication

2024-06-24 Thread shveta malik
On Mon, Jun 24, 2024 at 7:39 AM Zhijie Hou (Fujitsu) wrote: > > When testing the patch, I noticed a bug that when reporting the conflict > after calling ExecInsertIndexTuples(), we might find the tuple that we > just inserted and report it.(we should only report conflict if there are > other confl

Re: Conflict Detection and Resolution

2024-06-25 Thread shveta malik
On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila wrote: > > On Mon, Jun 24, 2024 at 1:47 PM shveta malik wrote: > > > > On Thu, Jun 20, 2024 at 6:41 PM Amit Kapila wrote: > > > > > > >> In the second patch, we can implement simple built-in resolution >

Re: Conflict Detection and Resolution

2024-06-26 Thread shveta malik
On Wed, Jun 26, 2024 at 2:33 PM Amit Kapila wrote: > > On Tue, Jun 25, 2024 at 3:39 PM shveta malik wrote: > > > > On Tue, Jun 25, 2024 at 3:12 PM Amit Kapila wrote: > > > > > > On Mon, Jun 24, 2024 at 1:47 PM shveta malik > > > wrote: > &g

Re: Conflict Detection and Resolution

2024-07-01 Thread shveta malik
On Mon, Jul 1, 2024 at 11:47 AM Masahiko Sawada wrote: > > Hi, > > On Thu, May 23, 2024 at 3:37 PM shveta malik wrote: > > > > DELETE > > > > Conflict Type: > > > > delete_missing: An incoming delete is trying to

Re: Conflict Detection and Resolution

2024-07-02 Thread shveta malik
On Wed, Jun 19, 2024 at 1:52 PM Dilip Kumar wrote: > > On Tue, Jun 18, 2024 at 3:29 PM shveta malik wrote: > > On Tue, Jun 18, 2024 at 11:34 AM Dilip Kumar wrote: > > > > I tried to work out a few scenarios with this, where the apply worker > > will w

Re: Conflict Detection and Resolution

2024-07-02 Thread shveta malik
On Wed, Jul 3, 2024 at 10:47 AM Dilip Kumar wrote: > > On Tue, Jul 2, 2024 at 2:40 PM shveta malik wrote: > > > > On Wed, Jun 19, 2024 at 1:52 PM Dilip Kumar wrote: > > > > > > On Tue, Jun 18, 2024 at 3:29 PM shveta malik > > > wrote: > &g

Re: Conflict Detection and Resolution

2024-07-03 Thread shveta malik
On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik wrote: > > > > > Yes, I also think it should be independent of CDR. IMHO, it should be > > > based on the user-configured maximum clock skew tolerance and c

Re: Conflict Detection and Resolution

2024-07-03 Thread shveta malik
On Wed, Jul 3, 2024 at 3:35 PM Amit Kapila wrote: > > On Wed, Jul 3, 2024 at 2:16 PM Dilip Kumar wrote: > > > > On Wed, Jul 3, 2024 at 12:30 PM Amit Kapila wrote: > > > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > > But waiting after applying the operations and before applyi

Re: Conflict Detection and Resolution

2024-07-03 Thread shveta malik
On Wed, Jul 3, 2024 at 4:12 PM Dilip Kumar wrote: > > On Wed, Jul 3, 2024 at 4:02 PM shveta malik wrote: > > > > On Wed, Jul 3, 2024 at 11:29 AM Dilip Kumar wrote: > > > > > > On Wed, Jul 3, 2024 at 11:00 AM shveta malik > > > wrote: >

Re: Conflict detection and logging in logical replication

2024-07-10 Thread shveta malik
On Wed, Jul 3, 2024 at 8:31 AM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, June 26, 2024 10:58 AM Zhijie Hou (Fujitsu) > wrote: > > > > Hi, > > As suggested by Sawada-san in another thread[1]. > > I am attaching the V4 patch set which tracks the delete_differ > conflict in logical replication.

Re: Conflict detection and logging in logical replication

2024-07-10 Thread shveta malik
On Thu, Jul 11, 2024 at 7:47 AM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, July 10, 2024 5:39 PM shveta malik > wrote: > > > > On Wed, Jul 3, 2024 at 8:31 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > On Wednesday, June 26,

Re: Conflict detection and logging in logical replication

2024-07-11 Thread shveta malik
On Wed, Jul 10, 2024 at 3:09 PM shveta malik wrote: > > On Wed, Jul 3, 2024 at 8:31 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Wednesday, June 26, 2024 10:58 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > Hi, > > > > As suggeste

Re: Synchronizing slots from primary to standby

2023-07-20 Thread shveta malik
On Fri, Jun 16, 2023 at 3:26 PM Amit Kapila wrote: > > On Mon, Apr 17, 2023 at 7:37 PM Drouvot, Bertrand > wrote: > > > 3. As mentioned in the initial email, I think it would be better to > replace LIST_SLOTS command with a SELECT query. > I had a look at this thread. I am interested to work on

Re: Synchronizing slots from primary to standby

2023-07-21 Thread shveta malik
On Fri, Jul 21, 2023 at 11:36 AM Bharath Rupireddy wrote: > > On Thu, Jul 20, 2023 at 5:05 PM shveta malik wrote: > > > > On Fri, Jun 16, 2023 at 3:26 PM Amit Kapila wrote: > > > > > > On Mon, Apr 17, 2023 at 7:37 PM Drouvot, Bertrand > > > wr

Re: Synchronizing slots from primary to standby

2023-07-25 Thread shveta malik
On Mon, Jul 24, 2023 at 9:00 AM Amit Kapila wrote: > > On Mon, Jul 24, 2023 at 8:03 AM Bharath Rupireddy > wrote: > > > > On Fri, Jul 21, 2023 at 5:16 PM shveta malik wrote: > > > > > > Thanks Bharat for letting us know. It is okay to split the patch, it

Re: Synchronizing slots from primary to standby

2023-07-26 Thread shveta malik
On Mon, Jul 24, 2023 at 8:03 AM Bharath Rupireddy wrote: > > On Fri, Jul 21, 2023 at 5:16 PM shveta malik wrote: > > > > Thanks Bharat for letting us know. It is okay to split the patch, it > > may definitely help to understand the modules better but shall we take &g

Re: Synchronizing slots from primary to standby

2023-07-26 Thread shveta malik
On Thu, Jul 27, 2023 at 10:55 AM Amit Kapila wrote: > > On Wed, Jul 26, 2023 at 10:31 AM shveta malik wrote: > > > > On Mon, Jul 24, 2023 at 9:00 AM Amit Kapila wrote: > > > > > > On Mon, Jul 24, 2023 at 8:03 AM Bharath Rupireddy > > > wrote: >

Re: Synchronizing slots from primary to standby

2023-08-01 Thread shveta malik
On Fri, Jul 28, 2023 at 8:54 PM Bharath Rupireddy wrote: > > On Thu, Jul 27, 2023 at 10:55 AM Amit Kapila wrote: > > > > I wonder if we anyway some sort of design like this because we > > shouldn't allow to spawn as many workers as the number of databases. > > There has to be some existing or new

Re: Synchronizing slots from primary to standby

2023-08-03 Thread shveta malik
On Thu, Aug 3, 2023 at 12:28 AM Bharath Rupireddy wrote: > > On Tue, Aug 1, 2023 at 5:01 PM shveta malik wrote: > > > > > The work division amongst the sync workers can > > > be simple, the logical replication launcher builds a shared memory > > > structu

Re: Synchronizing slots from primary to standby

2023-08-04 Thread shveta malik
On Fri, Aug 4, 2023 at 2:44 PM Drouvot, Bertrand wrote: > > Hi, > > On 7/28/23 4:39 PM, Bharath Rupireddy wrote: > > On Mon, Jul 24, 2023 at 9:00 AM Amit Kapila wrote: > >> > >>> 2. All candidate standbys will start one slot sync worker per logical > >>> slot which might not be scalable. > >> > >

Re: Synchronizing slots from primary to standby

2023-08-07 Thread shveta malik
On Tue, Aug 1, 2023 at 4:52 PM shveta malik wrote: > > On Thu, Jul 27, 2023 at 12:13 PM shveta malik wrote: > > > > On Thu, Jul 27, 2023 at 10:55 AM Amit Kapila > > wrote: > > > > > > On Wed, Jul 26, 2023 at 10:31 AM shveta malik > > > wro

Re: Synchronizing slots from primary to standby

2023-08-07 Thread shveta malik
On Mon, Aug 7, 2023 at 3:17 PM Drouvot, Bertrand wrote: > > Hi, > > On 8/4/23 1:32 PM, shveta malik wrote: > > On Fri, Aug 4, 2023 at 2:44 PM Drouvot, Bertrand > > wrote: > >> On 7/28/23 4:39 PM, Bharath Rupireddy wrote: > > >> Sorry to be late, b

Re: Synchronizing slots from primary to standby

2023-08-14 Thread shveta malik
On Tue, Aug 8, 2023 at 11:11 AM Drouvot, Bertrand wrote: > > Hi, > > On 8/8/23 7:01 AM, shveta malik wrote: > > On Mon, Aug 7, 2023 at 3:17 PM Drouvot, Bertrand > > wrote: > >> > >> Hi, > >> > >> On 8/4/23 1:32 PM, shveta malik w

Re: Synchronizing slots from primary to standby

2023-08-14 Thread shveta malik
On Mon, Aug 14, 2023 at 3:22 PM shveta malik wrote: > > On Tue, Aug 8, 2023 at 11:11 AM Drouvot, Bertrand > wrote: > > > > Hi, > > > > On 8/8/23 7:01 AM, shveta malik wrote: > > > On Mon, Aug 7, 2023 at 3:17 PM Drouvot, Bertrand > > > wrote

Re: Synchronizing slots from primary to standby

2023-08-16 Thread shveta malik
On Thu, Aug 17, 2023 at 11:44 AM Drouvot, Bertrand wrote: > > Hi, > > On 8/14/23 11:52 AM, shveta malik wrote: > > > > > We (myself and Ajin) performed the tests to compute the lag in standby > > slots as compared to primary slots with different number of

Re: Synchronizing slots from primary to standby

2023-08-17 Thread shveta malik
On Thu, Aug 17, 2023 at 11:55 AM shveta malik wrote: > > On Thu, Aug 17, 2023 at 11:44 AM Drouvot, Bertrand > wrote: > > > > Hi, > > > > On 8/14/23 11:52 AM, shveta malik wrote: > > > > > > > > We (myself and Ajin) performed the tests

Re: Adding a LogicalRepWorker type field

2023-08-17 Thread shveta malik
On Fri, Aug 18, 2023 at 8:50 AM Amit Kapila wrote: > > On Mon, Aug 14, 2023 at 12:08 PM Peter Smith wrote: > > > > The main patch for adding the worker type enum has been pushed [1]. > > > > Here is the remaining (rebased) patch for changing some previous > > cascading if/else to switch on the Lo

Re: Synchronizing slots from primary to standby

2023-08-22 Thread shveta malik
On Thu, Aug 17, 2023 at 11:55 AM shveta malik wrote: > > On Thu, Aug 17, 2023 at 11:44 AM Drouvot, Bertrand > wrote: > > > > Hi, > > > > On 8/14/23 11:52 AM, shveta malik wrote: > > > > > > > > We (myself and Ajin) performed the tests

Re: Synchronizing slots from primary to standby

2023-08-23 Thread shveta malik
On Thu, Aug 17, 2023 at 4:09 PM shveta malik wrote: > > On Thu, Aug 17, 2023 at 11:55 AM shveta malik wrote: > > > > On Thu, Aug 17, 2023 at 11:44 AM Drouvot, Bertrand > > wrote: > > > > > > Hi, > > > > > > On 8/14/23 11:52 AM, s

Re: Synchronizing slots from primary to standby

2023-08-24 Thread shveta malik
On Wed, Aug 23, 2023 at 4:21 PM Dilip Kumar wrote: > > On Wed, Aug 23, 2023 at 3:38 PM shveta malik wrote: > > > I have reviewed the v12-0002 patch and I have some comments. I see the > latest version posted sometime back and if any of this comment is > already fixed in t

Re: promotion related handling in pg_sync_replication_slots()

2024-04-11 Thread shveta malik
On Sat, Apr 6, 2024 at 11:49 AM Amit Kapila wrote: > > On Fri, Apr 5, 2024 at 10:31 AM shveta malik wrote: > > > > Please find v2. Changes are: > > 1) Rebased the patch as there was conflict due to recent commit 6f132ed. > > 2) Added an Assert in update_synced_sl

Re: promotion related handling in pg_sync_replication_slots()

2024-04-11 Thread shveta malik
On Sat, Apr 6, 2024 at 12:25 PM Bharath Rupireddy wrote: > > On Fri, Apr 5, 2024 at 10:31 AM shveta malik wrote: > > > > Please find v2. Changes are: > > Thanks for the patch. Here are some comments. Thanks for reviewing. > > 1. Can we have a clear saying in the s

Re: promotion related handling in pg_sync_replication_slots()

2024-04-12 Thread shveta malik
On Fri, Apr 12, 2024 at 4:18 PM Amit Kapila wrote: > > On Fri, Apr 12, 2024 at 7:57 AM shveta malik wrote: > > > > On Sat, Apr 6, 2024 at 12:25 PM Bharath Rupireddy > > wrote: > > > > > > On Fri, Apr 5, 2024 at 10:31 AM shveta malik > > >

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread shveta malik
On Mon, Apr 15, 2024 at 2:29 PM Amit Kapila wrote: > > On Fri, Apr 12, 2024 at 5:25 PM shveta malik wrote: > > > > Please find v3 addressing race-condition and one other comment. > > > > Up-thread it was suggested that, probably, checking > > SlotSync

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread shveta malik
On Tue, Apr 16, 2024 at 9:27 AM Zhijie Hou (Fujitsu) wrote: > > On Monday, April 15, 2024 6:09 PM shveta malik wrote: > > > > Please find v4 addressing the above comments. > > Thanks for the patch. > > Here are few comments: Thanks f

Re: promotion related handling in pg_sync_replication_slots()

2024-04-16 Thread shveta malik
On Tue, Apr 16, 2024 at 12:03 PM Bertrand Drouvot wrote: > > > I think for now let's restrict their usage in parallel and make the > > promotion behavior consistent both for worker and API. > > Okay, let's do it that way. Is it worth to add a few words in the doc related > to > pg_sync_replicatio

Re: promotion related handling in pg_sync_replication_slots()

2024-04-16 Thread shveta malik
On Tue, Apr 16, 2024 at 1:55 PM Zhijie Hou (Fujitsu) wrote: > > On Tuesday, April 16, 2024 2:52 PM Bertrand Drouvot > wrote: > > > Hi, > > > On Tue, Apr 16, 2024 at 10:00:04AM +0530, shveta malik wrote: > > > Please find v5 addressing above comments. > &g

Re: Disallow changing slot's failover option in transaction block

2024-04-16 Thread shveta malik
On Tue, Apr 16, 2024 at 1:45 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Hou, > > > Kuroda-San reported an issue off-list that: > > > > If user execute ALTER SUBSCRIPTION SET (failover) command inside a txn block > > and rollback, only the subscription option change can be rolled back, while > > t

Re: Disallow changing slot's failover option in transaction block

2024-04-17 Thread shveta malik
On Thu, Apr 18, 2024 at 11:22 AM Amit Kapila wrote: > > On Tue, Apr 16, 2024 at 5:06 PM shveta malik wrote: > > > > On Tue, Apr 16, 2024 at 1:45 PM Hayato Kuroda (Fujitsu) > > wrote: > > > > > > Dear Hou, > > > > > > > Kuroda-San r

Re: promotion related handling in pg_sync_replication_slots()

2024-04-18 Thread shveta malik
On Tue, Apr 16, 2024 at 2:52 PM Bertrand Drouvot wrote: > > Hi, > > On Tue, Apr 16, 2024 at 02:06:45PM +0530, shveta malik wrote: > > On Tue, Apr 16, 2024 at 1:55 PM Zhijie Hou (Fujitsu) > > wrote: > > > I personally feel adding the additional check for

Re: Disallow changing slot's failover option in transaction block

2024-04-18 Thread shveta malik
On Thu, Apr 18, 2024 at 11:40 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Shveta, > > Sorry for delay response. I missed your post. > > > +1. > > > > Similar to ALTER SUB, CREATE SUB also needs to be fixed. This is > > because we call alter_replication_slot in CREATE SUB as well, for the > > case w

Re: promotion related handling in pg_sync_replication_slots()

2024-04-18 Thread shveta malik
On Thu, Apr 18, 2024 at 12:35 PM shveta malik wrote: > > To fix above issues, these changes have been made in v7: Please find v8 attached. Changes are: 1) It fixes ShutDownSlotSync() issue, where we perform kill(SlotSyncCtx->pid). There are chances that after we release spin-lock and

Re: promotion related handling in pg_sync_replication_slots()

2024-04-18 Thread shveta malik
On Fri, Apr 19, 2024 at 10:53 AM Bertrand Drouvot wrote: > > Hi, > > On Thu, Apr 18, 2024 at 05:36:05PM +0530, shveta malik wrote: > > Please find v8 attached. Changes are: > > Thanks! > > A few comments: Thanks for reviewing. > 1 === > > @@ -1440,7

Re: promotion related handling in pg_sync_replication_slots()

2024-04-19 Thread shveta malik
On Fri, Apr 19, 2024 at 11:37 AM shveta malik wrote: > > On Fri, Apr 19, 2024 at 10:53 AM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Thu, Apr 18, 2024 at 05:36:05PM +0530, shveta malik wrote: > > > Please find v8 attached. Changes are: > > &g

Re: Disallow changing slot's failover option in transaction block

2024-04-19 Thread shveta malik
On Fri, Apr 19, 2024 at 6:09 AM Zhijie Hou (Fujitsu) wrote: > > Here is V2 patch which includes the changes for CREATE SUBSCRIPTION as > suggested. Since we don't connect pub to alter slot when (create_slot=false) > anymore, the restriction that disallows failover=true when connect=false is > also

Re: Disallow changing slot's failover option in transaction block

2024-04-21 Thread shveta malik
On Mon, Apr 22, 2024 at 5:57 AM Zhijie Hou (Fujitsu) wrote: > > On Friday, April 19, 2024 10:54 AM Kuroda, Hayato/黒田 隼人 > wrote: > > In your patch, the pg_dump.c was updated. IIUC the main reason was that > > pg_restore executes some queries as single transactions so that ALTER > > SUBSCRIPTION

Re: promotion related handling in pg_sync_replication_slots()

2024-04-22 Thread shveta malik
On Mon, Apr 22, 2024 at 5:10 PM Amit Kapila wrote: > > On Fri, Apr 19, 2024 at 1:52 PM shveta malik wrote: > > > > Please find v9 with the above comments addressed. > > > > I have made minor modifications in the comments and a function name. > Please see the

Re: promotion related handling in pg_sync_replication_slots()

2024-04-23 Thread shveta malik
On Tue, Apr 23, 2024 at 9:07 AM Amit Kapila wrote: > > On Mon, Apr 22, 2024 at 7:04 PM Masahiko Sawada wrote: > > > > On Mon, Apr 22, 2024 at 9:02 PM shveta malik wrote: > > > > > > Thanks for the patch, the changes look good Amit. Please find the mer

Re: Synchronizing slots from primary to standby

2024-04-28 Thread shveta malik
On Mon, Apr 29, 2024 at 10:57 AM Zhijie Hou (Fujitsu) wrote: > > On Friday, March 15, 2024 10:45 PM Bertrand Drouvot > wrote: > > > > Hi, > > > > On Thu, Mar 14, 2024 at 02:22:44AM +, Zhijie Hou (Fujitsu) wrote: > > > Hi, > > > > > > Since the standby_slot_names patch has been committed, I a

Re: Synchronizing slots from primary to standby

2024-04-29 Thread shveta malik
On Mon, Apr 29, 2024 at 11:38 AM shveta malik wrote: > > On Mon, Apr 29, 2024 at 10:57 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Friday, March 15, 2024 10:45 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Th

Re: Synchronizing slots from primary to standby

2024-04-29 Thread shveta malik
On Mon, Apr 29, 2024 at 11:38 AM shveta malik wrote: > > On Mon, Apr 29, 2024 at 10:57 AM Zhijie Hou (Fujitsu) > wrote: > > > > On Friday, March 15, 2024 10:45 PM Bertrand Drouvot > > wrote: > > > > > > Hi, > > > > > > On Th

Re: Synchronizing slots from primary to standby

2024-04-29 Thread shveta malik
On Mon, Apr 29, 2024 at 5:28 PM Zhijie Hou (Fujitsu) wrote: > > On Monday, April 29, 2024 5:11 PM shveta malik wrote: > > > > On Mon, Apr 29, 2024 at 11:38 AM shveta malik > > wrote: > > > > > > On Mon, Apr 29, 2024 at 10:57 AM Zhijie Hou (Fujitsu) >

Conflict Detection and Resolution

2024-05-22 Thread shveta malik
Hello hackers, Please find the proposal for Conflict Detection and Resolution (CDR) for Logical replication. Introduction In case the node is subscribed to multiple providers, or when local writes happen on a subscriber, conflicts can arise for the incoming changes. CDR is the

<    1   2   3   4   5   6   >