Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-18 Thread Amit Kapila
On Thu, Sep 18, 2025 at 1:07 PM Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > > I considered a test, please see attached files. > Few comments: 1. +step "s0_compare" { +SELECT s0.lsn < s1.lsn +FROM local_lsn_store as s0, local_lsn_store as s1 +WHERE s0.session = 0 AND s1.sessio

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-18 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > Few comments: > 1. +step "s0_compare" { > +SELECT s0.lsn < s1.lsn > +FROM local_lsn_store as s0, local_lsn_store as s1 > +WHERE s0.session = 0 AND s1.session = 1; > +} > > This appears to be a bit tricky to compare the values. Doing a > sequential scan won't guarantee

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-18 Thread Hayato Kuroda (Fujitsu)
Dear hackers, > I considered a test, please see attached files. 0001 was not changed from v6 > and > 0002 contained tests. Here, two sessions were opened and confirmed that they > can > set the same origin. After considering and verifying more, it is more efficient to test via isolation tester.

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-17 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > Can we think of writing a few tests for this newly exposed functionality? I considered a test, please see attached files. 0001 was not changed from v6 and 0002 contained tests. Here, two sessions were opened and confirmed that they can set the same origin. BTW, while testing I found

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-16 Thread Amit Kapila
On Tue, Sep 16, 2025 at 10:07 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Doruk, > > Thanks for updating the patch and sorry for being late. > The new patch looks good to me. > Can we think of writing a few tests for this newly exposed functionality? -- With Regards, Amit Kapila.

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-16 Thread Doruk Yilmaz
Dear Hayato, Thanks for the feedback on the patch, I'm glad the latest version looks good. I was wondering if there is anything else I need to do on my end, or any other process I should be aware of, for this patch to move forward? I'm happy to make any further adjustments or provide more informat

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-15 Thread Hayato Kuroda (Fujitsu)
Dear Doruk, Thanks for updating the patch and sorry for being late. The new patch looks good to me. Best regards, Hayato Kuroda FUJITSU LIMITED

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-08 Thread Doruk Yilmaz
> Then why didn't you specified PARALLEL UNSAFE as well? You are correct, I missed marking the function as PARALLEL UNSAFE. I’ve attached a revised patch with the correct annotation. > BTW, yesterday a new thread started with the same requirement [1]. It > uses a slightly different way to define

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-05 Thread Amit Kapila
On Wed, Sep 3, 2025 at 6:13 PM Doruk Yilmaz wrote: > > Dear Hayato, > > > So, your python process establishes two connections, for publisher > > (replication connection) > > and subscriber (normal connection). It receives changes from the publisher, > > constructs SQL statements from the received

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-09-03 Thread Doruk Yilmaz
Dear Hayato, > So, your python process establishes two connections, for publisher > (replication connection) > and subscriber (normal connection). It receives changes from the publisher, > constructs SQL statements from the received results, and sends to subscriber's > backend, is it right? Actu

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-25 Thread Hayato Kuroda (Fujitsu)
Dear Doruk, > In our system the workers aren't background workers and we don't ship > a server-side extension; they're plain external processes (Python in > our case) talking over standard database connections. In many > deployments -especially managed Postgres- we can't load custom C code > even

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-22 Thread Doruk Yilmaz
Dear Hayato, > Can you explain more why we must extend the SQL interface? In our system the workers aren't background workers and we don't ship a server-side extension; they're plain external processes (Python in our case) talking over standard database connections. In many deployments -especiall

RE: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-21 Thread Hayato Kuroda (Fujitsu)
Dear Doruk, > That sounds reasonable. I’ve updated the patch and added more > information to the documentation covering the topics you mentioned. > I also added a Caution block so potential users won’t miss it. I hope > this patch meets your expectations. Can you explain more why we must extend t

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-18 Thread Doruk Yilmaz
> Your use looks good to me. So, maybe we can update the docs with the > dangers if the users of API doesn't follow commit order then it may > lead to data inconsistency should be sufficient. Additionally, we may > want to give an example as to how to use this API for parallel apply. That sounds r

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-12 Thread Amit Kapila
On Mon, Aug 11, 2025 at 10:41 PM Doruk Yilmaz wrote: > > On Mon, Aug 11, 2025 at 9:44 AM Amit Kapila wrote: > > How do you advance the origin? Did you use > > > pg_replication_origin_advance()? If so, you should be aware that it > > can be used for initial setup; see comment in that API code...

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-11 Thread Doruk Yilmaz
On Mon, Aug 11, 2025 at 9:44 AM Amit Kapila wrote: > How do you advance the origin? Did you use > pg_replication_origin_advance()? > If so, you should be aware that it > can be used for initial setup; see comment in that API code... No, we don't use pg_replication_origin_advance(). We use pg_rep

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-08-10 Thread Amit Kapila
On Wed, Jul 30, 2025 at 12:00 AM Doruk Yilmaz wrote: > > On Mon, Jul 29, 2025 at 8:13 AM Amit Kapila wrote: > > That is true but I still feel there has to be some mechanism where we > > can catch and give an ERROR to the user, if it doesn't follow the > > same. For example, pg_replication_origin_

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-29 Thread Doruk Yilmaz
On Mon, Jul 29, 2025 at 8:13 AM Amit Kapila wrote: > That is true but I still feel there has to be some mechanism where we > can catch and give an ERROR to the user, if it doesn't follow the > same. For example, pg_replication_origin_advance() always allows going > backwards in terms of LSN which

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-28 Thread Amit Kapila
On Tue, Jul 29, 2025 at 2:43 AM Doruk Yilmaz wrote: > > On Mon, Mar 3, 2025 at 6:39 AM Amit Kapila wrote: > > > > To use replication_origin by multiple processes, one must maintain the > > commit order as we do internally by allowing the leader process to > > wait for the parallel worker to finis

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-07-28 Thread Doruk Yilmaz
On Mon, Mar 3, 2025 at 6:39 AM Amit Kapila wrote: > > To use replication_origin by multiple processes, one must maintain the > commit order as we do internally by allowing the leader process to > wait for the parallel worker to finish the commit. See comments atop > replorigin_session_setup(). Now

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-03-02 Thread Amit Kapila
On Thu, Jan 9, 2025 at 3:26 AM Euler Taveira wrote: > > On Thu, Aug 15, 2024, at 5:53 PM, Doruk Yilmaz wrote: > > Hello again, > > On Tue, Aug 13, 2024 at 12:48 AM Euler Taveira wrote: > > I'm curious about your use case. Is it just because the internal function > > has a > > different signature

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-03-01 Thread Doruk Yilmaz
I noticed that the patch needs rebasing, so here is the rebased version. Hopefully it makes to the commitfest. Doruk Yılmaz From 74a74fd02bce786093c19a23bef9444d0b8ef41d Mon Sep 17 00:00:00 2001 From: Doruk Date: Thu, 15 Aug 2024 23:34:26 +0300 Subject: [PATCH v4] pg_replication_origin_session_se

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-01-08 Thread Euler Taveira
On Thu, Aug 15, 2024, at 5:53 PM, Doruk Yilmaz wrote: > Hello again, > > On Tue, Aug 13, 2024 at 12:48 AM Euler Taveira wrote: > > I'm curious about your use case. Is it just because the internal function > > has a > > different signature or your tool is capable of apply logical replication > >

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2024-08-15 Thread Doruk Yilmaz
is covered by CI [2][3]. Sadly I still can't log in to the Commitfest due to the cool-off period. I will create an entry as soon as this period ends. Thanks for all the feedback, Doruk Yılmaz From b9c54f3d217f67c24ce74ffa7c1f2812d784333e Mon Sep 17 00:00:00 2001 From: Doruk Date: Thu, 15 Aug

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2024-08-12 Thread Euler Taveira
On Mon, Aug 12, 2024, at 3:43 PM, Doruk Yilmaz wrote: > Hello all, Hi! > While working on our internal tools that utilise replication, we > realised that a new parameter was added to the internal C function > corresponding to pg_replication_origin_session_setup. > However this parameter wasn't in

[Patch] add new parameter to pg_replication_origin_session_setup

2024-08-12 Thread Doruk Yilmaz
Hello all, While working on our internal tools that utilise replication, we realised that a new parameter was added to the internal C function corresponding to pg_replication_origin_session_setup. However this parameter wasn't included in the user-facing API [1]. In 'src/backend/replication/logica