Re: Restrict publishing of partitioned table with a foreign table as partition

2025-06-08 Thread Shlok Kyal
On Wed, 4 Jun 2025 at 16:12, Ajin Cherian wrote: > > On Tue, May 20, 2025 at 2:33 AM Shlok Kyal wrote: > > > > This approach seems better to me. I have created a patch with the > > above approach. > > > > Thanks and Regards, > > Shlok Kyal > > Some quick comments on the patch: > 1. In doc/src/sgm

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-06-04 Thread Ajin Cherian
On Tue, May 20, 2025 at 2:33 AM Shlok Kyal wrote: > > This approach seems better to me. I have created a patch with the > above approach. > > Thanks and Regards, > Shlok Kyal Some quick comments on the patch: 1. In doc/src/sgml/ref/create_subscription.sgml: +has partitioned table with foreign

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-05-19 Thread Shlok Kyal
On Thu, 15 May 2025 at 18:19, Amit Kapila wrote: > > On Sun, May 11, 2025 at 6:53 AM Álvaro Herrera > wrote: > > > > But the non-idiomatic locking of pg_partitioned_table appears to > > continue to be the pain point of this patch. My impression is that > > using a lock is the wrong approach to

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-05-15 Thread Amit Kapila
On Sun, May 11, 2025 at 6:53 AM Álvaro Herrera wrote: > > But the non-idiomatic locking of pg_partitioned_table appears to > continue to be the pain point of this patch. My impression is that > using a lock is the wrong approach to solve the concurrency problem. > Maybe we can use a ConditionVari

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-05-10 Thread Álvaro Herrera
Hello, I think reimplementing list_member_oid() under a different name (is_ancestor_member_relids) is pointless and should not be done. It also appears to me that we haven't nailed the error messages just yet. I tried to fix it upthread, but didn't really get it correct. For instance, consider

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-05-09 Thread Shlok Kyal
On Mon, 28 Apr 2025 at 19:57, Álvaro Herrera wrote: > > On 2025-Apr-28, Shlok Kyal wrote: > > > 2. > > + * We also take a ShareLock on pg_partitioned_table to restrict addition > > + * of new partitioned table which may contain a foreign partition while > > + * publication is being created. XXX

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-28 Thread Álvaro Herrera
On 2025-Apr-28, Shlok Kyal wrote: > 2. > + * We also take a ShareLock on pg_partitioned_table to restrict addition > + * of new partitioned table which may contain a foreign partition while > + * publication is being created. XXX this is quite weird actually. > > This change was added to resolv

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-28 Thread Shlok Kyal
On Mon, 7 Apr 2025 at 09:43, Sergey Tatarintsev wrote: > > 07.04.2025 03:27, Álvaro Herrera пишет: > > On 2025-Apr-01, Shlok Kyal wrote: > > I have modified the comment in create_publication.sgml and also added > comment in the restrictions section of logical-replication.sgml. > I have also added

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-28 Thread Shlok Kyal
On Mon, 7 Apr 2025 at 18:09, Álvaro Herrera wrote: > > Here's the additional changes I made here before giving up on this. > I think it needs some additional rethinking, not going to happen for 18. > Hi Alvaro, Thanks for reviewing the patch. The changes shared by you in [1], look good to me an

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-10 Thread Álvaro Herrera
Here's the additional changes I made here before giving up on this. I think it needs some additional rethinking, not going to happen for 18. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "The Gord often wonders why people threaten never to come back after they'v

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-07 Thread Álvaro Herrera
On 2025-Apr-07, Sergey Tatarintsev wrote: > I think this is a wrong assumption: > > ScanKeyInit(&key[keycount++], Anum_pg_class_relispartition, > BTEqualStrategyNumber, F_BOOLEQ, BoolGetDatum(false)); > > In this case sch5.part1 is partitioned table, but it also partition of table > in different

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-07 Thread Álvaro Herrera
As promised, here's a rundown of the changes I did, mostly in order the patch shows them: - I reworded the documentation changes to read more coherent with the surrounding text. - It seemed wrong to have check_publication_add_relation() have the relation first as argument and publication later, s

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-06 Thread Sergey Tatarintsev
07.04.2025 03:27, Álvaro Herrera пишет: On 2025-Apr-01, Shlok Kyal wrote: I have modified the comment in create_publication.sgml and also added comment in the restrictions section of logical-replication.sgml. I have also added a more detailed explanation in comment of 'check_foreign_tables' I

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-06 Thread Álvaro Herrera
On 2025-Apr-01, Shlok Kyal wrote: > I have modified the comment in create_publication.sgml and also added > comment in the restrictions section of logical-replication.sgml. > I have also added a more detailed explanation in comment of > 'check_foreign_tables' > > I have attached the updated v11 p

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-05 Thread Shlok Kyal
On Fri, 4 Apr 2025 at 10:36, Sergey Tatarintsev wrote: > > 01.04.2025 21:48, Shlok Kyal пишет: > > On Fri, 28 Mar 2025 at 16:35, Álvaro Herrera > > wrote: > >> On 2025-Mar-28, Shlok Kyal wrote: > >> > >>> On Mon, 24 Mar 2025 at 21:17, Álvaro Herrera > >>> wrote: > Also, surely we should d

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-03 Thread Sergey Tatarintsev
01.04.2025 21:48, Shlok Kyal пишет: On Fri, 28 Mar 2025 at 16:35, Álvaro Herrera wrote: On 2025-Mar-28, Shlok Kyal wrote: On Mon, 24 Mar 2025 at 21:17, Álvaro Herrera wrote: Also, surely we should document this restriction in the SGML docs somewhere. I have added comment in create_publicat

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-04-01 Thread Shlok Kyal
On Fri, 28 Mar 2025 at 16:35, Álvaro Herrera wrote: > > On 2025-Mar-28, Shlok Kyal wrote: > > > On Mon, 24 Mar 2025 at 21:17, Álvaro Herrera > > wrote: > > > > Also, surely we should document this restriction in the SGML docs > > > somewhere. > > > > I have added comment in create_publication.sg

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-03-28 Thread Álvaro Herrera
On 2025-Mar-28, Shlok Kyal wrote: > On Mon, 24 Mar 2025 at 21:17, Álvaro Herrera wrote: > > Also, surely we should document this restriction in the SGML docs > > somewhere. > > I have added comment in create_publication.sgml Hmm, okay, but "We cannot" is not the style used in the documentation.

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-03-28 Thread Shlok Kyal
On Mon, 24 Mar 2025 at 21:17, Álvaro Herrera wrote: > > One thing that bothers me a bit about this is that there's no single > code comment where this restriction it documented in full; in fact it > doesn't seem documented anywhere, only in the commit message. > > I think check_foreign_tables() is

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-03-24 Thread Álvaro Herrera
One thing that bothers me a bit about this is that there's no single code comment where this restriction it documented in full; in fact it doesn't seem documented anywhere, only in the commit message. I think check_foreign_tables() is a good place to add an explanatory comment; other places can re

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-03-24 Thread Shlok Kyal
On Thu, 20 Feb 2025 at 21:18, vignesh C wrote: > > On Tue, 18 Feb 2025 at 15:59, Shlok Kyal wrote: > > > > On Mon, 17 Feb 2025 at 20:13, vignesh C wrote: > > > > > > On Fri, 14 Feb 2025 at 12:59, Shlok Kyal wrote: > > > > > > > > I have used the changes suggested by you. Also I have updated the

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-20 Thread vignesh C
On Tue, 18 Feb 2025 at 15:59, Shlok Kyal wrote: > > On Mon, 17 Feb 2025 at 20:13, vignesh C wrote: > > > > On Fri, 14 Feb 2025 at 12:59, Shlok Kyal wrote: > > > > > > I have used the changes suggested by you. Also I have updated the > > > comments and the function name. > > > > There is another

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-18 Thread Shlok Kyal
On Mon, 17 Feb 2025 at 20:13, vignesh C wrote: > > On Fri, 14 Feb 2025 at 12:59, Shlok Kyal wrote: > > > > I have used the changes suggested by you. Also I have updated the > > comments and the function name. > > There is another concurrency issue possible: > +/* Check if a partitioned table has

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-17 Thread vignesh C
On Fri, 14 Feb 2025 at 12:59, Shlok Kyal wrote: > > I have used the changes suggested by you. Also I have updated the > comments and the function name. There is another concurrency issue possible: +/* Check if a partitioned table has a foreign partition */ +bool +check_partrel_has_foreign_table(F

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-13 Thread Shlok Kyal
On Thu, 13 Feb 2025 at 20:12, vignesh C wrote: > > On Thu, 13 Feb 2025 at 15:50, Shlok Kyal wrote: > > > > > > I have fixed the issue. Attached the updated v6 patch. > > There is another concurrency issue: > In case of create publication for all tables with > publish_via_partition_root we will ca

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-13 Thread vignesh C
On Thu, 13 Feb 2025 at 15:50, Shlok Kyal wrote: > > > I have fixed the issue. Attached the updated v6 patch. There is another concurrency issue: In case of create publication for all tables with publish_via_partition_root we will call check_foreign_tables: @@ -876,6 +876,10 @@ CreatePublication(P

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-13 Thread Shlok Kyal
On Thu, 13 Feb 2025 at 11:25, vignesh C wrote: > > On Tue, 11 Feb 2025 at 16:55, Shlok Kyal wrote: > > I have handled the above cases and added tests for the same. > > There is a concurrency issue with the patch: > +check_partrel_has_foreign_table(Form_pg_class relform) > +{ > + bool

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-12 Thread vignesh C
On Tue, 11 Feb 2025 at 16:55, Shlok Kyal wrote: > I have handled the above cases and added tests for the same. There is a concurrency issue with the patch: +check_partrel_has_foreign_table(Form_pg_class relform) +{ + boolhas_foreign_tbl = false; + + if (relform->relkind ==

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-11 Thread Shlok Kyal
On Mon, 10 Feb 2025 at 16:11, vignesh C wrote: > > On Tue, 4 Feb 2025 at 18:31, vignesh C wrote: > > > > On Thu, 30 Jan 2025 at 17:32, Shlok Kyal wrote: > > > > > > @@ -1428,6 +1427,12 @@ check_foreign_tables_in_schema(Oid schemaid) > > > errdetail("foreign table \"%s\" is a

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-11 Thread Shlok Kyal
On Wed, 5 Feb 2025 at 14:14, Álvaro Herrera wrote: > > On 2025-Feb-05, vignesh C wrote: > > > We can maintain the behavior you suggested when the > > PUBLISH_VIA_PARTITION_ROOT option is set to false. However, when > > PUBLISH_VIA_PARTITION_ROOT is true, the table data is copied from the > > root

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-10 Thread vignesh C
On Tue, 4 Feb 2025 at 18:31, vignesh C wrote: > > On Thu, 30 Jan 2025 at 17:32, Shlok Kyal wrote: > > > > @@ -1428,6 +1427,12 @@ check_foreign_tables_in_schema(Oid schemaid) > > errdetail("foreign table \"%s\" is a partition of > > partitioned table \"%s\"", > >

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-05 Thread Álvaro Herrera
On 2025-Feb-05, vignesh C wrote: > We can maintain the behavior you suggested when the > PUBLISH_VIA_PARTITION_ROOT option is set to false. However, when > PUBLISH_VIA_PARTITION_ROOT is true, the table data is copied from the > root table (as intended by the user), which will also include the > fo

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-04 Thread vignesh C
On Tue, 4 Feb 2025 at 21:21, Álvaro Herrera wrote: > > On 2025-Feb-04, vignesh C wrote: > > > We should throw an error for partitioned tables that contain foreign > > partitions, as this would include the data from these foreign tables > > during the initial sync, while incremental changes would n

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-04 Thread Álvaro Herrera
On 2025-Feb-04, vignesh C wrote: > We should throw an error for partitioned tables that contain foreign > partitions, as this would include the data from these foreign tables > during the initial sync, while incremental changes would not be > replicated. Hmm, I would support the idea of allowing

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-04 Thread vignesh C
On Thu, 30 Jan 2025 at 17:32, Shlok Kyal wrote: > > @@ -1428,6 +1427,12 @@ check_foreign_tables_in_schema(Oid schemaid) > errdetail("foreign table \"%s\" is a partition of > partitioned table \"%s\"", >get_rel_name(foreign_tbl_relid), > parent_

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-04 Thread vignesh C
On Fri, 31 Jan 2025 at 10:38, Sergey Tatarintsev wrote: > > Ok, but maybe it will be correct to raise an WARNING (or at least LOG) > that some tables was skipped during publication. What do you think? > > And I think we need check tables which was really published in case of > 'FOR ALL TABLES' and

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-01-30 Thread Sergey Tatarintsev
30.01.2025 19:02, Shlok Kyal пишет: On Wed, 29 Jan 2025 at 19:21, Sergey Tatarintsev wrote: 29.01.2025 12:16, Shlok Kyal пишет: Hi, As part of a discussion in [1], I am starting this thread to address the issue reported for foreign tables. Logical replication of foreign tables is not suppo

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-01-30 Thread Shlok Kyal
On Wed, 29 Jan 2025 at 19:21, Sergey Tatarintsev wrote: > > > 29.01.2025 12:16, Shlok Kyal пишет: > > Hi, > > > > As part of a discussion in [1], I am starting this thread to address > > the issue reported for foreign tables. > > > > Logical replication of foreign tables is not supported, and we t

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-01-29 Thread Sergey Tatarintsev
29.01.2025 12:16, Shlok Kyal пишет: Hi, As part of a discussion in [1], I am starting this thread to address the issue reported for foreign tables. Logical replication of foreign tables is not supported, and we throw an error in this case. But when we create a publication on a partitioned tabl