RE: tablesync copy ignores publication actions

2022-06-15 Thread shiy.f...@fujitsu.com
On Tue, Jun 14, 2022 3:36 PM Peter Smith wrote: > > PSA v2 of the patch, based on all feedback received. > > ~~~ > > Main differences from v1: > > * Rewording and more explanatory text. > > * The examples were moved to the "Subscription" [1] page and also > extended to show some normal replic

RE: Replica Identity check of partition table on subscriber

2022-06-15 Thread shiy.f...@fujitsu.com
On Wed, Jun 15, 2022 8:30 PM Amit Kapila wrote: > > I have pushed the first bug-fix patch today. > Thanks. Attached the remaining patches which are rebased. Regards, Shi yu v9-0002-fix-memory-leak-about-attrmap.patch Description: v9-0002-fix-memory-leak-about-attrmap.patch v9-0001-Check-

RE: Replica Identity check of partition table on subscriber

2022-06-16 Thread shiy.f...@fujitsu.com
On Thu, Jun 16, 2022 2:13 PM Amit Langote wrote: > > Hi, > > On Thu, Jun 16, 2022 at 2:07 PM shiy.f...@fujitsu.com > wrote: > > On Wed, Jun 15, 2022 8:30 PM Amit Kapila > wrote: > > > I have pushed the first bug-fix patch today. > > > > Att

RE: Replica Identity check of partition table on subscriber

2022-06-16 Thread shiy.f...@fujitsu.com
On Fri Jun 17, 2022 11:06 AM shiy.f...@fujitsu.com wrote: > > Attached the new version of patch set. I also moved the partitioned table > check > in logicalrep_rel_mark_updatable() to check_relation_updatable() as > discussed > [2]. > Attached back-branch patches of the

RE: Handle infinite recursion in logical replication setup

2022-06-19 Thread shiy.f...@fujitsu.com
On Thu, Jun 16, 2022 6:18 PM vignesh C wrote: > > Thanks for the comments, the attached v21 patch has the changes for the > same. > Thanks for updating the patch. Here are some comments. 0002 patch == 1. + publisher to only send changes that originated locally. Setting +

RE: Replica Identity check of partition table on subscriber

2022-06-19 Thread shiy.f...@fujitsu.com
On Mon, Jun 20, 2022 1:33 PM Amit Kapila wrote: > > On Fri, Jun 17, 2022 at 11:22 AM shiy.f...@fujitsu.com > wrote: > > > > On Fri Jun 17, 2022 11:06 AM shiy.f...@fujitsu.com > wrote: > > > > > > Attached the new version of patch set. I also moved

RE: Replica Identity check of partition table on subscriber

2022-06-21 Thread shiy.f...@fujitsu.com
On Tuesday, June 21, 2022 4:49 PM Amit Kapila wrote: > > On Tue, Jun 21, 2022 at 12:50 PM Amit Langote > wrote: > > > > On Tue, Jun 21, 2022 at 3:35 PM houzj.f...@fujitsu.com > > wrote: > > > > Attached a patch containing the above to consider as an alternative. > > > > Thanks, the patch looks

RE: Handle infinite recursion in logical replication setup

2022-06-22 Thread shiy.f...@fujitsu.com
On Mon, Jun 20, 2022 7:55 PM vignesh C wrote: > > Thanks for the comment, the v22 patch attached has the changes for the > same. Thanks for updating the patch, here are some comments on 0003 patch. 1. 032_origin.pl +###

RE: tablesync copy ignores publication actions

2022-06-22 Thread shiy.f...@fujitsu.com
On Wed, Jun 22, 2022 4:49 PM Peter Smith wrote: > > On Wed, Jun 22, 2022 at 2:18 PM Amit Kapila > wrote: > > > > On Thu, Jun 16, 2022 at 6:07 AM Peter Smith > wrote: > > > > > > > > Thank you for your review comments. Those reported mistakes are fixed > > > in the attached patch v3. > > > > > >

RE: Handle infinite recursion in logical replication setup

2022-06-29 Thread shiy.f...@fujitsu.com
On Tue, Jun 28, 2022 2:18 PM vignesh C wrote: > > Thanks for the comments, the attached v25 patch has the changes for the > same. > Thanks for updating the patch. Here are some comments. 0002 patch: == 1. +# Test the CREATE SUBSCRIPTION 'origin' parameter and its interaction with

RE: Handle infinite recursion in logical replication setup

2022-07-04 Thread shiy.f...@fujitsu.com
On Sun, Jul 3, 2022 11:00 PM vignesh C wrote: > > Thanks for the comments, the attached v27 patch has the changes for the > same. > Thanks for updating the patch. A comment on 0003 patch: + /* +* No need to throw an error for the tables that are in ready state, +

RE: Handle infinite recursion in logical replication setup

2022-07-04 Thread shiy.f...@fujitsu.com
On Mon, Jul 4, 2022 4:17 PM shiy.f...@fujitsu.com wrote: > > On Sun, Jul 3, 2022 11:00 PM vignesh C wrote: > > > > Thanks for the comments, the attached v27 patch has the changes for the > > same. > > > > Thanks for updating the patch. > > A comment

RE: Perform streaming logical transactions by background workers and parallel apply

2022-07-06 Thread shiy.f...@fujitsu.com
On Tue, Jun 28, 2022 11:22 AM Wang, Wei/王 威 wrote: > > I also improved patches as suggested by Peter-san in [1] and [2]. > Thanks for Shi Yu to improve the patches by addressing the comments in [2]. > > Attach the new patches. > Thanks for updating the patch. Here are some comments. 0001 pat

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-11 Thread shiy.f...@fujitsu.com
On Tue, Jul 12, 2022 8:49 AM Masahiko Sawada wrote: > > I've attached an updated patch. > > While trying this idea, I noticed there is no API to get the length of > dlist, as we discussed offlist. Alternative idea was to use List > (T_XidList) but I'm not sure it's a great idea since deleting an

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-12 Thread shiy.f...@fujitsu.com
On Tue, Jul 12, 2022 8:49 AM Masahiko Sawada wrote: > > I've attached an updated patch. > Hi, I met a segmentation fault in test_decoding test after applying the patch for master branch. Attach the backtrace. It happened when executing the following code because it tried to free a NULL point

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-13 Thread shiy.f...@fujitsu.com
On Tue, Jul 12, 2022 5:23 PM Masahiko Sawada wrote: > > On Tue, Jul 12, 2022 at 5:58 PM shiy.f...@fujitsu.com > wrote: > > > > It happened when executing the following code because it tried to free a > NULL > > pointer (catchange_xip). > > > >

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-14 Thread shiy.f...@fujitsu.com
On Mon, Jul 11, 2022 9:54 PM Masahiko Sawada wrote: > > I've attached an updated patch, please review it. > Thanks for your patch. Here are some comments for the REL14-v1 patch. 1. + Sizesz = sizeof(TransactionId) * nxacts;; There is a redundant semicolon at the end.

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-17 Thread shiy.f...@fujitsu.com
On Fri, Jul 15, 2022 10:39 PM Masahiko Sawada wrote: > > This patch should have the fix for the issue that Shi yu reported. Shi > yu, could you please test it again with this patch? > Thanks for updating the patch! I have tested and confirmed that the problem I found has been fixed. Regards, S

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-25 Thread shiy.f...@fujitsu.com
Hi, I did some performance test for the master branch patch (based on v6 patch) to see if the bsearch() added by this patch will cause any overhead. I tested them three times and took the average. The results are as follows, and attach the bar chart. case 1 - No catalog modifying transa

RE: Handle infinite recursion in logical replication setup

2022-07-26 Thread shiy.f...@fujitsu.com
On Sun, Jul 24, 2022 1:28 AM vignesh C wrote: > > Added a note for the same and referred it to the conflicts section. > > Thanks for the comments, the attached v38 patch has the changes for the > same. > Thanks for updating the patch. A comment on the test in 0001 patch. +# Alter subscriptio

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-26 Thread shiy.f...@fujitsu.com
On Tue, Jul 26, 2022 3:52 PM Masahiko Sawada wrote: > > On Tue, Jul 26, 2022 at 2:18 PM Amit Kapila > wrote: > > > > On Tue, Jul 26, 2022 at 7:00 AM Masahiko Sawada > wrote: > > > > > > On Mon, Jul 25, 2022 at 7:57 PM shiy.f...@fujitsu.com &g

RE: Introduce wait_for_subscription_sync for TAP tests

2022-07-27 Thread shiy.f...@fujitsu.com
On Tue, Jul 26, 2022 3:42 PM Masahiko Sawada wrote: > > I've attached an updated patch as well as a patch to remove duplicated > waits in 007_ddl.pl. > Thanks for your patch. Here are some comments. 1. I think some comments need to be changed in the patch. For example: # Also wait for initial

RE: Handle infinite recursion in logical replication setup

2022-07-31 Thread shiy.f...@fujitsu.com
On Fri, Jul 29, 2022 1:22 PM vignesh C wrote: > > > Thanks for the comments, the attached v41 patch has the changes for the > same. > Thanks for updating the patch. I wonder in the case that the publisher uses PG15 (or before), subscriber uses PG16, should we have this check (check if publica

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-08-02 Thread shiy.f...@fujitsu.com
On Mon, Aug 1, 2022 10:31 PM Amit Kapila wrote: > > On Mon, Aug 1, 2022 at 7:46 AM Masahiko Sawada > wrote: > > > > On Fri, Jul 29, 2022 at 3:45 PM Amit Kapila > wrote: > > > > > > > I've attached updated patches for all branches. Please review them. > > > > Thanks, the patches look mostly goo

RE: Handle infinite recursion in logical replication setup

2022-08-02 Thread shiy.f...@fujitsu.com
On Fri, Jul 29, 2022 1:22 PM vignesh C wrote: > > On Fri, Jul 29, 2022 at 8:31 AM Peter Smith > wrote: > > > > Thanks for the comments, the attached v41 patch has the changes for the > same. > Thanks for updating the patch. A comment for 0002 patch. In the example in section 31.11.4 (Generi

RE: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-08-02 Thread shiy.f...@fujitsu.com
On Wed, Aug 3, 2022 12:06 PM Masahiko Sawada wrote: > > I've attached updated patches that incorporated the above comments as > well as the comments from Shi yu. Please review them. > Thanks for updating the patch. I noticed that in SnapBuildXidSetCatalogChanges(), "i" is initialized in the if

RE: Introduce wait_for_subscription_sync for TAP tests

2022-08-04 Thread shiy.f...@fujitsu.com
On Thu, Aug 4, 2022 2:28 PM Masahiko Sawada wrote: > > On Thu, Aug 4, 2022 at 10:37 AM Amit Kapila > wrote: > > > > On Wed, Aug 3, 2022 at 10:21 AM Amit Kapila > wrote: > > > > > > Pushed this one and now I'll look at your other patch. > > > > > > > I have pushed the second patch as well after

RE: Introduce wait_for_subscription_sync for TAP tests

2022-08-04 Thread shiy.f...@fujitsu.com
On Thu, Aug 4, 2022 5:49 PM shiy.f...@fujitsu.com wrote: > > On Thu, Aug 4, 2022 2:28 PM Masahiko Sawada > wrote: > > > > On Thu, Aug 4, 2022 at 10:37 AM Amit Kapila > > wrote: > > > > > > On Wed, Aug 3, 2022 at 10:21 AM Amit Kapila > > &g

RE: Perform streaming logical transactions by background workers and parallel apply

2022-08-05 Thread shiy.f...@fujitsu.com
On Thu, Aug 4, 2022 2:36 PM Wang, Wei/王 威 wrote: > > I also did some other improvements based on the suggestions posted in this > thread. Attach the new patches. > Thanks for updating the patch. Here are some comments on v20-0001 patch. 1. +typedef struct ApplyBgworkerShared +{ + slock_t

RE: Perform streaming logical transactions by background workers and parallel apply

2022-08-17 Thread shiy.f...@fujitsu.com
On Wed, Aug 17, 2022 2:28 PM Wang, Wei/王 威 wrote: > > On Tues, August 16, 2022 15:33 PM I wrote: > > Attach the new patches. > > I found that cfbot has a failure. > After investigation, I think it is because the worker's exit state is not set > correctly. So I made some slight modifications. >

RE: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

2022-08-22 Thread shiy.f...@fujitsu.com
On Sat, Aug 20, 2022 7:02 PM Önder Kalacı wrote: > Hi, > > I'm a little late to catch up with your comments, but here are my replies: Thanks for your patch. Here are some comments. 1. In FilterOutNotSuitablePathsForReplIdentFull(), is "nonPartialIndexPathList" a good name for the list? Indexes

RE: Handle infinite recursion in logical replication setup

2022-08-31 Thread shiy.f...@fujitsu.com
On Wed, Aug 31, 2022 1:06 AM vignesh C wrote: > > The attached v43 patch has the changes for the same. > Thanks for updating the patch. Here is a comment on the 0001 patch. + if (!isnewtable) + { + pfree(nspname); + pfree

RE: Column Filtering in Logical Replication

2022-09-04 Thread shiy.f...@fujitsu.com
On Mon, Sep 5, 2022 8:28 AM Peter Smith wrote: > > I have rebased the remaining patch (v6-0001 is the same as v5-0002) > Thanks for updating the patch. Here are some comments. 1. + the will be successful but later + the WalSender on the publisher, or the subscriber may throw an error.

RE: Handle infinite recursion in logical replication setup

2022-09-05 Thread shiy.f...@fujitsu.com
On Tue, Sep 6, 2022 11:14 AM vignesh C wrote: > > Thanks for the comments, the attached patch has the changes for the same. > Thanks for updating the patch. Here are some comments. 1. + if (subrel_count) + { + /* +* In case of ALTER SUBSCRIPTION ... RE

RE: Handle infinite recursion in logical replication setup

2022-09-07 Thread shiy.f...@fujitsu.com
On Wed, Sep 7, 2022 12:23 PM vignesh C wrote: > > > Thanks for the comments, the attached v47 patch has the changes for the > same. > Thanks for updating the patch. Here is a comment. + for (i = 0; i < subrel_count; i++) + { + Oid relid = subrel_

RE: Perform streaming logical transactions by background workers and parallel apply

2022-09-19 Thread shiy.f...@fujitsu.com
On Mon, Sept 19, 2022 11:26 AM Wang, Wei/王 威 wrote: > > > Improved as suggested. > Thanks for updating the patch. Here are some comments on 0001 patch. 1. + case TRANS_LEADER_SERIALIZE: - oldctx = MemoryContextSwitchTo(ApplyContext); + /* +

<    1   2