Handle infinite recursion in logical replication setup

2022-02-22 Thread vignesh C
Hi, In logical replication, currently Walsender sends the data that is generated locally and the data that are replicated from other instances. This results in infinite recursion in circular logical replication setup. Here the user is trying to have a 2-way replication setup with node 1 publishin

Re: Handle infinite recursion in logical replication setup

2022-03-01 Thread vignesh C
On Tue, Mar 1, 2022 at 4:12 PM kuroda.hay...@fujitsu.com wrote: > > Hi Vignesh, > > > In logical replication, currently Walsender sends the data that is > > generated locally and the data that are replicated from other > > instances. This results in infinite recursion in circular logical > > repli

Re: Handle infinite recursion in logical replication setup

2022-03-02 Thread vignesh C
On Tue, Mar 1, 2022 at 4:12 PM kuroda.hay...@fujitsu.com wrote: > > Hi Vignesh, > > > In logical replication, currently Walsender sends the data that is > > generated locally and the data that are replicated from other > > instances. This results in infinite recursion in circular logical > > repli

Re: Handle infinite recursion in logical replication setup

2022-03-06 Thread vignesh C
On Mon, Mar 7, 2022 at 10:26 AM Peter Smith wrote: > > Hi Vignesh, I also have not looked at the patch yet, but I have what > seems like a very fundamental (and possibly dumb) question... > > Basically, I do not understand the choice of syntax for setting things up. > > IMO that "only-local" optio

Re: Handle infinite recursion in logical replication setup

2022-03-06 Thread vignesh C
On Tue, Mar 1, 2022 at 4:12 PM kuroda.hay...@fujitsu.com wrote: > > Hi Vignesh, > > > In logical replication, currently Walsender sends the data that is > > generated locally and the data that are replicated from other > > instances. This results in infinite recursion in circular logical > > repli

Re: Handle infinite recursion in logical replication setup

2022-03-06 Thread vignesh C
On Mon, Mar 7, 2022 at 11:45 AM Peter Smith wrote: > > On Mon, Mar 7, 2022 at 4:20 PM vignesh C wrote: > > > > On Mon, Mar 7, 2022 at 10:26 AM Peter Smith wrote: > > > > > > Hi Vignesh, I also have not looked at the patch yet, but I have what > > >

Re: Handle infinite recursion in logical replication setup

2022-03-07 Thread vignesh C
On Mon, Mar 7, 2022 at 1:45 PM Peter Smith wrote: > > On Mon, Mar 7, 2022 at 6:17 PM vignesh C wrote: > > > > On Mon, Mar 7, 2022 at 11:45 AM Peter Smith wrote: > > > > > > On Mon, Mar 7, 2022 at 4:20 PM vignesh C wrote: > > > > >

Re: Handle infinite recursion in logical replication setup

2022-03-07 Thread vignesh C
On Mon, Mar 7, 2022 at 5:51 PM Amit Kapila wrote: > > On Mon, Mar 7, 2022 at 5:01 PM Ashutosh Bapat > wrote: > > > > Hi Vignesh, > > I agree with Peter's comment that the changes to > > FilterRemoteOriginData() should be part of FilterByOrigin() > > > > Further, I wonder why "onlylocal_data" is a

Re: Handle infinite recursion in logical replication setup

2022-03-08 Thread vignesh C
On Mon, Mar 7, 2022 at 2:28 PM Peter Smith wrote: > > Hi Vignesh, > > Here are some review comments for patch v2. > > == > > 1. Question about syntax > > I already posted some questions about why the syntax is on the CREATE > SUBSCRCRIBER side. > IMO "local_only" is a publisher option, so it s

Re: Handle infinite recursion in logical replication setup

2022-03-08 Thread vignesh C
On Mon, Mar 7, 2022 at 9:57 PM vignesh C wrote: > > On Mon, Mar 7, 2022 at 5:51 PM Amit Kapila wrote: > > > > On Mon, Mar 7, 2022 at 5:01 PM Ashutosh Bapat > > wrote: > > > > > > Hi Vignesh, > > > I agree with Peter's comment that the cha

Re: Handle infinite recursion in logical replication setup

2022-03-11 Thread vignesh C
On Wed, Mar 2, 2022 at 3:59 PM Amit Kapila wrote: > > On Wed, Feb 23, 2022 at 11:59 AM vignesh C wrote: > > > ... > ... > > I have attached a basic patch for this, if the idea is accepted, I > > will work further to test more scenarios, add documentation, and test

Re: Handle infinite recursion in logical replication setup

2022-03-11 Thread vignesh C
On Fri, Mar 11, 2022 at 4:28 PM kuroda.hay...@fujitsu.com wrote: > > Hi Vegnesh, > > While considering about second problem, I was very confusing about it. > I'm happy if you answer my question. > > > To handle this if user has specified only_local option, we could throw > > a warning or error out

Re: Printing backtrace of postgres processes

2022-03-11 Thread vignesh C
lots of log messages and which can lead to denial of service. On receipt of the request, at the next CHECK_FOR_INTERRUPTS(), the target backend logs its backtrace at LOG_SERVER_ONLY level, so that the backtrace will appear in the server log but not be sent to the client. Bump catalog version. Auth

Re: Logical replication - schema change not invalidating the relation cache

2022-03-11 Thread vignesh C
On Fri, Dec 3, 2021 at 3:21 PM vignesh C wrote: > > On Fri, Dec 3, 2021 at 1:13 PM Michael Paquier wrote: > > > > On Thu, Aug 26, 2021 at 09:00:39PM +0530, vignesh C wrote: > > > The previous patch was failing because of the recent test changes made > > > by

Tab completion not listing schema list for create/alter publication for all tables in schema

2022-03-13 Thread vignesh C
Hi, I noticed that the following commands "CREATE PUBLICATION pub1 FOR ALL TABLES IN SCHEMA" and "ALTER PUBLICATION pub1 ADD ALL TABLES IN SCHEMA" does not complete with the schema list. I feel this is because of the following code in tab-complete.c: . COMPLETE_WITH_QUERY_PLUS(Query_for_l

Re: Tab completion not listing schema list for create/alter publication for all tables in schema

2022-03-14 Thread vignesh C
On Mon, Mar 14, 2022 at 5:23 AM Tom Lane wrote: > > vignesh C writes: > > Here "pg_%" should be "pg_%%". > > Right you are. Patch pushed, thanks! Thanks for pushing the patch. Regards, Vignesh

Re: Handle infinite recursion in logical replication setup

2022-03-15 Thread vignesh C
On Tue, Mar 15, 2022 at 7:09 AM kuroda.hay...@fujitsu.com wrote: > > Dear Vignesh, > > Thank you for updating your patch! > > > Let's consider an existing Multi master logical replication setup > > between Node1 and Node2 that is created using the following steps: > > a) Node1 - Publication publis

Re: Handle infinite recursion in logical replication setup

2022-03-15 Thread vignesh C
On Tue, Mar 15, 2022 at 9:55 AM kuroda.hay...@fujitsu.com wrote: > > Dear Vignesh, > > > Thanks for kind explanation. > > I read above and your doc in 0002, and I put some comments. > > I forgot a comment about 0002 doc. > > 5. create_subscription.sgml - about your example > > Three possibilities

Skipping schema changes in publication

2022-03-22 Thread vignesh C
Hi, This feature adds an option to skip changes of all tables in specified schema while creating publication. This feature is helpful for use cases where the user wants to subscribe to all the changes except for the changes present in a few schemas. Ex: CREATE PUBLICATION pub1 FOR ALL TABLES SKIP

Re: Identify missing publications from publisher while create/alter subscription.

2022-03-22 Thread vignesh C
On Tue, Mar 22, 2022 at 5:29 AM Andres Freund wrote: > > On 2022-02-13 19:34:05 +0530, vignesh C wrote: > > Thanks for the comments, the attached v14 patch has the changes for the > > same. > > The patch needs a rebase, it currently fails to apply: > http://cfbot.cp

Re: Printing backtrace of postgres processes

2020-12-01 Thread vignesh C
On Tue, Dec 1, 2020 at 9:31 AM Tom Lane wrote: > > Andres Freund writes: > > It should be quite doable to emit such backtraces directly to stderr, > > instead of using appendStringInfoString()/elog(). > > No, please no. > > (1) On lots of logging setups (think syslog), anything that goes to > std

Added missing copy related data structures to typedefs.list

2020-12-07 Thread vignesh C
EnterpriseDB: http://www.enterprisedb.com From d656c2f0bfbf68f5ceb98a0eb205e5e77f21602f Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Mon, 7 Dec 2020 13:48:31 +0530 Subject: [PATCH] Added missing copy related data structures. Added missing copy related data structures to typedefs.list, these data

Re: Printing backtrace of postgres processes

2020-12-08 Thread vignesh C
On Tue, Dec 1, 2020 at 2:15 PM vignesh C wrote: > > On Tue, Dec 1, 2020 at 9:31 AM Tom Lane wrote: > > > > Andres Freund writes: > > > It should be quite doable to emit such backtraces directly to stderr, > > > instead of using appendStringInfoString()/elog(

Re: Parallel INSERT (INTO ... SELECT ...)

2020-12-08 Thread vignesh C
On Mon, Dec 7, 2020 at 2:35 PM Greg Nancarrow wrote: > > On Fri, Nov 20, 2020 at 7:44 PM Greg Nancarrow wrote: > > > > Posting an updated set of patches, with some additional testing and > > documentation updates, and including the latest version of the > > Parallel Insert patch. > > Any feedback

Re: Parallel copy

2020-12-09 Thread vignesh C
On Mon, Dec 7, 2020 at 3:00 PM Hou, Zhijie wrote: > > > Attached v11 patch has the fix for this, it also includes the changes to > > rebase on top of head. > > Thanks for the explanation. > > I think there is still chances we can know the size. > > +* line_size will be set. Read th

Re: Parallel INSERT (INTO ... SELECT ...)

2020-12-09 Thread vignesh C
On Wed, Dec 9, 2020 at 10:11 AM Greg Nancarrow wrote: > > On Wed, Dec 9, 2020 at 1:35 AM vignesh C wrote: > > > > Most of the code present in > > v9-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT.patch is > > applicable for parallel copy patch also. The patch

Re: Parallel Inserts in CREATE TABLE AS

2020-12-23 Thread vignesh C
On Tue, Dec 22, 2020 at 2:16 PM Bharath Rupireddy wrote: > > On Tue, Dec 22, 2020 at 12:32 PM Bharath Rupireddy > Attaching v14 patch set that has above changes. Please consider this > for further review. > Few comments: In the below case, should create be above Gather? postgres=# explain create

Re: Parallel Inserts in CREATE TABLE AS

2020-12-24 Thread vignesh C
On Thu, Dec 24, 2020 at 11:29 AM Amit Kapila wrote: > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > > > On Tue, Dec 22, 2020 at 2:16 PM Bharath Rupireddy > > wrote: > > > > > > On Tue, Dec 22, 2020 at 12:32 PM Bharath Rupireddy > >

Re: Added missing copy related data structures to typedefs.list

2020-12-26 Thread vignesh C
On Thu, Dec 17, 2020 at 4:28 AM Bruce Momjian wrote: > > On Mon, Dec 7, 2020 at 01:56:50PM +0530, vignesh C wrote: > > Hi, > > > > Added missing copy related data structures to typedefs.list, these > > data structures were added while copy files were split during

Re: Parallel copy

2020-12-26 Thread vignesh C
On Wed, Dec 23, 2020 at 3:05 PM Hou, Zhijie wrote: > > Hi > > > Yes this optimization can be done, I will handle this in the next patch > > set. > > > > I have a suggestion for the parallel safety-check. > > As designed, The leader does not participate in the insertion of data. > If User use (PARA

Re: Parallel Inserts in CREATE TABLE AS

2020-12-26 Thread vignesh C
On Thu, Dec 24, 2020 at 1:07 PM Bharath Rupireddy wrote: > > On Thu, Dec 24, 2020 at 10:25 AM vignesh C wrote: > > You could change intoclause_len = strlen(intoclausestr) to > > strlen(intoclausestr) + 1 and use intoclause_len in the remaining > > places. We can avoid th

Re: Parallel Inserts in CREATE TABLE AS

2020-12-27 Thread vignesh C
On Sun, Dec 27, 2020 at 2:28 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > > On Sat, Dec 26, 2020 at 9:20 PM vignesh C wrote: > > +-- parallel inserts must occur > > +select explain_pictas( > > +'create table parallel_write as

Re: Parallel copy

2020-12-28 Thread vignesh C
35.625, 20), (109.219, 136.144, 30) Attached is a patch that was used for the same. The patch is written on top of the parallel copy patch. The design Amit, Andres & myself voted for that is the leader identifying the line bound design and sharing it in shared memory is performing better. R

Re: Parallel Inserts in CREATE TABLE AS

2020-12-30 Thread vignesh C
On Wed, Dec 30, 2020 at 10:47 AM Bharath Rupireddy wrote: > > On Wed, Dec 30, 2020 at 10:32 AM Dilip Kumar wrote: > > I have completed reviewing 0001, I don't have more comments, just one > > question. Soon I will review the remaining patches. > > Thanks. > > > +/* If parallel inserts are to

Re: Parallel Inserts in CREATE TABLE AS

2020-12-30 Thread vignesh C
On Wed, Dec 30, 2020 at 9:25 AM Bharath Rupireddy wrote: > > On Wed, Dec 30, 2020 at 5:22 AM Zhihong Yu wrote: > > w.r.t. v17-0004-Enable-CTAS-Parallel-Inserts-For-Append.patch > > > > + * Push the dest receiver to Gather node when it is either at the top of > > the > > + * plan or under top App

Re: Added missing copy related data structures to typedefs.list

2021-01-04 Thread vignesh C
On Wed, Dec 30, 2020 at 7:10 PM Amit Kapila wrote: > > On Sat, Dec 26, 2020 at 9:16 PM vignesh C wrote: > > > > On Thu, Dec 17, 2020 at 4:28 AM Bruce Momjian wrote: > > > > > > On Mon, Dec 7, 2020 at 01:56:50PM +0530, vignesh C wrote: > > > > Hi,

Re: Parallel Inserts in CREATE TABLE AS

2021-01-04 Thread vignesh C
On Mon, Jan 4, 2021 at 3:07 PM Bharath Rupireddy wrote: > > On Wed, Dec 30, 2020 at 5:28 PM vignesh C wrote: > > Few comments: > > - /* > > -* To allow parallel inserts, we need to ensure that they are safe > > to be > > -* performed in

Added schema level support for publication.

2021-01-07 Thread vignesh C
production_quarterly_publication DROP SCHEMA production_july; Attached is a POC patch for the same. I felt this feature would be quite useful. Thoughts? Regards, Vignesh EnterpriseDB: http://www.enterprisedb.com From f9b5134182229f718bbc1a9162b6043f879a6410 Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Thu, 7 Jan 2021 11:38

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-08 Thread vignesh C
On Fri, Dec 11, 2020 at 4:30 PM Greg Nancarrow wrote: > > Posting an updated set of patches to address recent feedback: > > - Removed conditional-locking code used in parallel-safety checking > code (Tsunakawa-san feedback). It turns out that for the problem test > case, no parallel-safety checkin

Re: Added schema level support for publication.

2021-01-09 Thread vignesh C
On Fri, Jan 8, 2021 at 4:32 PM Amit Kapila wrote: > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: > > > > This feature adds schema option while creating publication. Users will > > be able to specify one or more schemas while creating publication, > > when th

Re: Added schema level support for publication.

2021-01-10 Thread vignesh C
Thanks for your comments Bharath, please find my opinion below. On Sat, Jan 9, 2021 at 8:08 PM Bharath Rupireddy wrote: > I think this feature can be useful, in case a user has a lot of tables > to publish inside a schema. Having said that, I wonder if this feature > mandates users to create the

Re: Added schema level support for publication.

2021-01-10 Thread vignesh C
On Sat, Jan 9, 2021 at 8:14 PM Bharath Rupireddy wrote: > > One more point - if the publication is created for a schema with no or > some initial tables, will all the future tables that may get added to > the schema will be replicated too? > I agree on this, when a relation is added to the schema

Re: Libpq support to connect to standby server as priority

2021-02-11 Thread vignesh C
On Fri, Feb 12, 2021 at 7:07 AM Greg Nancarrow wrote: > > On Wed, Feb 10, 2021 at 5:09 PM vignesh C wrote: > > > > Modified. > > These comments are handled in v22 patch posted in my earlier mail. > > > > Thanks, just one minor thing I missed in doc/src/sgml/

Re: logical replication seems broken

2021-02-14 Thread vignesh C
On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > On 02/13/2021 11:49 AM Amit Kapila wrote: > > > > On Fri, Feb 12, 2021 at 10:00 PM wrote: > > > > > > > On 02/12/2021 1:51 PM Amit Kapila wrote: > > > > > > > > On Fri, Feb 12, 2021 at 6:04 PM Erik Rijkers wrote: > > > > > > > > > > I a

Re: logical replication seems broken

2021-02-15 Thread vignesh C
On Mon, Feb 15, 2021 at 5:02 PM Amit Kapila wrote: > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > > > > > > I compiled just now a binary from HEAD, and a binary from HEAD

Re: logical replication seems broken

2021-02-15 Thread vignesh C
On Mon, Feb 15, 2021 at 6:14 PM wrote: > > > > On 2021.02.15. 12:31 Amit Kapila wrote: > > On Mon, Feb 15, 2021 at 11:53 AM vignesh C wrote: > > > On Sat, Feb 13, 2021 at 5:58 PM Erik Rijkers wrote: > > > > I compiled just now a binary from HEAD, and a

Re: repeated decoding of prepared transactions

2021-02-25 Thread vignesh C
On Wed, Feb 24, 2021 at 5:06 PM Ajin Cherian wrote: > > On Wed, Feb 24, 2021 at 4:48 PM Ajin Cherian wrote: > > > I plan to split this into two patches next. But do review and let me > > know if you have any comments. > > Attaching an updated patch-set with the changes for > snapshot_was_exported

Re: repeated decoding of prepared transactions

2021-02-26 Thread vignesh C
On Fri, Feb 26, 2021 at 4:13 PM Ajin Cherian wrote: > > On Fri, Feb 26, 2021 at 7:47 PM Ajin Cherian wrote: > > > I've updated snapshot_was_exported_at_ member to pg_replication_slots as > > well. > > Do have a look and let me know if there are any comments. > > Update with both patches. Thank

Re: repeated decoding of prepared transactions

2021-02-27 Thread vignesh C
On Sat, Feb 27, 2021 at 8:29 AM Amit Kapila wrote: > > On Fri, Feb 26, 2021 at 7:26 PM vignesh C wrote: > > > > On Fri, Feb 26, 2021 at 4:13 PM Ajin Cherian wrote: > > > > > > On Fri, Feb 26, 2021 at 7:47 PM Ajin Cherian wrote: > > > > > &g

Re: repeated decoding of prepared transactions

2021-02-27 Thread vignesh C
On Sat, Feb 27, 2021 at 5:36 PM Amit Kapila wrote: > > On Sat, Feb 27, 2021 at 11:38 AM Amit Kapila wrote: > > > > On Fri, Feb 26, 2021 at 4:13 PM Ajin Cherian wrote: > > > > > > On Fri, Feb 26, 2021 at 7:47 PM Ajin Cherian wrote: > > > > > > > I've updated snapshot_was_exported_at_ member to

Re: repeated decoding of prepared transactions

2021-02-28 Thread vignesh C
On Mon, Mar 1, 2021 at 7:23 AM Ajin Cherian wrote: > > On Sat, Feb 27, 2021 at 11:06 PM Amit Kapila wrote: > > > Few comments on 0002 patch: > > = > > 1. > > + > > + /* > > + * Disable two-phase here, it will be set in the core if it was > > + * enabled whole creating the

Re: repeated decoding of prepared transactions

2021-03-01 Thread vignesh C
On Tue, Mar 2, 2021 at 6:37 AM Ajin Cherian wrote: > > On Mon, Mar 1, 2021 at 8:08 PM Amit Kapila wrote: > > > Few minor comments on 0002 patch > > = > > 1. > > ctx->streaming &= enable_streaming; > > - ctx->twophase &= enable_twophase; > > + > > } > > > > Spurious

Re: repeated decoding of prepared transactions

2021-03-01 Thread vignesh C
On Tue, Mar 2, 2021 at 9:33 AM Amit Kapila wrote: > > On Tue, Mar 2, 2021 at 8:20 AM vignesh C wrote: > > > > > > I have a minor comment regarding the below: > > + > > + > > + two_phase bool > > + > > + > &g

Buildfarm failure in crake

2021-03-02 Thread vignesh C
Hi, I noticed there is buildfarm failure in crake, it fails with the following error: Mar 02 21:22:56 ./src/test/recovery/t/001_stream_rep.pl: Variable declared in conditional statement at line 88, column 2. Declare variables outside of the condition. ([Variables::ProhibitConditionalDeclarations]

Re: Libpq support to connect to standby server as priority

2021-03-02 Thread vignesh C
On Wed, Mar 3, 2021 at 7:37 AM Tom Lane wrote: > > Greg Nancarrow writes: > > I've marked this as "Ready for Committer". > > I've pushed this after whacking it around a fair amount. A lot of > that was cosmetic, but one thing that wasn't is that I got rid of the > proposed "which_primary_host" v

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-05 Thread vignesh C
On Fri, Mar 5, 2021 at 12:21 PM Ajin Cherian wrote: > > On Thu, Mar 4, 2021 at 9:53 PM Peter Smith wrote: > > > [05a] Now syncing the psf file at prepare time > > The patch v46-0008 does not handle spooling of streaming prepare if > the Subscription is configured for both two-phase and streaming.

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-07 Thread vignesh C
On Mon, Mar 8, 2021 at 7:17 AM Peter Smith wrote: > > Please find attached the latest patch set v52* > Few comments: +logicalrep_read_begin_prepare(StringInfo in, LogicalRepBeginPrepareData *begin_data) +{ + /* read fields */ + begin_data->final_lsn = pq_getmsgint64(in); + if (b

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Mon, Mar 8, 2021 at 11:30 AM Ajin Cherian wrote: > > On Fri, Mar 5, 2021 at 9:25 PM vignesh C wrote: > > > Created new patch v53: Thanks for the updated patch. I had noticed one issue, publisher does not get stopped normally in the following case: # Publisher steps psql

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Mon, Mar 8, 2021 at 6:25 PM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 4:20 PM vignesh C wrote: > > > > On Mon, Mar 8, 2021 at 11:30 AM Ajin Cherian wrote: > > > > > > On Fri, Mar 5, 2021 at 9:25 PM vignesh C wrote: > > > > > > >

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Tue, Mar 9, 2021 at 9:14 AM Peter Smith wrote: > > On Mon, Mar 8, 2021 at 4:58 PM vignesh C wrote: > > > > LOGICAL_REP_MSG_TYPE = 'Y', > > + LOGICAL_REP_MSG_BEGIN_PREPARE = 'b', > > + LOGICAL_REP_MSG_PREPARE = &#x

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-08 Thread vignesh C
On Tue, Mar 9, 2021 at 11:01 AM Amit Kapila wrote: > > On Mon, Mar 8, 2021 at 8:09 PM vignesh C wrote: > > > > On Mon, Mar 8, 2021 at 6:25 PM Amit Kapila wrote: > > > > > > > I think in case of two_phase option, replicatedPtr and sentPtr never > >

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-09 Thread vignesh C
On Tue, Mar 9, 2021 at 10:46 AM Peter Smith wrote: > > Please find attached the latest patch set v54* > > Differences from v53* are: > > * Rebased to HEAD @ today > > * Addresses some recent feedback issues for patch 0001 > > Feedback from Amit @ 7/March [ak] > - (36) Fixed. Comment about the psf

Do we support upgrade of logical replication?

2021-03-10 Thread vignesh C
Hi, I was reviewing logical decoding of two-phase transactions feature, while reviewing the feature I was checking if there is any impact on publisher/subscriber upgrade. I checked the existing pg_upgrade behaviour with logical replication. I made a logical replication data instance with publishe

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-11 Thread vignesh C
On Thu, Mar 11, 2021 at 7:20 AM Peter Smith wrote: > > On Thu, Mar 11, 2021 at 12:46 PM Peter Smith wrote: > > > > Please find attached the latest patch set v57* > > > > Differences from v56* are: > > > > * Rebased to HEAD @ today > > > > * Addresses the following feedback issues: > > > > (24) [v

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-12 Thread vignesh C
On Fri, Mar 12, 2021 at 2:29 PM Peter Smith wrote: > > On Fri, Mar 12, 2021 at 4:07 PM vignesh C wrote: > > Hi Vignesh, > > Thanks for the review comments. > > But can you please resend it with each feedback enumerated as 1. 2. > 3., or have some other clear

Re: locking [user] catalog tables vs 2pc vs logical rep

2021-03-15 Thread vignesh C
On Tue, Feb 23, 2021 at 3:59 AM Andres Freund wrote: > > Hi, > > The 2pc decoding added in > > commit a271a1b50e9bec07e2ef3a05e38e7285113e4ce6 > Author: Amit Kapila > Date: 2021-01-04 08:34:50 +0530 > > Allow decoding at prepare time in ReorderBuffer. > > has a deadlock danger when used in

Re: subscriptionCheck failures

2021-03-16 Thread vignesh C
On Tue, Mar 16, 2021 at 12:29 PM Amit Kapila wrote: > > On Tue, Mar 16, 2021 at 9:00 AM Amit Kapila wrote: > > > > On Mon, Mar 15, 2021 at 6:00 PM Thomas Munro wrote: > > > > > > Hi, > > > > > > This seems to be a new low frequency failure, I didn't see it mentioned > > > already: > > > > > > >

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-16 Thread vignesh C
On Mon, Mar 15, 2021 at 6:14 PM Ajin Cherian wrote: > > On Mon, Mar 15, 2021 at 2:04 PM Amit Kapila wrote: >> >> I think something on these lines should be much >> easier than the spool-file implementation unless we see any problem >> with this idea. >> > > Here's a new patch-set that implements

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-16 Thread vignesh C
On Mon, Mar 15, 2021 at 6:14 PM Ajin Cherian wrote: > > On Mon, Mar 15, 2021 at 2:04 PM Amit Kapila wrote: >> >> I think something on these lines should be much >> easier than the spool-file implementation unless we see any problem >> with this idea. >> > > Here's a new patch-set that implements

Re: Parallel copy

2020-10-27 Thread vignesh C
On Wed, Oct 21, 2020 at 4:20 PM Bharath Rupireddy wrote: > > On Wed, Oct 21, 2020 at 3:18 PM Bharath Rupireddy > wrote: > > > > 17. Remove extra lines after #define IsHeaderLine() > > (cstate->header_line && cstate->cur_lineno == 1) in copy.h > > > > I missed one comment: > > 18. I think we nee

Re: Parallel copy

2020-10-27 Thread vignesh C
Thanks Heikki for reviewing and providing your comments. Please find my thoughts below. On Fri, Oct 23, 2020 at 2:01 PM Heikki Linnakangas wrote: > > I had a brief look at at this patch. Important work! A couple of first > impressions: > > 1. The split between patches > 0002-Framework-for-leader-

Re: Parallel copy

2020-10-27 Thread vignesh C
On Wed, Oct 21, 2020 at 3:50 PM Amit Kapila wrote: > > On Wed, Oct 21, 2020 at 3:19 PM Bharath Rupireddy > wrote: > > > > > > 9. Instead of calling CopyStringToSharedMemory() for each string > > variable, can't we just create a linked list of all the strings that > > need to be copied into shm an

Re: Parallel copy

2020-10-27 Thread vignesh C
Thanks Ashutosh for reviewing and providing your comments. On Fri, Oct 23, 2020 at 5:43 PM Ashutosh Sharma wrote: > > Hi Vignesh, > > Thanks for the updated patches. Here are some more comments that I can > find after reviewing your latest patches: > > +/* > + * This structure helps in storing th

Re: Parallel copy

2020-10-27 Thread vignesh C
On Fri, Oct 23, 2020 at 6:58 PM Ashutosh Sharma wrote: > > > > > I think, if possible, all these if-else checks in CopyFrom() can be > > moved to a single function which can probably be named as > > IdentifyCopyInsertMethod() and this function can be called in > > IsParallelCopyAllowed(). This wil

Log message for GSS connection is missing once connection authorization is successful.

2020-10-27 Thread vignesh C
ep 17 00:00:00 2001 From: Vignesh C Date: Wed, 28 Oct 2020 08:19:06 +0530 Subject: [PATCH v1] Log message for GSS connection is missing once connection authorization is successful. Log message for GSS connection is missing once connection authorization is successful. We have similar log message fo

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread vignesh C
Thanks Stephen for your comments. On Wed, Oct 28, 2020 at 9:44 PM Stephen Frost wrote: > > Greetings, > > * vignesh C (vignes...@gmail.com) wrote: > > Log message for GSS connection is missing once connection > > authorization is successful. We have similar log messages

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread vignesh C
Thanks Bharath for your comments. On Wed, Oct 28, 2020 at 9:48 AM Bharath Rupireddy wrote: > > On Wed, Oct 28, 2020 at 8:29 AM vignesh C wrote: > > > > Log message for GSS connection is missing once connection > > authorization is successful. We have similar log messages

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-30 Thread vignesh C
On Thu, Oct 29, 2020 at 7:26 PM Stephen Frost wrote: > > Greetings, > > * vignesh C (vignes...@gmail.com) wrote: > > I have made a v2 patch based on the changes you have suggested. The > > patch for the same is attached. > > > From b067cf823750f200102be0a0cad9a26a

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-30 Thread vignesh C
Thanks for the comments Bharath. On Thu, Oct 29, 2020 at 12:15 PM Bharath Rupireddy wrote: > 1. Instead of just "on/off" after GSS %s in the log message, wouldn't it be > informative if we have authenticated and/or encrypted as suggested by Stephen? > > So the log message would look like this: >

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-31 Thread vignesh C
machine. make check of 001_auth.pl > is passing. > I have changed this within double quotes now as it includes passing of the variable also. Removed the escape sequence which is not required. The v4 patch attached has the fixes for this. Regards, Vignesh EnterpriseDB: http://www.enterprisedb.c

Re: Split copy.c

2020-11-02 Thread vignesh C
On Mon, Nov 2, 2020 at 2:33 PM Heikki Linnakangas wrote: > > While looking at the parallel copy patches, it started to annoy me how > large copy.c is. It confuses my little head. (Ok, it's annoyed me many > times in the past, but I haven't done anything about it.) > +1 for having copy from & copy

Re: Parallel INSERT (INTO ... SELECT ...)

2020-11-02 Thread vignesh C
> > See attached patches. > Thanks for providing the patches. I had reviewed v6-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT.patch, please find my comments: -> commandType is not used, we can remove it. + * Prepare for entering parallel mode by assigning a FullTransactionId, to be + * incl

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-02 Thread vignesh C
quot;. If you decided to keep this code as > is, > the printed message won't follow the grammar rules. You will have "replicação > conexão autorizada" instead of "conexão de replicação autorizada". The former > isn't grammatically correct. Avoid splitting

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-06 Thread vignesh C
> > after > > feature freeze. > > > > Thanks. > > On Tue, Nov 3, 2020 at 12:49 PM vignesh C wrote: > > > > Thanks for the explanation, I have attached a v5 patch with the > > changes where the translation should not have any problem. > > > &g

Re: Parallel copy

2020-11-07 Thread vignesh C
On Thu, Nov 5, 2020 at 6:33 PM Hou, Zhijie wrote: > > Hi > > > > > my $bytes = $ARGV[0]; > > for(my $i = 0; $i < $bytes; $i+=8){ > > print "longdata"; > > } > > print "\n"; > > > > > > postgres=# copy longdata from program 'perl /tmp/longdata.pl 1' > > with (parallel 2); > >

Re: Parallel copy

2020-11-10 Thread vignesh C
n which all cases we can parallel-writes > (parallel copy) is allowed, for example need to identify whether table > or one of its partitions has any constraint/expression which is > parallel-unsafe. > I have worked to provide a patch for the parallel safety checks. It checks if paralle

Re: Parallel copy

2020-11-11 Thread vignesh C
On Tue, Nov 10, 2020 at 7:27 PM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 7:12 PM vignesh C wrote: > > > > On Tue, Nov 3, 2020 at 2:28 PM Amit Kapila wrote: > > > > > > > I have worked to provide a patch for the parallel safety checks. It > &

Re: Parallel copy

2020-11-18 Thread vignesh C
On Thu, Oct 29, 2020 at 2:20 PM Heikki Linnakangas wrote: > > On 27/10/2020 15:36, vignesh C wrote: > > Attached v9 patches have the fixes for the above comments. > > I did some testing: > > /tmp/longdata.pl: > > #!/usr/bin/perl > #

Re: Parallel copy

2020-11-18 Thread vignesh C
On Wed, Oct 28, 2020 at 5:36 PM Hou, Zhijie wrote: > > Hi > > I found some issue in v9-0002 > > 1. > + > + elog(DEBUG1, "[Worker] Processing - line position:%d, block:%d, unprocessed lines:%d, offset:%d, line size:%d", > +write_pos, lineInfo->first_block, > + pg_atomic_read_

Re: Parallel copy

2020-11-18 Thread vignesh C
On Thu, Oct 29, 2020 at 2:26 PM Daniel Westermann (DWE) wrote: > > On 27/10/2020 15:36, vignesh C wrote: > >> Attached v9 patches have the fixes for the above comments. > > >I did some testing: > > I did some testing as well and have a cosmetic remark: > > post

Re: Parallel copy

2020-11-18 Thread vignesh C
On Fri, Nov 13, 2020 at 2:25 PM Amit Kapila wrote: > > On Wed, Nov 11, 2020 at 10:42 PM vignesh C wrote: > > > > On Tue, Nov 10, 2020 at 7:27 PM Amit Kapila wrote: > > > > > > On Tue, Nov 10, 2020 at 7:12 PM vignesh C wrote: > > > > &g

Re: Parallel copy

2020-11-18 Thread vignesh C
On Sat, Oct 31, 2020 at 2:07 AM Tomas Vondra wrote: > > Hi, > > I've done a bit more testing today, and I think the parsing is busted in > some way. Consider this: > > test=# create extension random; > CREATE EXTENSION > > test=# create table t (a text); > CREATE TABLE > >

Re: Parallel copy

2020-11-18 Thread vignesh C
On Sat, Nov 7, 2020 at 7:01 PM vignesh C wrote: > > On Thu, Nov 5, 2020 at 6:33 PM Hou, Zhijie wrote: > > > > Hi > > > > > > > > my $bytes = $ARGV[0]; > > > for(my $i = 0; $i < $bytes; $i+=8){ > > > print "longdata&qu

Printing backtrace of postgres processes

2020-11-21 Thread vignesh C
for the same. Thoughts? Regards, Vignesh EnterpriseDB: http://www.enterprisedb.com From c1006110bdeac2135d1c8e9220f65d50cd49ab63 Mon Sep 17 00:00:00 2001 From: Vignesh C Date: Sun, 22 Nov 2020 05:58:24 +0530 Subject: [PATCH] Print backtrace of postgres process that are part of this instance. The id

Re: Printing backtrace of postgres processes

2020-11-29 Thread vignesh C
On Sun, Nov 22, 2020 at 11:55 AM Tom Lane wrote: > > vignesh C writes: > > The idea here is to implement & expose pg_print_callstack function, > > internally what this function does is, the connected backend will send > > SIGUSR1 signal by setting PMSIGNAL_BACK

Re: Added schema level support for publication.

2021-01-12 Thread vignesh C
On Mon, Jan 11, 2021 at 11:45 AM Bharath Rupireddy wrote: > > On Sun, Jan 10, 2021 at 11:21 PM vignesh C wrote: > > On Sat, Jan 9, 2021 at 8:08 PM Bharath Rupireddy > > wrote: > > > I think this feature can be useful, in case a user has a lot of tables > > >

Re: Added schema level support for publication.

2021-01-15 Thread vignesh C
Thanks Rahila for your comments, please find my thoughts below. On Tue, Jan 12, 2021 at 5:16 PM Rahila Syed wrote: > > Hi Vignesh, > > I had a look at the patch, please consider following comments. > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: >> >> Hi,

Re: Printing backtrace of postgres processes

2021-01-16 Thread vignesh C
On Sat, Jan 16, 2021 at 1:40 AM Andres Freund wrote: > > On 2021-01-15 09:53:05 +0100, Peter Eisentraut wrote: > > On 2020-12-08 10:38, vignesh C wrote: > > > I have implemented printing of backtrace based on handling it in > > > CHECK_FOR_INTERRUPTS. This patch als

Re: Printing backtrace of postgres processes

2021-01-17 Thread vignesh C
On Sat, Jan 16, 2021 at 11:10 PM Andres Freund wrote: > > Hi, > > On Sat, Jan 16, 2021, at 09:34, vignesh C wrote: > > On Sat, Jan 16, 2021 at 1:40 AM Andres Freund wrote: > > > > > > On 2021-01-15 09:53:05 +0100, Peter Eisentraut wrote: > > > > O

Re: Added schema level support for publication.

2021-01-18 Thread vignesh C
On Sat, Jan 9, 2021 at 5:21 PM vignesh C wrote: > > On Fri, Jan 8, 2021 at 4:32 PM Amit Kapila wrote: > > > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: > > > > > > This feature adds schema option while creating publication. Users will > > >

  1   2   3   4   5   6   7   8   9   10   >