RE: Failed transaction statistics to measure the logical replication progress

2022-02-21 Thread tanghy.f...@fujitsu.com
On Mon, Feb 21, 2022 11:46 AM osumi.takami...@fujitsu.com wrote: > > On Saturday, February 19, 2022 12:00 AM osumi.takami...@fujitsu.com > wrote: > > On Friday, February 18, 2022 3:34 PM Tang, Haiying/唐 海英 > > wrote: > > > On Wed, Jan 12, 2022 8:35 PM osumi.takami...@fujitsu.com > > > wrote:

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-22 Thread tanghy.f...@fujitsu.com
On Tue, Feb 22, 2022 1:45 PM Masahiko Sawada wrote: > > I've attached a patch that changes pg_stat_subscription_workers view. > It removes non-cumulative values such as error details such as > error-XID and the error message from the view, and consequently the > view now has only cumulative stati

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-23 Thread tanghy.f...@fujitsu.com
Hi Osumi-san, I have a comment on v21 patch. I wonder if we really need subscription s2 in 028_disable_on_error.pl. I think for subscription s2, we only tested some normal cases(which could be tested with s1), and didn't test any error case, which means it wouldn't be automatically disabled.

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-23 Thread tanghy.f...@fujitsu.com
On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada wrote: > > Thank you for the comments! I've attached the latest version patch > that incorporated all comments I got so far. The primary change from > the previous version is that the subscription statistics live globally > rather than per-database. >

RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-27 Thread tanghy.f...@fujitsu.com
On Mon, Feb 28, 2022 12:32 PM Amit Kapila wrote: > > On Mon, Feb 28, 2022 at 8:17 AM Masahiko Sawada > wrote: > > > > On Mon, Feb 28, 2022 at 11:33 AM Amit Kapila > wrote: > > > > > > > > > > > (2) doc/src/sgml/monitoring.sgml > > > > > > > > +Resets statistics for a single subscription

Tuples unbalance distribution among workers in underlying parallel select with serial insert

2021-02-22 Thread tanghy.f...@fujitsu.com
Hi Hackers, Recently, I took some performance measurements for CREATE TABLE AS. https://www.postgresql.org/message-id/34549865667a4a3bb330ebfd035f85d3%40G08CNEXMBPEKD05.g08.fujitsu.local Then I found an issue about the tuples unbalance distribution(99% tuples read by one worker) among workers u

RE: Parallel INSERT (INTO ... SELECT ...)

2021-02-26 Thread tanghy.f...@fujitsu.com
From: Tsunakawa, Takayuki/綱川 貴之 >the current patch showd nice performance improvement in some (many?) patterns. > >So, I think it can be committed in PG 14, when it has addressed the plan cache >issue that Amit Langote-san posed. Agreed. I summarized my test results for the current patch(V18)

RE: row filtering for logical replication

2021-12-01 Thread tanghy.f...@fujitsu.com
On Thursday, December 2, 2021 5:21 AM Peter Smith wrote: > > PSA the v44* set of patches. > Thanks for the new patch. Few comments: 1. This is an example in publication doc, but in fact it's not allowed. Should we change this example? +CREATE PUBLICATION active_departments FOR TABLE departme

RE: Alter all tables in schema owner fix

2021-12-02 Thread tanghy.f...@fujitsu.com
On Friday, December 3, 2021 1:31 PM vignesh C wrote: > > On Fri, Dec 3, 2021 at 9:53 AM Greg Nancarrow wrote: > > > > On Fri, Dec 3, 2021 at 2:06 PM vignesh C wrote: > > > > > > Currently while changing the owner of ALL TABLES IN SCHEMA > > > publication, it is not checked if the new owner has

RE: row filtering for logical replication

2021-12-06 Thread tanghy.f...@fujitsu.com
On Friday, December 3, 2021 10:09 AM Peter Smith wrote: > > On Thu, Dec 2, 2021 at 2:32 PM tanghy.f...@fujitsu.com > wrote: > > > > On Thursday, December 2, 2021 5:21 AM Peter Smith > wrote: > > > > > > PSA the v44* set of patches. > >

RE: [PATCH]Comment improvement in publication.sql

2021-12-07 Thread tanghy.f...@fujitsu.com
On Wednesday, December 8, 2021 1:49 PM, vignesh C wrote: > The patch no longer applies, could you post a rebased patch. Thanks for your kindly reminder. Attached a rebased patch. Some changes in v4 patch has been fixed by 5a28324, so I deleted those changes. Regards, Tang v5-0001-Fix-comment

RE: parallel vacuum comments

2021-12-13 Thread tanghy.f...@fujitsu.com
On Monday, December 13, 2021 2:12 PM Masahiko Sawada wrote: > > On Mon, Dec 13, 2021 at 2:09 PM Amit Kapila wrote: > > > > On Mon, Dec 13, 2021 at 10:33 AM Masahiko Sawada > wrote: > > > > > > On Fri, Dec 10, 2021 at 9:08 PM Amit Kapila > wrote: > > > > > > > > On Thu, Dec 9, 2021 at 6:05 PM

RE: row filtering for logical replication

2021-12-19 Thread tanghy.f...@fujitsu.com
On Wednesday, December 8, 2021 2:29 PM Amit Kapila wrote: > > On Mon, Dec 6, 2021 at 6:04 PM Euler Taveira wrote: > > > > On Mon, Dec 6, 2021, at 3:35 AM, Dilip Kumar wrote: > > > > On Mon, Dec 6, 2021 at 6:49 AM Euler Taveira wrote: > > > > > > On Fri, Dec 3, 2021, at 8:12 PM, Euler Taveira w

RE: row filtering for logical replication

2021-12-20 Thread tanghy.f...@fujitsu.com
On Monday, December 20, 2021 11:24 AM tanghy.f...@fujitsu.com > > On Wednesday, December 8, 2021 2:29 PM Amit Kapila > wrote: > > > > On Mon, Dec 6, 2021 at 6:04 PM Euler Taveira wrote: > > > > > > On Mon, Dec 6, 2021, at 3:35 AM, Dilip Kumar wrote: >

RE: row filtering for logical replication

2021-12-20 Thread tanghy.f...@fujitsu.com
On Monday, December 20, 2021 4:47 PM tanghy.f...@fujitsu.com wrote: > On Monday, December 20, 2021 11:24 AM tanghy.f...@fujitsu.com > > > > > On Wednesday, December 8, 2021 2:29 PM Amit Kapila > > wrote: > > > > > > On Mon, Dec 6, 2021 at 6:04 PM Eu

RE: row filtering for logical replication

2021-12-20 Thread tanghy.f...@fujitsu.com
On Tuesday, December 21, 2021 3:03 PM, tanghy.f...@fujitsu.com wrote: > To: Amit Kapila ; Euler Taveira > Cc: Dilip Kumar ; Peter Smith ; > Greg Nancarrow ; Hou, Zhijie/侯 志杰 > ; vignesh C ; Ajin Cherian > ; Rahila Syed ; Peter Eisentraut > ; Önder Kalacı ; > japin ; Mi

RE: row filtering for logical replication

2021-12-28 Thread tanghy.f...@fujitsu.com
On Mon, Dec 27, 2021 9:16 PM houzj.f...@fujitsu.com wrote: > > On Thur, Dec 23, 2021 4:28 PM Peter Smith wrote: > > Here is the v54* patch set: > > Attach the v55 patch set which add the following testcases in 0003 patch. > 1. Added a test to cover the case where TOASTed values are not include

RE: Failed transaction statistics to measure the logical replication progress

2021-12-30 Thread tanghy.f...@fujitsu.com
On Wednesday, December 22, 2021 6:14 PM osumi.takami...@fujitsu.com wrote: > > Attached the new patch v19. > Thanks for your patch. I think it's better if you could add this patch to the commitfest. Here are some comments: 1) + commit_count bigint + + + Number of tran

RE: [PATCH]Comment improvement in publication.sql

2021-12-30 Thread tanghy.f...@fujitsu.com
On Monday, December 13, 2021 11:53 PM vignesh C wrote: > > On Wed, Dec 8, 2021 at 11:07 AM tanghy.f...@fujitsu.com > wrote: > > > > On Wednesday, December 8, 2021 1:49 PM, vignesh C > wrote: > > > > > The patch no longer applies, could you post a rebase

RE: Optionally automatically disable logical replication subscriptions on error

2022-01-05 Thread tanghy.f...@fujitsu.com
On Wednesday, January 5, 2022 8:53 PM osumi.takami...@fujitsu.com wrote: > > On Tuesday, December 28, 2021 11:53 AM Wang, Wei/王 威 > wrote: > > On Thursday, December 16, 2021 8:51 PM osumi.takami...@fujitsu.com > > wrote: > > > Attached the updated patch v14. > > > > A comment to the timing of

RE: Support tab completion for upper character inputs in psql

2022-01-06 Thread tanghy.f...@fujitsu.com
On Thursday, January 6, 2022 11:57 PM, Tom Lane wrote: > > Peter Eisentraut writes: > > So the ordering of the suggested completions is different. I don't know > > offhand how that ordering is determined. Perhaps it's dependent on > > locale, readline version, or operating system. In any case

RE: Support tab completion for upper character inputs in psql

2022-01-06 Thread tanghy.f...@fujitsu.com
On Friday, January 7, 2022 1:08 PM, Japin Li wrote: > +/* > + * pg_string_tolower - Fold a string to lower case if the string is not > quoted > + * and only contains ASCII characters. > + * For German/Turkish etc text, no change will be made. > + * > + * The returned value has to be freed. > + */

RE: row filtering for logical replication

2022-01-11 Thread tanghy.f...@fujitsu.com
On Tuesday, January 11, 2022 10:16 AM houzj.f...@fujitsu.com wrote: > > Attach the v62 patch set which address the above comments and slightly > adjust the commit message in 0002 patch. > I saw a possible problem about Row-Filter tablesync SQL, which is related to partition table. If a parent

[PATCH]Add tab completion for foreigh table

2022-01-11 Thread tanghy.f...@fujitsu.com
Hi Attached a patch to improve the tab completion for foreigh table. Also modified some DOC description of ALTER TABLE at [1] in according with CREATE INDEX at [2]. In [1], we use "ALTER INDEX ATTACH PARTITION" In [2], we use "ALTER INDEX ... ATTACH PARTITION" I think the format in [2] is bett

RE: Skipping logical replication transactions on subscriber side

2022-01-12 Thread tanghy.f...@fujitsu.com
On Wed, Jan 12, 2022 2:02 PM Masahiko Sawada wrote: > > I've attached an updated patch that incorporated all comments I got so far. > Thanks for updating the patch. Here are some comments: 1) + Skip applying changes of the particular transaction. If incoming data Should "Skip" be "Skips

RE: [PATCH]Add tab completion for foreigh table

2022-01-12 Thread tanghy.f...@fujitsu.com
On Thursday, January 13, 2022 12:38 PM, Fujii Masao wrote: > Isn't it better to tab-complete not only "PARTITION OF" but also "(" for > CREATE > FOREIGN TABLE? Thanks for your review. Left bracket completion added. > IMO it's better to make the docs changes in separate patch because they are

RE: Column Filtering in Logical Replication

2022-01-14 Thread tanghy.f...@fujitsu.com
On Friday, January 14, 2022 7:52 PM Amit Kapila wrote: > > On Wed, Jan 12, 2022 at 2:40 AM Justin Pryzby wrote: > > > > Is there any coordination between the "column filter" patch and the "row > > filter" patch ? Are they both on track for PG15 ? Has anybody run them > > together ? > > > > Th

RE: Parallel Inserts in CREATE TABLE AS

2021-03-19 Thread tanghy.f...@fujitsu.com
From: Bharath Rupireddy >I analyzed performance of parallel inserts in CTAS for different cases >with tuple size 32bytes, 59bytes, 241bytes and 1064bytes. We could >gain if the tuple sizes are lower. But if the tuple size is larger >i..e 1064bytes, there's a regression with parallel inserts. Than

RE: [HACKERS] logical decoding of two-phase transactions

2021-03-22 Thread tanghy.f...@fujitsu.com
On Sunday, March 21, 2021 4:37 PM Amit Kapila wrote: >I have further updated the patch to implement unique GID on the >subscriber-side as discussed in the nearby thread [1]. I did some tests(cross version & synchronous) on the latest patch set v65*, all tests passed. Here is the detail, please t

RE: Support tab completion for upper character inputs in psql

2021-03-22 Thread tanghy.f...@fujitsu.com
On Tuesday, March 16, 2021 5:20 AM, Peter Eisentraut wrote: >The cases that complete with a query >result are not case insensitive right now. This affects things like > >UPDATE T > >as well. I think your first patch was basically right. But we need to >understand that this affects all com

RE: Logical Replication vs. 2PC

2021-03-24 Thread tanghy.f...@fujitsu.com
On Sunday, March 21, 2021 4:40 PM Amit Kapila wrote: >I have enhanced the patch for 2PC implementation on the >subscriber-side as per the solution discussed here [1]. FYI. I did the confirmation for the solution of unique GID problem raised at [1]. This problem in V61-patches at [2] is fixed in

Copyright update for nbtsearch.c

2021-03-29 Thread tanghy.f...@fujitsu.com
Hi Found one code committed at 2021.01.13 with copyright 2020. Fix it in the attached patch. Regards, Tang 0001-Update-copyright-for-2021.patch Description: 0001-Update-copyright-for-2021.patch

RE: Copyright update for nbtsearch.c

2021-03-29 Thread tanghy.f...@fujitsu.com
On Tue, Mar 30, 2021 at 11:09:47AM +0800, Julien Rouhaud wrote: > This is actually gistfuncs.c. Thanks, you are right. There's typo in the mail title. Sorry for your confusion. On Tuesday, March 30, 2021 12:08 PM, Michael Paquier wrote: >Thanks. If I look at the top of HEAD, it is not the only

RE: Support tab completion for upper character inputs in psql

2021-04-01 Thread tanghy.f...@fujitsu.com
On Wednesday, March 31, 2021 4:05 AM, David Zhang wrote > 8 postgres=# update tbl SET DATA = > 9 > 10 postgres=# update TBL SET > 11 > 12 postgres=# > >So, as you can see the difference is between line 8 and 10 in case 2. It >looks like the lowercase can auto complete more than the upperc

Table refer leak in logical replication

2021-04-05 Thread tanghy.f...@fujitsu.com
Hi I met a problem about trigger in logical replication. I created a trigger after inserting data at subscriber, but there is a warning in the log of subscriber when the trigger fired: WARNING: relcache reference leak: relation "xxx" not closed. Example of the procedure: --publisher-- c

RE: Table refer leak in logical replication

2021-04-05 Thread tanghy.f...@fujitsu.com
On Tuesday, April 6, 2021 2:25 PM Amit Langote wrote: >While updating the patch to do so, it occurred to me that maybe we >could move the ExecInitResultRelation() call into >create_estate_for_relation() too, in the spirit of removing >duplicative code. See attached updated patch. Thanks for your

Truncate in synchronous logical replication failed

2021-04-06 Thread tanghy.f...@fujitsu.com
Hi I met a problem in synchronous logical replication. The client hangs when TRUNCATE TABLE at publisher. Example of the procedure: --publisher-- create table test (a int primary key); create publication pub for table test; --subscriber-- create table test (a int primary key); c

RE: Truncate in synchronous logical replication failed

2021-04-07 Thread tanghy.f...@fujitsu.com
On Wednesday, April 7, 2021 5:28 PM Amit Kapila wrote >Can you please check if the behavior is the same for PG-13? This is >just to ensure that we have not introduced any bug in PG-14. Yes, same failure happens at PG-13, too. Regards, Tang

RE: Added schema level support for publication.

2021-10-13 Thread tanghy.f...@fujitsu.com
On Wednesday, October 13, 2021 4:10 PM Greg Nancarrow wrote: > Also, I found the following scenario where the data is double-published: > > (1) PUB: CREATE PUBLICATION pub FOR TABLE sch1.sale_201901, TABLE > sch1.sale_201902 WITH (publish_via_partition_root=true); > (2) SUB: CREATE SUBSCRIPTIO

RE: Added schema level support for publication.

2021-10-18 Thread tanghy.f...@fujitsu.com
On Monday, October 18, 2021 8:23 PM vignesh C wrote: > > Thanks for the comments, the attached v42 patch has the fixes for the same. Thanks for your new patch. I tried your patch and found that the permission check for superuser didn't work. For example: postgres=# create role r1; CREATE ROLE

RE: Added schema level support for publication.

2021-10-18 Thread tanghy.f...@fujitsu.com
On Tuesday, October 19, 2021 12:57 PM Amit Kapila wrote: > > On Tue, Oct 19, 2021 at 9:15 AM tanghy.f...@fujitsu.com > wrote: > > > > On Monday, October 18, 2021 8:23 PM vignesh C > wrote: > > > > > > Thanks for the comments, the attached v

RE: Added schema level support for publication.

2021-10-19 Thread tanghy.f...@fujitsu.com
On Tuesday, October 19, 2021 11:42 PM vignesh C wrote: > > This issue got induced in the v42 version, attached v43 patch has the > fixes for the same. > Thanks for your new patch. I confirmed that this issue has be fixed. All regression tests passed. I also tested V43 in some other scenarios

RE: Skipping logical replication transactions on subscriber side

2021-10-26 Thread tanghy.f...@fujitsu.com
On Thursday, October 21, 2021 12:59 PM Masahiko Sawada wrote: > > I've attached updated patches. In this version, in addition to the > review comments I go so far, I've changed the view name from > pg_stat_subscription_errors to pg_stat_subscription_workers as per the > discussion on including x

RE: Added schema level support for publication.

2021-10-28 Thread tanghy.f...@fujitsu.com
On Friday, October 29, 2021 12:35 PM, Amit Kapila wrote: > > On Thu, Oct 28, 2021 at 9:55 AM vignesh C wrote: > > > > Thanks for committing the patch, please find the remaining patches attached. > > Thanks Hou Zhijie and Greg Nancarrow for sharing a few comments > > offline, I have fixed those

RE: Skipping logical replication transactions on subscriber side

2021-11-02 Thread tanghy.f...@fujitsu.com
On Friday, October 29, 2021 1:24 PM Masahiko Sawada wrote: > > I've attached a new version patch. Since the syntax of skipping > transaction id is under the discussion I've attached only the error > reporting patch for now. > > Thanks for your patch. Some comments on 026_error_report.pl file.

RE: row filtering for logical replication

2021-11-08 Thread tanghy.f...@fujitsu.com
On Friday, November 5, 2021 1:14 PM, Peter Smith wrote: > > PSA new set of v37* patches. > Thanks for your patch. I have a problem when using this patch. The document about "create publication" in patch says: The WHERE clause should contain only columns that are part of the primary key

[BUG]Invalidate relcache when setting REPLICA IDENTITY

2021-11-09 Thread tanghy.f...@fujitsu.com
Hi I think I found a bug related to logical replication(REPLICA IDENTITY in specific). If I change REPLICA IDENTITY after creating publication, the DELETE/UPDATE operations won't be replicated as expected. For example: -- publisher CREATE TABLE tbl(a int, b int); ALTER TABLE tbl ALTER COLUMN

RE: Logical replication timeout problem

2021-11-12 Thread tanghy.f...@fujitsu.com
On Friday, November 12, 2021 2:24 PM Amit Kapila wrote: > > On Thu, Nov 11, 2021 at 11:15 PM Fabrice Chapuis > wrote: > > > > Hello, > > Our lab is ready now. Amit, I compile Postgres 10.18 with your patch.Tang, > > I > used your script to configure logical replication between 2 databases and

RE: row filtering for logical replication

2021-11-14 Thread tanghy.f...@fujitsu.com
On Wednesday, November 10, 2021 7:46 AM Peter Smith wrote: > > On Tue, Nov 9, 2021 at 2:03 PM tanghy.f...@fujitsu.com > wrote: > > > > On Friday, November 5, 2021 1:14 PM, Peter Smith > wrote: > > > > > > PSA new set of v37* patches. > > > &

RE: row filtering for logical replication

2021-11-14 Thread tanghy.f...@fujitsu.com
On Friday, November 12, 2021 6:20 PM Ajin Cherian wrote: > > Attaching version 39- > Thanks for the new patch. I met a problem when using "ALTER PUBLICATION ... SET TABLE ... WHERE ...", the publisher was crashed after executing this statement. Here is some information about this problem. St

RE: row filtering for logical replication

2021-11-16 Thread tanghy.f...@fujitsu.com
On Friday, November 12, 2021 6:20 PM Ajin Cherian wrote: > > Attaching version 39- > I met another problem when filtering out with the operator '~'. Data can't be replicated as expected. For example: -- publisher create table t (a text primary key); create publication pub for table t where (a ~

RE: Skipping logical replication transactions on subscriber side

2021-11-18 Thread tanghy.f...@fujitsu.com
On Tuesday, November 16, 2021 2:31 PM Masahiko Sawada wrote: > > Right. I've fixed this issue and attached an updated patch. > > Thanks for your patch. I read the discussion about stats entries for table sync worker[1], the statistics are retained after table sync worker finished its jobs and

RE: row filtering for logical replication

2021-11-22 Thread tanghy.f...@fujitsu.com
On Thursday, November 18, 2021 9:34 AM Peter Smith wrote: > > PSA new set of v40* patches. > I found a problem on v40. The check for Replica Identity in WHERE clause is not working properly. For example: postgres=# create table tbl(a int primary key, b int); CREATE TABLE postgres=# create pu

[BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-23 Thread tanghy.f...@fujitsu.com
Hi, I think I found a problem related to replica identity. According to PG doc at [1], replica identity includes only columns marked NOT NULL. But in fact users can accidentally break this rule as follows: create table tbl (a int not null unique); alter table tbl replica identity using INDEX t

RE: [BUG]Missing REPLICA IDENTITY check when DROP NOT NULL

2021-11-24 Thread tanghy.f...@fujitsu.com
On Wed, Nov 24, 2021 7:29 PM, Michael Paquier wrote: > > On Wed, Nov 24, 2021 at 07:04:51AM +, tanghy.f...@fujitsu.com wrote: > > create table tbl (a int not null unique); > > alter table tbl replica identity using INDEX tbl_a_key; > > alter table tbl alter column a d

RE: Skipping logical replication transactions on subscriber side

2021-11-25 Thread tanghy.f...@fujitsu.com
On Friday, November 26, 2021 9:30 AM Masahiko Sawada wrote: > > Indeed. Attached an updated patch. Thanks! Thanks for your patch. A small comment: + OID of the relation that the worker is synchronizing; null for the + main apply worker Should we modify it to "OID of the relation t

RE: row filtering for logical replication

2021-11-29 Thread tanghy.f...@fujitsu.com
On Thursday, November 25, 2021 11:22 AM Peter Smith wrote: > > Thanks for all the review comments so far! We are endeavouring to keep > pace with them. > > All feedback is being tracked and we will fix and/or reply to everything ASAP. > > Meanwhile, PSA the latest set of v42* patches. > > Thi

RE: Skipping logical replication transactions on subscriber side

2022-01-17 Thread tanghy.f...@fujitsu.com
On Mon, Jan 17, 2022 2:18 PM Masahiko Sawada wrote: > > I've attached an updated patch. Please review it. > Thanks for updating the patch. Few comments: 1) /* Two_phase is only supported in v15 and higher */ if (pset.sversion >= 15) a

RE: Support tab completion for upper character inputs in psql

2022-01-19 Thread tanghy.f...@fujitsu.com
On Sunday, January 16, 2022 3:51 AM, Tom Lane said: > Peter Eisentraut writes: > > The rest of the patch seems ok in principle, since AFAICT enums are the > > only query result in tab-complete.c that are not identifiers and thus > > subject to case issues. > > I spent some time looking at this p

RE: row filtering for logical replication

2022-01-20 Thread tanghy.f...@fujitsu.com
On Thu, Jan 20, 2022 9:13 AM houzj.f...@fujitsu.com wrote: > Attach the V68 patch set which addressed the above comments and changes. > The version patch also fix the error message mentioned by Greg[1] > I saw a problem about this patch, which is related to Replica Identity check. For example:

RE: Skipping logical replication transactions on subscriber side

2022-01-23 Thread tanghy.f...@fujitsu.com
On Fri, Jan 21, 2022 7:55 PM Amit Kapila wrote: > > 2. > +stop_skipping_changes(bool clear_subskipxid, XLogRecPtr origin_lsn, > + TimestampTz origin_timestamp) > +{ > + Assert(is_skipping_changes()); > + > + ereport(LOG, > + (errmsg("done skipping logical replication transaction %u", > + skip_x

RE: Support tab completion for upper character inputs in psql

2022-01-24 Thread tanghy.f...@fujitsu.com
On Monday, January 24, 2022 6:36 PM, Peter Eisentraut wrote: > The way your patch works now is that the case-insensitive behavior you > are implementing only works if the client encoding is a single-byte > encoding. This isn't what downcase_identifier() does; > downcase_identifier() always works

RE: Support tab completion for upper character inputs in psql

2022-01-25 Thread tanghy.f...@fujitsu.com
On Tuesday, January 25, 2022 6:44 PM, Julien Rouhaud wrote: > Thanks for updating the patch. When you do so, please check and update the > commitfest entry accordingly to make sure that people knows it's ready for > review. I'm switching the entry to Needs Review. > Thanks for your reminder. I

RE: Support tab completion for upper character inputs in psql

2022-01-27 Thread tanghy.f...@fujitsu.com
On Friday, January 28, 2022 5:24 AM, Tom Lane wrote: > Here's a fleshed-out patch series for this idea. Thanks for you patch. I did some tests on it and here are something cases I feel we need to confirm whether they are suitable. 1) postgres=# create table atest(id int, "iD" int, "ID" int); 2

RE: Support tab completion for upper character inputs in psql

2022-01-29 Thread tanghy.f...@fujitsu.com
On Saturday, January 29, 2022 1:03 AM, Tom Lane wrote: > "tanghy.f...@fujitsu.com" writes: > > I did some tests on it and here are something cases I feel we need to > > confirm > > whether they are suitable. > > > 1) postgres=# create table atest(id

RE: Support tab completion for upper character inputs in psql

2022-01-29 Thread tanghy.f...@fujitsu.com
On Saturday, January 29, 2022 7:17 AM, Tom Lane wrote: > Sigh ... per the cfbot, this was already blindsided by 95787e849. > As I said, I don't want to sit on this for very long. Thanks for your V16 patch, I tested it. The results LGTM. Regards, Tang

RE: [BUG]Update Toast data failure in logical replication

2022-02-08 Thread tanghy.f...@fujitsu.com
On Mon, Feb 7, 2022 2:55 PM Amit Kapila wrote: > > On Sat, Feb 5, 2022 at 6:10 AM Amit Kapila wrote: > > > > On Fri, Feb 4, 2022 at 9:06 PM Alvaro Herrera > > wrote: > > > > > > > > > I have some suggestions > > > on the comments and docs though. > > > > > > > Thanks, your suggestions look go

RE: [BUG]Update Toast data failure in logical replication

2022-02-08 Thread tanghy.f...@fujitsu.com
On Tue, Feb 8, 2022 3:18 AM Andres Freund wrote: > > On 2022-02-07 08:44:00 +0530, Amit Kapila wrote: > > Right, and it is getting changed. We are just printing the first 200 > > characters (by using SQL [1]) from the decoded tuple so what is shown > > in the results is the initial 200 bytes. >

RE: [BUG]Update Toast data failure in logical replication

2022-02-10 Thread tanghy.f...@fujitsu.com
On Thu, Feb 10, 2022 9:34 PM Amit Kapila wrote: > > On Mon, Feb 7, 2022 at 1:27 PM Dilip Kumar wrote: > > > > On Mon, Feb 7, 2022 at 12:25 PM Amit Kapila > wrote: > > > > > > Attached please find the modified patches. > > > > I have looked into the latest modification and back branch patches an

RE: Failed transaction statistics to measure the logical replication progress

2022-02-17 Thread tanghy.f...@fujitsu.com
On Wed, Jan 12, 2022 8:35 PM osumi.takami...@fujitsu.com wrote: > > The attached v21 has a couple of other minor updates > like a modification of error message text. > > Thanks for updating the patch. Here are some comments. 1) I saw the following description about pg_stat_subscription_worke

RE: Support tab completion for upper character inputs in psql

2021-04-14 Thread tanghy.f...@fujitsu.com
On Thursday, April 8, 2021 4:14 PM, Peter Eisentraut wrote >Seeing the tests you provided, it's pretty obvious that the current >behavior is insufficient. I think we could probably think of a few more >tests, for example exercising the "If case insensitive matching was >requested initially,

RE: Support tab completion for upper character inputs in psql

2021-04-22 Thread tanghy.f...@fujitsu.com
On Wednesday, April 21, 2021 1:24 PM, Peter Smith Wrote >I tried playing a bit with your psql patch V5 and I did not find any >problems - it seemed to work as advertised. > >Below are a few code review comments. Thanks for you review. I've updated the patch to V6 according to your comments. >1.

use pg_strncasecmp to replace strncmp when compare "pg_"

2021-04-22 Thread tanghy.f...@fujitsu.com
Hi When try to improve the tab compleation feature in [1], I found an existing problem and a typo. The patch was attached, please kindly to take a look at it. Thanks. [1] https://www.postgresql.org/message-id/OS0PR01MB61131A4347D385F02F60E123FB469%40OS0PR01MB6113.jpnprd01.prod.outlook.com Regar

RE: Support tab completion for upper character inputs in psql

2021-04-26 Thread tanghy.f...@fujitsu.com
Hi I've updated the patch to V7 based on the following comments. On Friday, April 23, 2021 11:58 AM, Kyotaro Horiguchi wrote >All usages of pg_string_tolower don't need a copy. >So don't we change the function to in-place converter? Refer to your later discussion with Tom. Keep the code as i

RE: use pg_strncasecmp to replace strncmp when compare "pg_"

2021-04-26 Thread tanghy.f...@fujitsu.com
On Friday, April 23, 2021 2:06 PM, Tom Lane wrote >>Kyotaro Horiguchi writes: >> That doesn't matter at all for now since we match schema identifiers >> case-sensitively. Maybe it should be a part of the patch in [1]. > >Yeah --- maybe this'd make sense as part of a full patch to improve >tab-c

RE: Truncate in synchronous logical replication failed

2021-04-26 Thread tanghy.f...@fujitsu.com
On Tuesday, April 27, 2021 1:17 PM, Amit Kapila wrote >Seeing no other suggestions, I have pushed this in HEAD only. Thanks! Sorry for the later reply on this issue. I tested with the latest HEAD, the issue is fixed now. Thanks a lot. Regards Tang

RE: [BUG] "FailedAssertion" reported when streaming in logical replication

2021-04-27 Thread tanghy.f...@fujitsu.com
> I have modified the patch based on the above comments. Thanks for your patch. I tested again after applying your patch and the problem is fixed. Regards Tang

[BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-04-28 Thread tanghy.f...@fujitsu.com
Hi I met an assertion failure at the publisher in lazy_scan_heap() when synchronous running logical replication. Could someone please take a look at it? Here's what I did to produce the problem. First, use './configure --enable-cassert' to build the PG. Then, I created multiple publications at

RE: [BUG]"FailedAssertion" reported in lazy_scan_heap() when running logical replication

2021-04-28 Thread tanghy.f...@fujitsu.com
On Thursday, April 29, 2021 1:22 PM, Peter Geoghegan wrote >Is setting all_visible_according_to_vm false as below enough to avoid >the assertion failure? > >diff --git a/src/backend/access/heap/vacuumlazy.c >b/src/backend/access/heap/vacuumlazy.c >index c3fc12d76c..76c17e063e 100644 >--- a/src/ba

RE: [BUG] "FailedAssertion" reported when streaming in logical replication

2021-04-28 Thread tanghy.f...@fujitsu.com
On Thursday, April 29, 2021 3:18 PM, Dilip Kumar wrote >I tried to think about how to write a test case for this scenario, but >I think it will not be possible to generate an automated test case for this. >Basically, we need 2 concurrent transactions and out of that, >we need one transaction w

Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-09 Thread tanghy.f...@fujitsu.com
Hi When using psql help with SQL commands, I found an inconsistency tab-completion for command "DELETE" as follows. =# \h de[TAB] deallocate declare delete from =# \help[TAB] ABORT CLUSTERDELETE FROM =# \help[ENTER] Available help: ... ANALYZE

RE: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-09 Thread tanghy.f...@fujitsu.com
On Monday, May 10, 2021 2:48 PM, Julien Rouhaud worte >I think the behavior now is correct. The goal of autocompletion is to save >keystrokes and time. As the only valid keyword after a DELETE (at least in a >DeleteStmt) is FROM, it's a good thing that you get back "DELETE FROM" directly >rathe

RE: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-10 Thread tanghy.f...@fujitsu.com
On Monday, May 10, 2021 4:15 PM, Julien Rouhaud wrote >We should change all to DELETE FROM (apart from \help of course), and same for >INSERT, change to INSERT INTO everywhere it makes sense. Thanks for the reply. Your advice sounds reasonable to me. So I tried to change all "DELETE" to "DELETE F

RE: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread tanghy.f...@fujitsu.com
On Tuesday, May 11, 2021 2:53 PM, Michael Paquier wrote >else if (TailMatches("DELETE", "FROM", MatchAny)) >COMPLETE_WITH("USING", "WHERE"); >- /* XXX: implement tab completion for DELETE ... USING */ > >Why are you removing that? This sentence is still true, no? IIRC, XXX in c

RE: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread tanghy.f...@fujitsu.com
On Tuesday, May 11, 2021 5:44 PM, Dilip Kumar wrote: >But your patch is doing nothing to add the implementation for DELETE.. >USING. Basically, the tab completion support for DELETEUSING is >still pending right? I see, maybe I have a misunderstanding here, I thought tab completion for "DELE

RE: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread tanghy.f...@fujitsu.com
On Tuesday, May 11, 2021 6:55 PM, Dilip Kumar wrote: >Basically, it just complete with USING, now after USING tab-completion >support is not yet there, e.g. DELETE FROM t1 USING t1 WHERE cond. >but the current code will not suggest anything after USING. Thanks for your kindly explanation. That's

RE: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-17 Thread tanghy.f...@fujitsu.com
On Monday, May 17, 2021 5:47 PM, Amit Kapila wrote > +$node_publisher->safe_psql('postgres', > + "ALTER SYSTEM SET synchronous_standby_names TO 'any 2(sub5_1, > sub5_2)'"); > +$node_publisher->safe_psql('postgres', "SELECT pg_reload_conf()"); > > Do you really need these steps to reproduce the pr

RE: [HACKERS] logical decoding of two-phase transactions

2021-05-18 Thread tanghy.f...@fujitsu.com
Hi Ajin >The above patch had some changes missing which resulted in some tap >tests failing. Sending an updated patchset. Keeping the patchset >version the same. Thanks for your patch. I see a problem about Segmentation fault when using it. Please take a look at this. The steps to reproduce the

RE: "ERROR: deadlock detected" when replicating TRUNCATE

2021-05-20 Thread tanghy.f...@fujitsu.com
On Thursday, May 20, 2021 3:05 PM, Amit Kapila wrote: > Okay, I have prepared the patches for all branches (11...HEAD). Each > version needs minor changes in the test, the code doesn't need much > change. Some notable changes in the tests: > 1. I have removed the conf change for max_logical_replic

RE: [HACKERS] logical decoding of two-phase transactions

2021-05-24 Thread tanghy.f...@fujitsu.com
> > 13. > > @@ -507,7 +558,16 @@ CreateSubscription(CreateSubscriptionStmt *stmt, > > bool isTopLevel) > > { > > Assert(slotname); > > > > - walrcv_create_slot(wrconn, slotname, false, > > + /* > > + * Even if two_phase is set, don't create the slot with > > + * two-phase enabled. Will enable i

RE: Added schema level support for publication.

2021-05-25 Thread tanghy.f...@fujitsu.com
On Monday, May 24, 2021 at 8:31 PM vignesh C wrote: > The earlier patch does not apply on the head. The v4 patch attached > has the following changes: > a) Rebased it on head. b) Removed pubschemas, pubtables columns and > replaced it with pubtype in pg_publication table. c) List the schemas > in

RE: [HACKERS] logical decoding of two-phase transactions

2021-05-26 Thread tanghy.f...@fujitsu.com
On Wed, May 26, 2021 10:13 PM Ajin Cherian wrote: > > I've attached a patch that fixes this issue. Do test and confirm. > Thanks for your patch. I have tested and confirmed that the issue I reported has been fixed. Regards Tang

[BUG]Update Toast data failure in logical replication

2021-05-27 Thread tanghy.f...@fujitsu.com
Hi I think I just found a bug in logical replication. Data couldn't be synchronized while updating toast data. Could anyone take a look at it? Here is the steps to proceduce the BUG: --publisher-- CREATE TABLE toasted_key ( id serial, toasted_key text PRIMARY KEY, toasted_col

RE: [BUG]Update Toast data failure in logical replication

2021-05-27 Thread tanghy.f...@fujitsu.com
On Friday, May 28, 2021 2:16 PM, tanghy.f...@fujitsu.com wrote: >I think I just found a bug in logical replication. Data couldn't be >synchronized while updating toast data. >Could anyone take a look at it? FYI. The problem also occurs in PG-13. I will try to check from which v

RE: [BUG]Update Toast data failure in logical replication

2021-05-28 Thread tanghy.f...@fujitsu.com
On Friday, May 28, 2021 3:02 PM, tanghy.f...@fujitsu.com wrote: > FYI. The problem also occurs in PG-13. I will try to check from which version > it > got introduced. I reproduced it in PG-10,11,12,13. I think the problem has been existing since Logical replication introduced

RE: [BUG]Update Toast data failure in logical replication

2021-05-30 Thread tanghy.f...@fujitsu.com
On Friday, May 28, 2021 6:51 PM, Dilip Kumar wrote: > Seems you did not set the replica identity for updating the tuple. > Try this before updating, and it should work. Thanks for your reply. I tried it. > ALTER TABLE toasted_key REPLICA IDENTITY USING INDEX toasted_key_pkey; This didn't work.

RE: [BUG]Update Toast data failure in logical replication

2021-05-31 Thread tanghy.f...@fujitsu.com
On Mon, May 31, 2021 5:12 PM Dilip Kumar wrote: > > The problem is if the key attribute is not changed we don't log it as > it should get logged along with the updated tuple, but if it is > externally stored then the complete key will never be logged because > there is no log from the toast table

RE: [BUG]Update Toast data failure in logical replication

2021-05-31 Thread tanghy.f...@fujitsu.com
Hi I have some questions with your patch. Here are two cases I used to check the bug. Case1(PK toasted_key is short), data could be synchronized on HEAD. --- INSERT INTO toasted_key(toasted_key, toasted_col1) VALUES('111', repeat('9876543210', 200)); UPDATE toasted_key SET toasted_c

RE: [BUG]Update Toast data failure in logical replication

2021-06-02 Thread tanghy.f...@fujitsu.com
On Wed, Jun 2, 2021 2:44 PM Dilip Kumar wrote: > Attached patch fixes that, I haven't yet added the test case. Once > someone confirms on the approach then I will add a test case to the > patch. key_tuple = heap_form_tuple(desc, values, nulls); *copy = true; ...

tab-complete for CREATE TYPE ... SUBSCRIPT

2021-06-02 Thread tanghy.f...@fujitsu.com
Hi Attached a patch to support tab completion for CREATE TYPE ... SUBSCRIPT introduced at c7aba7c14e. Regards, Tang 0001-psql-tab-complete-CREATE-TYPE-.-SUBSCRIPT.patch Description: 0001-psql-tab-complete-CREATE-TYPE-.-SUBSCRIPT.patch

  1   2   >