Re: pgbench: using prepared BEGIN statement in a pipeline could cause an error

2021-07-16 Thread Fabien COELHO
Hello Yugo-san, [...] One way to avoid these errors is to send Parse messages before pipeline mode starts. I attached a patch to fix to prepare commands at starting of a script instead of at the first execution of the command. What do you think? ISTM that moving prepare out of command ex

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-16 Thread Bharath Rupireddy
On Fri, Jul 16, 2021 at 4:47 PM Dean Rasheed wrote: > > On Fri, 16 Jul 2021 at 10:26, Bharath Rupireddy > wrote: > > > > Thanks. The v5 patch LGTM. > > OK, I'll push that in a while. Thanks. > There are a few cases where def->defname isn't necessarily the name > that was specified by the user (

Re: What are exactly bootstrap processes, auxiliary processes, standalone backends, normal backends(user sessions)?

2021-07-16 Thread Bharath Rupireddy
On Thu, Jul 15, 2021 at 8:17 PM Justin Pryzby wrote: > > On Fri, Jul 09, 2021 at 09:24:19PM +0530, Bharath Rupireddy wrote: > > I've always had a hard time distinguishing various types of > > processes/terms used in postgres. I look at the source code every time > > to understand them, yet I don't

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-16 Thread David Rowley
On Fri, 16 Jul 2021 at 22:00, David Rowley wrote: > > On Fri, 16 Jul 2021 at 18:04, David Rowley wrote: > > What we maybe could consider instead would be to pick the first Aggref > > then look for the most sorted derivative of that then tally up the > > number of Aggrefs that can be sorted using

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Justin Pryzby
On Fri, Jul 16, 2021 at 06:01:12PM -0400, Alvaro Herrera wrote: > On 2021-Jul-16, Justin Pryzby wrote: > > CREATE TABLE p(i int) PARTITION BY RANGE(i); > > CREATE TABLE p1 PARTITION OF p FOR VALUES FROM (1)TO(2); > > CREATE FUNCTION foo() returns trigger LANGUAGE plpgsql AS $$begin end$$; > > CREAT

Re: speed up verifying UTF-8

2021-07-16 Thread John Naylor
Forgot the attachments... -- John Naylor EDB: http://www.enterprisedb.com v17-0001-Rewrite-pg_utf8_verifystr-for-speed.patch Description: Binary data v17-0004-Second-attempt-at-addressing-performance-regress.patch Description: Binary data v17-0002-Use-integer-chunk-for-fast-path-multibyte-ch

Re: speed up verifying UTF-8

2021-07-16 Thread John Naylor
My v16 experimental patches were a bit messy, so I've organized an experimental series that applies cumulatively, to try to trace the effects of various things. v17-0001 is the same as v14. 0002 is a stripped-down implementation of Amit's chunk idea for multibyte, and it's pretty good on x86. On P

Re: pg14b1 stuck in lazy_scan_prune/heap_page_prune of pg_statistic

2021-07-16 Thread Peter Geoghegan
On Wed, Jun 16, 2021 at 1:21 PM Peter Geoghegan wrote: > Oh yeah, I think that I get it now. Tell me if this sounds right to you: > > It's not so much that HeapTupleSatisfiesVacuum() "disagrees" with > heap_prune_satisfies_vacuum() in a way that actually matters to > VACUUM. While there does seem

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Alvaro Herrera
On 2021-Jul-16, Justin Pryzby wrote: > CREATE TABLE p(i int) PARTITION BY RANGE(i); > CREATE TABLE p1 PARTITION OF p FOR VALUES FROM (1)TO(2); > CREATE FUNCTION foo() returns trigger LANGUAGE plpgsql AS $$begin end$$; > CREATE TRIGGER x AFTER DELETE ON p1 EXECUTE FUNCTION foo(); > CREATE TRIGGER x

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Alvaro Herrera
I noticed that in pg_dump --clean, we were still emitting DROP lines for the triggers in the partitions, which raises errors; so I emptied that too. On 2021-Jul-14, Alvaro Herrera wrote: > Looking over the tests added by your (Justin's) patch, there was a > problem checking for non-existance of t

Re: row filtering for logical replication

2021-07-16 Thread Alvaro Herrera
On 2021-Jul-16, Greg Nancarrow wrote: > On Fri, Jul 16, 2021 at 3:50 PM Amit Kapila wrote: > > > > I am not so sure about different filters for old and new rows but it > > makes sense to by default apply the filter to both old and new rows. > > Then also provide a way for user to specify if the f

Re: [PROPOSAL] new diagnostic items for the dynamic sql

2021-07-16 Thread Pavel Stehule
Hi pá 16. 7. 2021 v 21:47 odesílatel Dinesh Chemuduru napsal: > Hi Everyone, > > We would like to propose the below 2 new plpgsql diagnostic items, > related to parsing. Because, the current diag items are not providing > the useful diagnostics about the dynamic SQL statements. > > 1. PG_PARSE_S

[PROPOSAL] new diagnostic items for the dynamic sql

2021-07-16 Thread Dinesh Chemuduru
Hi Everyone, We would like to propose the below 2 new plpgsql diagnostic items, related to parsing. Because, the current diag items are not providing the useful diagnostics about the dynamic SQL statements. 1. PG_PARSE_SQL_STATEMENT (returns parse failed sql statement) 2. PG_PARSE_SQL_STATEMENT_P

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Justin Pryzby
On Fri, Jul 16, 2021 at 02:15:26PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I think there's still an issue that comments on child triggers aren't > > preserved, right ? > > Do we care? That seems like messing with a system-internal object. > In general we won't promise to preserve the

Re: cleaning up PostgresNode.pm

2021-07-16 Thread Andrew Dunstan
On 6/28/21 1:02 PM, Andrew Dunstan wrote: > On 4/24/21 3:14 PM, Alvaro Herrera wrote: >> On 2021-Apr-24, Andrew Dunstan wrote: >> >>> I would like to undertake some housekeeping on PostgresNode.pm. >>> >>> 1. OO modules in perl typically don't export anything. We should remove >>> the export setti

Re: Consider parallel for lateral subqueries with limit

2021-07-16 Thread James Coleman
On Thu, May 27, 2021 at 9:01 PM Greg Nancarrow wrote: > > On Tue, Dec 8, 2020 at 10:46 AM James Coleman wrote: > > > > While I haven't actually tracked down to guarantee this is handled > > elsewhere, a thought experiment -- I think -- shows it must be so. > > Here's why: suppose we don't have a

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Tom Lane
Justin Pryzby writes: > I think there's still an issue that comments on child triggers aren't > preserved, right ? Do we care? That seems like messing with a system-internal object. In general we won't promise to preserve the results of doing so. regards, tom lane

Re: CREATE TABLE .. PARTITION OF fails to preserve tgenabled for inherited row triggers

2021-07-16 Thread Justin Pryzby
Thanks for handling this. I think there's still an issue that comments on child triggers aren't preserved, right ? -- Justin

Re: pg_upgrade test for binary compatibility of core data types

2021-07-16 Thread Jacob Champion
On Fri, 2021-04-30 at 13:33 -0500, Justin Pryzby wrote: > On Sat, Mar 06, 2021 at 03:01:43PM -0500, Tom Lane wrote: > > v4-0001 mostly teaches test.sh about specific changes that have to be > > made to historic versions of the regression database to allow them > > to be reloaded into current server

Re: Logical replication - schema change not invalidating the relation cache

2021-07-16 Thread vignesh C
On Sat, Jul 3, 2021 at 11:23 AM Dilip Kumar wrote: > > On Fri, Jul 2, 2021 at 12:03 PM Dilip Kumar wrote: > > > > Yeah, this looks like a bug. I will look at the patch. > > > > While looking into this, I think the main cause of the problem is that > schema rename does not invalidate the relation

Re: psql tab auto-complete for CREATE PUBLICATION

2021-07-16 Thread vignesh C
On Fri, Jul 9, 2021 at 1:06 PM Peter Smith wrote: > > I found that the psql tab auto-complete was not working for some cases > of CREATE PUBLICATION [1]. > > CREATE PUBLICATION name > [ FOR TABLE [ ONLY ] table_name [ * ] [, ...] > | FOR ALL TABLES ] > [ WITH ( publication_parameter

Re: proposal: enhancing plpgsql debug API - returns text value of variable content

2021-07-16 Thread Pavel Stehule
Hi pá 16. 7. 2021 v 15:05 odesílatel Aleksander Alekseev < aleksan...@timescale.com> napsal: > Hi Pavel, > > > I would like to print content of variables - and now, I have to go some > > deeper than I would like. I need to separate between scalar, row, and > > record variables. PLpgSQL has code f

Re: pg_upgrade test for binary compatibility of core data types

2021-07-16 Thread Jacob Champion
On Fri, 2021-07-16 at 16:21 +, Jacob Champion wrote: > On Fri, 2021-04-30 at 13:33 -0500, Justin Pryzby wrote: > > On Sat, Mar 06, 2021 at 03:01:43PM -0500, Tom Lane wrote: > > > v4-0001 mostly teaches test.sh about specific changes that have to be > > > made to historic versions of the regress

Re: pg_upgrade test for binary compatibility of core data types

2021-07-16 Thread Jacob Champion
On Fri, 2021-04-30 at 13:33 -0500, Justin Pryzby wrote: > On Sat, Mar 06, 2021 at 03:01:43PM -0500, Tom Lane wrote: > > v4-0001 mostly teaches test.sh about specific changes that have to be > > made to historic versions of the regression database to allow them > > to be reloaded into current server

Re: Remove redundant strlen call in ReplicationSlotValidateName

2021-07-16 Thread Ranier Vilela
Em sex., 16 de jul. de 2021 às 12:37, Alvaro Herrera < alvhe...@alvh.no-ip.org> escreveu: > On 2021-Jul-16, David Rowley wrote: > > > On Fri, 16 Jul 2021 at 20:35, Japin Li wrote: > > > > When I fix a bug about ALTER SUBSCRIPTION ... SET (slot_name) [1], > Ranier Vilela > > > > finds that Replica

Re: Remove redundant strlen call in ReplicationSlotValidateName

2021-07-16 Thread Alvaro Herrera
On 2021-Jul-16, David Rowley wrote: > On Fri, 16 Jul 2021 at 20:35, Japin Li wrote: > > > When I fix a bug about ALTER SUBSCRIPTION ... SET (slot_name) [1], Ranier > > > Vilela > > > finds that ReplicationSlotValidateName() has redundant strlen() call, > > > Since it's > > > not related to that

Re: Early Sort/Group resjunk column elimination.

2021-07-16 Thread James Coleman
On Tue, Jul 13, 2021 at 10:19 AM Ronan Dunklau wrote: > > Hello, > > I would like to know if there is any interest in working to reduce the usage > and propagation of resjunk columns in the planner. > > I think this topic is worth investigating, because most of the time when we > request a sorted

Re: 回复: Why is XLOG_FPI_FOR_HINT always need backups?

2021-07-16 Thread Fujii Masao
On 2021/07/16 16:31, Kyotaro Horiguchi wrote: Sorry, I missed that the XLogReadBufferForRedo is expected to return BLK_RESTORED. And XLogReadBufferForRedo errors out when it tries to read nonexistent page without having an FPI (this happens for FSM pages). Rather than teaching XLogReadBuffer

Re: Skipping logical replication transactions on subscriber side

2021-07-16 Thread Masahiko Sawada
On Wed, Jul 14, 2021 at 5:14 PM Masahiko Sawada wrote: > > On Mon, Jul 12, 2021 at 8:52 PM Amit Kapila wrote: > > > > On Mon, Jul 12, 2021 at 11:13 AM Masahiko Sawada > > wrote: > > > > > > On Mon, Jul 12, 2021 at 1:15 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, Jul 12, 2021 at 9:37 A

Added documentation for cascade and restrict option of drop statistics

2021-07-16 Thread vignesh C
Hi, Cascade and restrict options are supported for drop statistics syntax: drop statistics stat1 cascade; drop statistics stat2 restrict; The documentation for this was missing, attached a patch which includes the documentation for these options. Regards, Vignesh From a7f1bf7eb8e1d693274016e9371

Re: Add option --drop-cascade for pg_dump/restore

2021-07-16 Thread Andrew Dunstan
On 7/16/21 9:40 AM, Tom Lane wrote: > vignesh C writes: >> On Tue, Jul 13, 2021 at 9:16 PM Wu Haotian wrote: >>> I may not understand it correctly, are you saying >>> pg_dump will include the cascade option only for plain format, or >>> pg_dump will enable the cascade option for plain by defaul

Re: Add option --drop-cascade for pg_dump/restore

2021-07-16 Thread Tom Lane
vignesh C writes: > On Tue, Jul 13, 2021 at 9:16 PM Wu Haotian wrote: >> I may not understand it correctly, are you saying >> pg_dump will include the cascade option only for plain format, or >> pg_dump will enable the cascade option for plain by default? > pg_dump support plain, custom, tar and

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-16 Thread James Coleman
On Thu, Jul 15, 2021 at 11:45 PM David Rowley wrote: > > On Fri, 16 Jul 2021 at 02:53, Ronan Dunklau wrote: > > Please find attached a v9 just moving the flag setting to ExecInitSort, and > > my > > apologies if I misunderstood your point. > > I took this and adjusted a few things and ended up w

Re: Add option --drop-cascade for pg_dump/restore

2021-07-16 Thread vignesh C
On Tue, Jul 13, 2021 at 9:16 PM Wu Haotian wrote: > > > 2) I felt pg_dump will include the cascade option for plain format and > > pg_restore will include the cascade option from pg_restore for other > > formats. If my understanding is correct, should we document this? > > I may not understand it

Re: proposal: enhancing plpgsql debug API - returns text value of variable content

2021-07-16 Thread Aleksander Alekseev
Hi Pavel, > I would like to print content of variables - and now, I have to go some > deeper than I would like. I need to separate between scalar, row, and > record variables. PLpgSQL has code for it - but it is private. > [...] The patch seems OK, but I wonder - would it be possible to write a t

Re: Corrected documentation of data type for the logical replication message formats.

2021-07-16 Thread vignesh C
On Fri, Jul 16, 2021 at 8:52 AM Peter Smith wrote: > > Hi Vignesh. > > FYI - Because the other patch [1] was pushed ahead of this one, I > think your patch now needs to be updated for the new message types > that were introduced. Thanks, I have made the changes for the same in the v5 patch attach

Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

2021-07-16 Thread Ranier Vilela
Em sex., 16 de jul. de 2021 às 09:05, Aleksander Alekseev < aleksan...@timescale.com> escreveu: > Hi Rainer, > > > Here are the two patches. > > As suggested, reclassified as refactoring only. > > Please don't change the status of the patch on CF application before > it was reviewed. It will only

Re: Signed vs Unsigned (take 2) (src/backend/storage/ipc/procarray.c)

2021-07-16 Thread Aleksander Alekseev
Hi Rainer, > Here are the two patches. > As suggested, reclassified as refactoring only. Please don't change the status of the patch on CF application before it was reviewed. It will only slow things down. Your patch seems to have some problems on FreeBSD. Please see http://commitfest.cputube.or

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-16 Thread Dean Rasheed
On Fri, 16 Jul 2021 at 10:26, Bharath Rupireddy wrote: > > Thanks. The v5 patch LGTM. OK, I'll push that in a while. > Comment on errorConflictingDefElem: > I think that the message in errorConflictingDefElem should say > <>. I'm not sure why it > wasn't done. Almost, all the cases where errorCo

Re: Remove redundant strlen call in ReplicationSlotValidateName

2021-07-16 Thread Ranier Vilela
Em sex., 16 de jul. de 2021 às 07:05, David Rowley escreveu: > On Fri, 16 Jul 2021 at 20:35, Japin Li wrote: > > > When I fix a bug about ALTER SUBSCRIPTION ... SET (slot_name) [1], > Ranier Vilela > > > finds that ReplicationSlotValidateName() has redundant strlen() call, > Since it's > > > not

Re: Remove redundant strlen call in ReplicationSlotValidateName

2021-07-16 Thread Japin Li
On Fri, 16 Jul 2021 at 18:05, David Rowley wrote: > On Fri, 16 Jul 2021 at 20:35, Japin Li wrote: >> > When I fix a bug about ALTER SUBSCRIPTION ... SET (slot_name) [1], Ranier >> > Vilela >> > finds that ReplicationSlotValidateName() has redundant strlen() call, >> > Since it's >> > not rela

Re: [PATCH] Use optimized single-datum tuplesort in ExecSort

2021-07-16 Thread Ranier Vilela
Em sex., 16 de jul. de 2021 às 00:45, David Rowley escreveu: > On Fri, 16 Jul 2021 at 02:53, Ronan Dunklau > wrote: > > Please find attached a v9 just moving the flag setting to ExecInitSort, > and my > > apologies if I misunderstood your point. > > I took this and adjusted a few things and ende

Re: Added schema level support for publication.

2021-07-16 Thread vignesh C
On Fri, Jul 16, 2021 at 9:25 AM Greg Nancarrow wrote: > > On Wed, Jul 14, 2021 at 8:17 PM vignesh C wrote: > > > > Thanks for your comments, the attached v11 patch fixes the issues. > > > > Thanks for your work on this. > > I have some minor review comments on the documentation: > > (1) wrong lin

Re: Remove redundant strlen call in ReplicationSlotValidateName

2021-07-16 Thread David Rowley
On Fri, 16 Jul 2021 at 20:35, Japin Li wrote: > > When I fix a bug about ALTER SUBSCRIPTION ... SET (slot_name) [1], Ranier > > Vilela > > finds that ReplicationSlotValidateName() has redundant strlen() call, Since > > it's > > not related to that problem, so I start a new thread to discuss it.

Re: speed up verifying UTF-8

2021-07-16 Thread John Naylor
On Fri, Jul 16, 2021 at 1:44 AM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > > Have you considered shift-based DFA for a portable implementation https://gist.github.com/pervognsen/218ea17743e1442e59bb60d29b1aa725 ? I did consider some kind of DFA a while back and it was too slow. --

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2021-07-16 Thread David Rowley
On Fri, 16 Jul 2021 at 18:04, David Rowley wrote: > What we maybe could consider instead would be to pick the first Aggref > then look for the most sorted derivative of that then tally up the > number of Aggrefs that can be sorted using those pathkeys, then repeat > that process for the remaining

Re: [PATCH] Hooks at XactCommand level

2021-07-16 Thread Gilles Darold
Le 14/07/2021 à 21:26, Tom Lane a écrit : Gilles Darold writes: I have renamed the patch and the title of this proposal registered in the commitfest "Xact/SubXact event callback at command start" to reflect the last changes that do not include new hooks anymore. Hmm, it doesn't seem like this

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-16 Thread Bharath Rupireddy
On Fri, Jul 16, 2021 at 1:32 PM Dean Rasheed wrote: > > On Fri, 16 Jul 2021 at 06:40, Bharath Rupireddy > wrote: > > > > 1) Duplicate option check for "FROM" option is unnecessary and will be > > a dead code. Because the syntaxer anyways would catch if FROM is > > specified more than once, someth

Re: [PATCH] psql: \dn+ to show size of each schema (and \dA+ for AMs)

2021-07-16 Thread Laurenz Albe
On Thu, 2021-07-15 at 20:16 -0500, Justin Pryzby wrote: > On Wed, Jul 14, 2021 at 07:42:33AM +0200, Laurenz Albe wrote: > > Besides, schemas are not physical, but logical containers. So I see a > > point in > > measuring the storage used in a certain tablespace, but not so much by all > > object

Re: row filtering for logical replication

2021-07-16 Thread Tomas Vondra
On 7/16/21 5:26 AM, Amit Kapila wrote: On Wed, Jul 14, 2021 at 4:30 PM Dilip Kumar wrote: On Wed, Jul 14, 2021 at 3:58 PM Tomas Vondra wrote: Is there some reasonable rule which of the old/new tuples (or both) to use for the WHERE condition? Or maybe it'd be handy to allow referencing OL

Re: Why ALTER SUBSCRIPTION ... SET (slot_name='none') requires subscription disabled?

2021-07-16 Thread Japin Li
On Fri, 16 Jul 2021 at 14:06, Amit Kapila wrote: > On Fri, Jul 9, 2021 at 8:20 AM Japin Li wrote: >> >> On Thu, 08 Jul 2021 at 18:17, Amit Kapila wrote: >> > On Thu, Jul 8, 2021 at 3:43 PM Japin Li wrote: >> >> Please consider review v3 patch. v3-0001 adds slot_name verification in >> parse_su

Re: Remove redundant strlen call in ReplicationSlotValidateName

2021-07-16 Thread Japin Li
On Fri, 16 Jul 2021 at 16:26, Japin Li wrote: > Hi, hackers > > When I fix a bug about ALTER SUBSCRIPTION ... SET (slot_name) [1], Ranier > Vilela > finds that ReplicationSlotValidateName() has redundant strlen() call, Since > it's > not related to that problem, so I start a new thread to disc

Re: data corruption hazard in reorderbuffer.c

2021-07-16 Thread Tomas Vondra
On 7/16/21 1:07 AM, Mark Dilger wrote: On Jul 15, 2021, at 3:32 PM, Tomas Vondra wrote: I think it's mostly futile to list all the possible issues this might have caused - if you skip arbitrary decoded changes, that can trigger pretty much any bug in reorder buffer. But those bugs can be

A (but copied many) typo of char-mapping tables

2021-07-16 Thread Kyotaro Horiguchi
While I looked into a .map file in src/backend/utils/mb/Unicode, I notice of a typo in it. > static const pg_mb_radix_tree euc_jp_from_unicode_tree = > { > .. > 0x, /* offset of table for 1-byte inputs */ > ... > 0x0040, /* offset of table for 2-byte inputs */ > ... > 0x02c3, /*

Re: CREATE COLLATION - check for duplicate options and error out if found one

2021-07-16 Thread Dean Rasheed
On Fri, 16 Jul 2021 at 06:40, Bharath Rupireddy wrote: > > 1) Duplicate option check for "FROM" option is unnecessary and will be > a dead code. Because the syntaxer anyways would catch if FROM is > specified more than once, something like CREATE COLLATION mycoll1 FROM > FROM "C";. Hmm, it is pos

Re: row filtering for logical replication

2021-07-16 Thread Greg Nancarrow
On Fri, Jul 16, 2021 at 3:50 PM Amit Kapila wrote: > > I am not so sure about different filters for old and new rows but it > makes sense to by default apply the filter to both old and new rows. > Then also provide a way for user to specify if the filter can be > specified to just old or new row.

Re: 回复: Why is XLOG_FPI_FOR_HINT always need backups?

2021-07-16 Thread Kyotaro Horiguchi
At Thu, 15 Jul 2021 22:50:08 +0900, Fujii Masao wrote in > On 2021/07/07 16:11, Kyotaro Horiguchi wrote: > > The doc of wal_log_hints says that "*even* for non-critical > > modifications of so-called hint bits", which seems to me implies it is > > following full_page_writes (and I think it is no

Re: refactoring basebackup.c

2021-07-16 Thread Dilip Kumar
On Mon, Jul 12, 2021 at 5:51 PM tushar wrote: > > On 7/8/21 9:26 PM, Robert Haas wrote: > > Here at last is a new version. > Please refer this scenario ,where backup target using > --server-compression is closing the server > unexpectedly if we don't provide -no-manifest option > > [tushar@localho