Re: Implementing Incremental View Maintenance

2021-04-25 Thread Yugo NAGATA
On Tue, 20 Apr 2021 09:51:34 +0900 Yugo NAGATA wrote: > On Mon, 19 Apr 2021 17:40:31 -0400 > Tom Lane wrote: > > > Andrew Dunstan writes: > > > This patch (v22c) just crashed for me with an assertion failure on > > > Fedora 31. Here's the stack trace: > > > > > #2  0x0094a54a in Excep

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-25 Thread David Rowley
On Mon, 26 Apr 2021 at 05:03, Yura Sokolov wrote: > If your test so sensitive to hash function speed, then I'd suggest > to try something even simpler: > > static inline uint32 > relfilenodebackend_hash(RelFileNodeBackend *rnode) > { > uint32 h = 0; > #define step(x) h ^= (uint32)

Re: logical replication empty transactions

2021-04-25 Thread Peter Smith
On Fri, Apr 23, 2021 at 3:46 PM Ajin Cherian wrote: > > > > On Mon, Apr 19, 2021 at 6:22 PM Peter Smith wrote: >> >> >> Here are a some review comments: >> >> -- >> >> 1. The patch v3 applied OK but with whitespace warnings >> >> [postgres@CentOS7-x64 oss_postgres_2PC]$ git apply >> ../patche

Re: Table refer leak in logical replication

2021-04-25 Thread Michael Paquier
On Fri, Apr 23, 2021 at 09:38:01PM +0900, Amit Langote wrote: > On Thu, Apr 22, 2021 at 1:45 PM Michael Paquier wrote: >> On the other hand, the tests for partitions have much more value IMO, >> but looking closely I think that we can do better: >> - Create triggers on more relations of the partit

Re: Parallel INSERT SELECT take 2

2021-04-25 Thread Amit Kapila
On Mon, Apr 26, 2021 at 7:00 AM houzj.f...@fujitsu.com wrote: > > > Instead, how about > > postgres retries the query upon detecting the error that came from a > > parallel > > unsafe function during execution, disable parallelism and run the query? I > > think > > this kind of retry query featu

RE: Truncate in synchronous logical replication failed

2021-04-25 Thread osumi.takami...@fujitsu.com
On Monday, April 26, 2021 1:50 PM Amit Kapila wrote: > On Fri, Apr 23, 2021 at 7:18 PM osumi.takami...@fujitsu.com > wrote: > > > > The latest patch looks good to me. I have made a minor modification and > added a commit message in the attached. Thank you for updating the patch. I think we need

Re: Asynchronous Append on postgres_fdw nodes.

2021-04-25 Thread Andrey V. Lepikhov
On 4/23/21 8:12 AM, Etsuro Fujita wrote: I have committed the patch. While studying the capabilities of AsyncAppend, I noticed an inconsistency with the cost model of the optimizer: async_capable = off: Append (cost=100.00..695.00 ...) -> Foreign Scan on f1 part_1 (c

Re: GISTSTATE is too large

2021-04-25 Thread Andrey Borodin
> 26 апр. 2021 г., в 03:20, Andres Freund написал(а): > > So the basic GISTSTATE is 14kB large. And all the information needed to > call support functions for one attribute is spaced so far appart that > it's guaranteed to be on different cachelines and to be very unlikely to > be prefetched b

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-04-25 Thread Amit Kapila
On Fri, Apr 23, 2021 at 8:03 PM osumi.takami...@fujitsu.com wrote: > > On Saturday, April 17, 2021 4:13 PM Amit Kapila > wrote: > > > I also don't find a test for this. It is introduced in 5dfd1e5a6696, > > > wrote by Simon Riggs, Marco Nenciarini and Peter Eisentraut. Maybe > > > they can exp

Re: TRUNCATE on foreign table

2021-04-25 Thread Bharath Rupireddy
On Fri, Apr 23, 2021 at 9:50 PM Fujii Masao wrote: > Thanks for the review! I fixed this. Thanks for the updated patches. In docs v4 patch, I think we can combine below two lines into a single line: + supported by the foreign data wrapper, see . Other than the above minor change, both pat

Re: comment typo - misnamed function

2021-04-25 Thread Amit Kapila
On Mon, Apr 26, 2021 at 7:59 AM Peter Smith wrote: > > PSA patch to fix a misnamed function in a comment. > > typo: "DecodePreare" --> "DecodePrepare" > Pushed. -- With Regards, Amit Kapila.

Re: Truncate in synchronous logical replication failed

2021-04-25 Thread Amit Kapila
On Fri, Apr 23, 2021 at 7:18 PM osumi.takami...@fujitsu.com wrote: > The latest patch looks good to me. I have made a minor modification and added a commit message in the attached. I would like to once again ask whether anybody else thinks we should backpatch this? Just a summary for anybody not

Re: Does rewriteTargetListIU still need to add UPDATE tlist entries?

2021-04-25 Thread Amit Langote
On Mon, Apr 26, 2021 at 9:40 AM Tom Lane wrote: > The comments for rewriteTargetListIU say (or said until earlier today) > > * 2. For an UPDATE on a trigger-updatable view, add tlist entries for any > * unassigned-to attributes, assigning them their old values. These will > * later get expande

Re: Can a child process detect postmaster death when in pg_usleep?

2021-04-25 Thread Bharath Rupireddy
On Tue, Apr 20, 2021 at 7:36 AM Bharath Rupireddy wrote: > > On Thu, Apr 15, 2021 at 11:48 AM Bharath Rupireddy > wrote: > > > We definitely have replaced a lot of sleeps with latch.c primitives > > > over the past few years, since we got WL_EXIT_ON_PM_DEATH and > > > condition variables. There

Re: Addition of authenticated ID to pg_stat_activity

2021-04-25 Thread Justin Pryzby
On Mon, Apr 26, 2021 at 11:34:16AM +0900, Michael Paquier wrote: > +++ b/doc/src/sgml/config.sgml > @@ -7596,6 +7596,24 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' > WITH csv; > > > > + xreflabel="track_activity_authn_size"> > + track_activity_authn_size > (in

Remove redundant variable pageSize in gistinitpage

2021-04-25 Thread Bharath Rupireddy
Hi, In gistinitpage, pageSize variable looks redundant, instead we could just pass BLCKSZ. This will be consistent with its peers BloomInitPage, brin_page_init and SpGistInitPage. Attaching a small patch. Thoughts? With Regards, Bharath Rupireddy. EnterpriseDB: http://www.enterprisedb.com From b7

Re: Replication slot stats misgivings

2021-04-25 Thread Amit Kapila
On Mon, Apr 26, 2021 at 8:01 AM Masahiko Sawada wrote: > > On Fri, Apr 23, 2021 at 6:15 PM Amit Kapila wrote: > > > > On Mon, Apr 19, 2021 at 4:28 PM vignesh C wrote: > > > > > > I have made the changes to update the replication statistics at > > > replication slot release. Please find the patch

Re: PageGetItemIdCareful - should we MAXALIGN sizeof(BTPageOpaqueData)?

2021-04-25 Thread Bharath Rupireddy
On Sat, Apr 24, 2021 at 4:11 AM Peter Geoghegan wrote: > > PageInit always max aligns this structure, when we > > initialize the btree page in _bt_pageini and in all other places we > > max align it before use. Since this is an error throwing path, I think > > we should max align it just to be on

Addition of authenticated ID to pg_stat_activity

2021-04-25 Thread Michael Paquier
Hi all, 9afffcb has added the concept of authenticated identity to the information provided in log_connections for audit purposes, with this data stored in each backend's port. One extra thing that can be really useful for monitoring is the capability to track this information directly in pg_stat

Re: Replication slot stats misgivings

2021-04-25 Thread Masahiko Sawada
On Fri, Apr 23, 2021 at 6:15 PM Amit Kapila wrote: > > On Mon, Apr 19, 2021 at 4:28 PM vignesh C wrote: > > > > I have made the changes to update the replication statistics at > > replication slot release. Please find the patch attached for the same. > > Thoughts? > > > > Thanks, the changes look

comment typo - misnamed function

2021-04-25 Thread Peter Smith
Hi, PSA patch to fix a misnamed function in a comment. typo: "DecodePreare" --> "DecodePrepare" -- Kind Regards, Peter Smith. Fujitsu Australia v1-0001-Fix-typo-misnamed-function-in-comment.patch Description: Binary data

Re: [PATCH] add concurrent_abort callback for output plugin

2021-04-25 Thread Peter Smith
Hi, While testing another WIP patch [1] a clashing GID problem was found, which gives us apply worker errors like: 2021-04-26 10:07:12.883 AEST [22055] ERROR: transaction identifier "pg_gid_16403_608" is already in use 2021-04-26 10:08:05.149 AEST [22124] ERROR: transaction identifier "pg_gid_1

RE: Parallel INSERT SELECT take 2

2021-04-25 Thread houzj.f...@fujitsu.com
> > Based on above, we plan to move forward with the apporache 2) (declarative > idea). > > IIUC, the declarative behaviour idea attributes parallel > safe/unsafe/restricted > tags to each table with default being the unsafe. Does it mean for a parallel > unsafe table, no parallel selects, insert

Re: wal stats questions

2021-04-25 Thread Masahiro Ikeda
On 2021/04/23 16:30, Fujii Masao wrote: > > > On 2021/04/23 10:25, Andres Freund wrote: >> Hi, >> >> On 2021-04-23 09:26:17 +0900, Masahiro Ikeda wrote: >>> On 2021/04/23 0:36, Andres Freund wrote: On Thu, Apr 22, 2021, at 06:42, Fujii Masao wrote: > On 2021/04/21 18:31, Masahiro Ikeda

Does rewriteTargetListIU still need to add UPDATE tlist entries?

2021-04-25 Thread Tom Lane
The comments for rewriteTargetListIU say (or said until earlier today) * 2. For an UPDATE on a trigger-updatable view, add tlist entries for any * unassigned-to attributes, assigning them their old values. These will * later get expanded to the output values of the view. (This is equivalent

Re: decoupling table and index vacuum

2021-04-25 Thread Masahiko Sawada
On Sat, Apr 24, 2021 at 12:22 AM Robert Haas wrote: > > On Fri, Apr 23, 2021 at 7:04 AM Masahiko Sawada wrote: > > I think we can divide the TID fork into 16MB or 32MB chunks like WAL > > segment files so that we can easily remove old chunks. Regarding the > > efficient search part, I think we ne

GISTSTATE is too large

2021-04-25 Thread Andres Freund
Hi, On twitter it was mentioned [1] that gist index builds spend a lot of time in FunctionCall3Coll. Which could be addressed to a good degree by not using FunctionCall3Coll() which needs to call InitFunctionCallInfoData() every time, but instead doing so once by including the FunctionCallInfo in

Re: Fix dropped object handling in pg_event_trigger_ddl_commands

2021-04-25 Thread Alvaro Herrera
On 2021-Apr-25, Sven Klemm wrote: > On Sun, Apr 18, 2021 at 2:12 PM Sven Klemm wrote: > > when creating an event trigger for ddl_command_end that calls > > pg_event_trigger_ddl_commands certain statements will cause the > > trigger to fail with a cache lookup error. The error happens on > > maste

Re: Allowing to create LEAKPROOF functions to non-superuser

2021-04-25 Thread Tom Lane
Stephen Frost writes: > * Andrey Borodin (x4...@yandex-team.ru) wrote: >> Customer was restoring pg_dump of on-premise ERP known as 1C (something like >> TurboTax) with this add-on [0] > Erm, it's very clearly not leakproof and will happily return information > about the value passed in during s

Re: Allowing to create LEAKPROOF functions to non-superuser

2021-04-25 Thread Stephen Frost
Greetings, * Noah Misch (n...@leadboat.com) wrote: > On Mon, Apr 19, 2021 at 05:38:43PM -0400, Stephen Frost wrote: > > > > > On Fri, Apr 16, 2021 at 3:57 AM Noah Misch wrote: > > > > >> Hence, I do find it reasonable to let pg_read_all_data be sufficient > > > > >> for > > > > >> setting LEAKPR

Re: Allowing to create LEAKPROOF functions to non-superuser

2021-04-25 Thread Stephen Frost
Greetings, * Andrey Borodin (x4...@yandex-team.ru) wrote: > > 20 апр. 2021 г., в 02:38, Stephen Frost написал(а): > > Here's what I'd ask Andrey- what's the actual use-case here? Are these > > cases where users are actually adding new functions which they believe > > are leakproof where those fu

Better sanity checking for message length words

2021-04-25 Thread Tom Lane
We had a report [1] of a case where a broken client application sent some garbage to the server, which then hung up because it misinterpreted the garbage as a very long message length, and was sitting waiting for data that would never arrive. There is already a sanity check on the message type byt

Re: compute_query_id and pg_stat_statements

2021-04-25 Thread Julien Rouhaud
On Sun, Apr 25, 2021 at 01:17:03PM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > On Sun, Apr 25, 2021 at 11:39:55AM -0400, Tom Lane wrote: > >> I agree repeated warnings would be bad news. I was wondering if we could > >> arrange a single warning at the time pg_stat_statements is preloaded

Re: compute_query_id and pg_stat_statements

2021-04-25 Thread Tom Lane
Julien Rouhaud writes: > On Sun, Apr 25, 2021 at 11:39:55AM -0400, Tom Lane wrote: >> I agree repeated warnings would be bad news. I was wondering if we could >> arrange a single warning at the time pg_stat_statements is preloaded into >> the postmaster. In this way, if you tried to use a config

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-25 Thread Yura Sokolov
David Rowley писал 2021-04-25 16:36: On Sun, 25 Apr 2021 at 18:48, Yura Sokolov wrote: If you read comments in SH_START_ITERATE, you'll see: * Search for the first empty element. As deletions during iterations are * supported, we want to start/end at an element that cannot be affected

Re: compute_query_id and pg_stat_statements

2021-04-25 Thread Julien Rouhaud
On Sun, Apr 25, 2021 at 11:39:55AM -0400, Tom Lane wrote: > > I agree repeated warnings would be bad news. I was wondering if we could > arrange a single warning at the time pg_stat_statements is preloaded into > the postmaster. In this way, if you tried to use a configuration file > that used t

Re: compute_query_id and pg_stat_statements

2021-04-25 Thread Tom Lane
Julien Rouhaud writes: > On Sat, Apr 24, 2021 at 01:43:51PM -0400, Tom Lane wrote: >> I haven't looked, but did we put anything into pg_stat_statements >> to make it easy to tell if you've messed up this setting? > You mean apart from from having pg_stat_statements' view/SRFs returning > nothing?

Re: [PATCH] Automatic HASH and LIST partition creation

2021-04-25 Thread Nitin Jadhav
I have reviewed the v4 patch. The patch does not get applied on the latest source. Kindly rebase. However I have found few comments. 1. > +-- must fail because of wrong configuration > +CREATE TABLE tbl_hash_fail (i int) PARTITION BY HASH (i) > +CONFIGURATION (values in (1, 2), (3, 4) default part

Re: Use simplehash.h instead of dynahash in SMgr

2021-04-25 Thread David Rowley
On Sun, 25 Apr 2021 at 18:48, Yura Sokolov wrote: > If you read comments in SH_START_ITERATE, you'll see: > >* Search for the first empty element. As deletions during iterations > are >* supported, we want to start/end at an element that cannot be > affected >* by elements being shifte

Re: Allowing to create LEAKPROOF functions to non-superuser

2021-04-25 Thread Andrey Borodin
> 20 апр. 2021 г., в 02:38, Stephen Frost написал(а): > > Here's what I'd ask Andrey- what's the actual use-case here? Are these > cases where users are actually adding new functions which they believe > are leakproof where those functions don't require superuser already to > be created? Cle

Re: Allowing to create LEAKPROOF functions to non-superuser

2021-04-25 Thread Noah Misch
On Mon, Apr 19, 2021 at 05:38:43PM -0400, Stephen Frost wrote: > > > > On Fri, Apr 16, 2021 at 3:57 AM Noah Misch wrote: > > > >> Hence, I do find it reasonable to let pg_read_all_data be sufficient > > > >> for > > > >> setting LEAKPROOF. > I'm not really sure that attaching it to pg_read_all_d

Re: Fix dropped object handling in pg_event_trigger_ddl_commands

2021-04-25 Thread Sven Klemm
On Sun, Apr 18, 2021 at 2:12 PM Sven Klemm wrote: > when creating an event trigger for ddl_command_end that calls > pg_event_trigger_ddl_commands certain statements will cause the > trigger to fail with a cache lookup error. The error happens on > master, 13 and 12 I didnt test any previous versio

Re: compute_query_id and pg_stat_statements

2021-04-25 Thread Julien Rouhaud
On Sat, Apr 24, 2021 at 01:43:51PM -0400, Tom Lane wrote: > > I haven't looked, but did we put anything into pg_stat_statements > to make it easy to tell if you've messed up this setting? You mean apart from from having pg_stat_statements' view/SRFs returning nothing? I think it's a reasonable u

Some oversights in query_id calculation

2021-04-25 Thread Julien Rouhaud
Hi, While doing some sanity checks on the regression tests, I found some queries that are semantically different but end up with identical query_id. Two are an old issues: - the "ONLY" in FROM [ONLY] isn't hashed - the agglevelsup field in GROUPING isn't hashed Another one was introduced in pg1