Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-10 Thread Fabien COELHO
Hello Tom, The buildfarm is still complaining about the synopsis being too wide for PDF format. I think what we ought to do is give up on using a for log lines at all, and instead convert the documentation into a tabular list of fields. Proposal attached, which also fixes a couple of outrigh

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-10 Thread Etsuro Fujita
On Sat, Apr 9, 2022 at 1:58 AM Etsuro Fujita wrote: > On Fri, Apr 8, 2022 at 9:43 PM Justin Pryzby wrote: > > This patch seems to be causing the planner to crash. > > Here's a query reduced from sqlsmith. > > > > | explain SELECT 1 FROM information_schema.constraint_column_usage WHERE 1 > > <= p

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-10 Thread Etsuro Fujita
Hi, On Sat, Apr 9, 2022 at 1:24 AM Zhihong Yu wrote: > On Fri, Apr 8, 2022 at 5:43 AM Justin Pryzby wrote: >> This patch seems to be causing the planner to crash. >> Here's a query reduced from sqlsmith. >> >> | explain SELECT 1 FROM information_schema.constraint_column_usage WHERE 1 >> <= pg_t

Re: Can we automatically add elapsed times to tap test log?

2022-04-10 Thread Andrew Dunstan
On 4/8/22 09:51, Andrew Dunstan wrote: > On 4/7/22 19:55, Andrew Dunstan wrote: >> On 4/7/22 17:58, Andres Freund wrote: >>> Hi, >>> >>> On 2022-04-07 17:45:09 -0400, Tom Lane wrote: Andres Freund writes: > On 2022-04-07 17:21:09 -0400, Tom Lane wrote: >> I too think that the elapse

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-10 Thread Zhihong Yu
On Sun, Apr 10, 2022 at 3:42 AM Etsuro Fujita wrote: > On Sat, Apr 9, 2022 at 1:58 AM Etsuro Fujita > wrote: > > On Fri, Apr 8, 2022 at 9:43 PM Justin Pryzby > wrote: > > > This patch seems to be causing the planner to crash. > > > Here's a query reduced from sqlsmith. > > > > > > | explain SEL

Re: Can we automatically add elapsed times to tap test log?

2022-04-10 Thread Andres Freund
On 2022-04-10 09:23:16 -0400, Andrew Dunstan wrote: > In the absence of further comment I have pushed this. Thanks!

Re: suboverflowed subtransactions concurrency performance optimize

2022-04-10 Thread Simon Riggs
On Thu, 7 Apr 2022 at 00:36, Michael Paquier wrote: > > On Mon, Mar 07, 2022 at 10:17:41PM +0800, Julien Rouhaud wrote: > > On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote: > >> The patch was posted because TransactionLogFetch() has a cache, yet > >> SubTransGetTopmostTransaction() doe

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-10 Thread Tom Lane
Fabien COELHO writes: > While looking at the html outpout, the "pgbench" command line just below > wraps strangely: >pgbench --aggregate-interval=10 --time=20 --client=10 --log --rate=1000 >--latency-limit=10 --failures-detailed --max-tries=10 test > ISTM that there should be no nl in t

GSoC: pgBackRest port to Windows

2022-04-10 Thread Samuel Bassaly
Hello everyone, My name is Samuel Bassaly, and I would like to submit my proposal for this year's GSoC. Kindly find my proposal under the following link: https://docs.google.com/document/d/1cEGLLJaRmb5hkpt7GayIJA4oCllYtchOU2p0FQpLZ3U/edit?usp=sharing Your feedback is highly appreciated. Thank yo

Re: Schema variables - new implementation for Postgres 15+1

2022-04-10 Thread Justin Pryzby
On Sun, Apr 10, 2022 at 08:30:39PM +0200, Pavel Stehule wrote: > I am sending fresh rebased patch + separation to more patches. This split > is initial, and can be changed later The 0001 patch requires this, but it's not included until 0003. src/include/commands/session_variable.h Each patch shou

Re: A qsort template

2022-04-10 Thread Thomas Munro
Hi, David Rowley privately reported a performance regression when sorting single ints with a lot of duplicates, in a case that previously hit qsort_ssup() but now hits qsort_tuple_int32() and then has to call the tiebreaker comparator. Note that this comes up only for sorts in a query, not for eg

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Peter Geoghegan
On Fri, Apr 8, 2022 at 9:55 AM Matthias van de Meent wrote: > Here's generation 2 of this effort. Instead of proceeding to trying to > shoehorn all types of btree keys into one common code path, this > patchset acknowledges that there exist different shapes of keys that > each have a different bes

Re: A qsort template

2022-04-10 Thread Peter Geoghegan
On Sun, Apr 10, 2022 at 2:44 PM Thomas Munro wrote: > David Rowley privately reported a performance regression when sorting > single ints with a lot of duplicates, in a case that previously hit > qsort_ssup() but now hits qsort_tuple_int32() and then has to call the > tiebreaker comparator. That'

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Peter Geoghegan
On Sun, Apr 10, 2022 at 2:44 PM Peter Geoghegan wrote: > Can you post a version of this that compiles? I forgot to add: the patch also bitrot due to recent commit dbafe127. I didn't get stuck at this point (this is minor bitrot), but no reason not to rebase. -- Peter Geoghegan

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Matthias van de Meent
On Sun, 10 Apr 2022 at 23:45, Peter Geoghegan wrote: > > On Fri, Apr 8, 2022 at 9:55 AM Matthias van de Meent > wrote: > > Here's generation 2 of this effort. Instead of proceeding to trying to > > shoehorn all types of btree keys into one common code path, this > > patchset acknowledges that the

Re: make MaxBackends available in _PG_init

2022-04-10 Thread Michael Paquier
On Sat, Apr 09, 2022 at 09:24:42PM +0800, Julien Rouhaud wrote: > Yeah I would prefer this approach too, although it couldn't prevent extension > from directly modifying the underlying variables so I don't know how effective > it would be. > > On the bright side, I see that citus is using SetConfi

Re: A qsort template

2022-04-10 Thread David Rowley
On Mon, 11 Apr 2022 at 09:44, Thomas Munro wrote: > David Rowley privately reported a performance regression when sorting > single ints with a lot of duplicates, in a case that previously hit > qsort_ssup() but now hits qsort_tuple_int32() and then has to call the > tiebreaker comparator. Note th

Re: REINDEX blocks virtually any queries but some prepared queries.

2022-04-10 Thread Michael Paquier
On Fri, Apr 08, 2022 at 04:23:48PM +0200, Frédéric Yhuel wrote: > Thank you Michael. And done as of 8ac700a. -- Michael signature.asc Description: PGP signature

Re: Commitfest Closed

2022-04-10 Thread Michael Paquier
On Fri, Apr 08, 2022 at 08:48:57AM -0400, Greg Stark wrote: > Thanks to all the reviewers and committers who put a lot of work in, > especially in the last two weeks. I especially want to thank Andres > who showed me how to use the cfbot to check on patch statuses and did > a lot of work doing that

Re: Improving btree performance through specializing by key shape, take 2

2022-04-10 Thread Peter Geoghegan
On Sun, Apr 10, 2022 at 4:08 PM Matthias van de Meent wrote: > I'll send an updated patchset soon (I'm planning on moving around when > what is changed/added); but before that the attached incremental patch > should help. FYI, the patchset has been tested on commit 05023a23, and > compiles (with u

typos

2022-04-10 Thread Justin Pryzby
In docs and comments. Mostly for v15. Maybe Fabien will want to comment on the pgbench one. >From b1fb6d9bae1b0d69a30d59e65ed4f5a10f4141da Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Tue, 25 Jan 2022 10:47:55 -0600 Subject: [PATCH 01/19] doc: database SYSTEM since aa01051418f10afbdfa781b8

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-10 Thread Justin Pryzby
On Sun, Apr 10, 2022 at 07:43:48PM +0900, Etsuro Fujita wrote: > On Sat, Apr 9, 2022 at 1:58 AM Etsuro Fujita wrote: > > On Fri, Apr 8, 2022 at 9:43 PM Justin Pryzby wrote: > > > This patch seems to be causing the planner to crash. > > > Here's a query reduced from sqlsmith. > > > > > > | explain

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-10 Thread Zhihong Yu
On Sun, Apr 10, 2022 at 7:41 PM Justin Pryzby wrote: > On Sun, Apr 10, 2022 at 07:43:48PM +0900, Etsuro Fujita wrote: > > On Sat, Apr 9, 2022 at 1:58 AM Etsuro Fujita > wrote: > > > On Fri, Apr 8, 2022 at 9:43 PM Justin Pryzby > wrote: > > > > This patch seems to be causing the planner to crash

Re: generic plans and "initial" pruning

2022-04-10 Thread Amit Langote
On Fri, Apr 8, 2022 at 8:45 PM Amit Langote wrote: > Most looked fine changes to me except a couple of typos, so I've > adopted those into the attached new version, even though I know it's > too late to try to apply it. > > + * XXX is it worth doing a bms_copy() on glob->minLockRelids if > + * glo

Re: PG DOCS - logical replication filtering

2022-04-10 Thread Amit Kapila
On Fri, Apr 8, 2022 at 11:42 AM Peter Smith wrote: > > On Wed, Apr 6, 2022 at 8:58 PM Amit Kapila wrote: > > > > > 3. > > + > > +Whenever an UPDATE is processed, the row filter > > +expression is evaluated for both the old and new row (i.e. before > > +and after the data is updated

Re: generic plans and "initial" pruning

2022-04-10 Thread Zhihong Yu
On Sun, Apr 10, 2022 at 8:05 PM Amit Langote wrote: > On Fri, Apr 8, 2022 at 8:45 PM Amit Langote > wrote: > > Most looked fine changes to me except a couple of typos, so I've > > adopted those into the attached new version, even though I know it's > > too late to try to apply it. > > > > + * XX

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-04-10 Thread Peter Geoghegan
On Thu, Mar 31, 2022 at 4:57 PM Michail Nikolaev wrote: > I remember you had an idea about using the LP_REDIRECT bit in btree > indexes as some kind of “recently dead” flag (1). > Is this idea still in progress? Maybe an additional bit could provide > a space for a better solution. I think that t

Re: MERGE bug report

2022-04-10 Thread Richard Guo
On Sat, Apr 9, 2022 at 5:26 AM Alvaro Herrera wrote: > On 2022-Apr-06, Richard Guo wrote: > > > That's right. The varattno is set to zero for whole-row Var. And in this > > case these whole-row Vars are not included in the targetlist. > > > > Attached is an attempt for the fix. > > Wow, this is v

Re: API stability

2022-04-10 Thread Kyotaro Horiguchi
(a bit off-topic) I'm not sure where I am.. At Wed, 06 Apr 2022 10:36:30 +0900 (JST), Kyotaro Horiguchi wrote in me> > this if nobody else would like to do it, but let me ask whether me> > Kyotaro Horiguchi would like to propose a patch, since the original me> > patch did, and/or whether you w

Re: typos

2022-04-10 Thread David Rowley
( Added Joe and Robert for 0011 ) On Mon, 11 Apr 2022 at 14:03, Justin Pryzby wrote: > In docs and comments. Mostly for v15. 0001: Should this not use PostgreSQL? (new to master) 0002: The patch looks good. (new to v12) 0003: The patch looks good. (new to master) 0004: The patch looks go

Re: Support logical replication of DDLs

2022-04-10 Thread Amit Kapila
On Thu, Apr 7, 2022 at 3:46 PM Amit Kapila wrote: > > On Wed, Mar 23, 2022 at 10:39 AM Japin Li wrote: > > 2. For DDL replication, do we need to wait for a consistent point of > snapshot? For DMLs, that point is a convenient point to initialize > replication from, which is why we export a snapsho

Re: Support logical replication of DDLs

2022-04-10 Thread Amit Kapila
On Fri, Apr 8, 2022 at 5:04 PM Alvaro Herrera wrote: > > On 2022-Apr-08, Amit Kapila wrote: > > > > For runtime conditions, one of the things you have mentioned in that > > thread is to add schema name in the statement at the required places > > which this patch deals with in a different way by ex

Outdated copyright year in parse_jsontable.c

2022-04-10 Thread Julien Rouhaud
Hi, I just noticed that parse_jsontable.c was added with a wrong copyright year, trivial patch attached. While at it I also see that a lot of Spanish translation files seems to be stuck in 2019, but those were fixed in the pgtranslation repo last June so they will get synced eventually. >From ffd

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-04-10 Thread Kyotaro Horiguchi
At Sat, 9 Apr 2022 18:03:01 +0530, Bharath Rupireddy wrote in > On Tue, Jan 4, 2022 at 1:40 AM SATYANARAYANA NARLAPURAM > wrote: > > > > On Sun, Jan 2, 2022 at 11:56 PM Michael Paquier wrote: > >> Are you referring to the pre-padding when creating a new partial > >> segment, aka when we write

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-04-10 Thread Kyotaro Horiguchi
Sorry for the terrible typos.. At Sat, 9 Apr 2022 18:03:01 +0530, Bharath Rupireddy wrote in > On Tue, Jan 4, 2022 at 1:40 AM SATYANARAYANA NARLAPURAM > wrote: > > > > On Sun, Jan 2, 2022 at 11:56 PM Michael Paquier wrote: > >> Are you referring to the pre-padding when creating a new partial

Re: Practical Timing Side Channel Attacks on Memory Compression

2022-04-10 Thread Filip Janus
Thanks for all of your opinions. I have almost the same feeling. The best layer for mitigation should be probably a user application. There can be arranged the correct data layout in the database, set up access limit for the app, and many other mitigation mechanisms. -Filip- st 6. 4. 2022 v

RE: Logical replication timeout problem

2022-04-10 Thread wangw.f...@fujitsu.com
On Wed, Apr 7, 2022 at 1:34 PM Amit Kapila wrote: > On Wed, Apr 6, 2022 at 6:30 PM wangw.f...@fujitsu.com > wrote: > > > > On Wed, Apr 6, 2022 at 1:58 AM Amit Kapila wrote: > > On Wed, Apr 6, 2022 at 4:32 AM Amit Kapila wrote: > > > Also, let's try to evaluate how it impacts lag functionality

Re: pgsql: Add TAP test for archive_cleanup_command and recovery_end_comman

2022-04-10 Thread Thomas Munro
On Sat, Apr 9, 2022 at 12:59 PM Andres Freund wrote: > On 2022-04-08 17:55:51 -0400, Tom Lane wrote: > > I tried adjusting the patch so it does guarantee that (as attached), > > and in two out of two tries it got past the archive_cleanup_command > > failure but then hung up waiting for standby2 to

Fixes for compression options of pg_receivewal and refactoring of backup_compression.{c,h}

2022-04-10 Thread Michael Paquier
Hi all, (Added Robert and Georgios in CC:) Since ba5 and the introduction of LZ4, I have reworked the way compression is controlled for pg_receivewal, with two options: - --compress-method, settable to "gzip", "none" or "lz4". - --compress, to pass down a compression level, where the allowed r

Re: Outdated copyright year in parse_jsontable.c

2022-04-10 Thread Michael Paquier
On Mon, Apr 11, 2022 at 02:08:38PM +0800, Julien Rouhaud wrote: > I just noticed that parse_jsontable.c was added with a wrong copyright year, > trivial patch attached. Thanks, I'll go fix that in a bit. I am spotting four more, as of src/backend/replication/basebackup_*.c that point to 2020. --