Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread Peter Eisentraut
On 24.03.21 18:58, John Naylor wrote: > As a potential follow-up, should we perhaps add named arguments?  That > might make the invocations easier to read, depending on taste. I think it's quite possible some users will prefer that. All we need is to add something like proargnames => '{bin_

Re: pg_amcheck contrib application

2021-03-24 Thread Robert Haas
Mark, Here's a quick and very dirty sketch of what I think perhaps this logic could look like. This is pretty much untested and it might be buggy, but at least you can see whether we're thinking at all in the same direction. -- Robert Haas EDB: http://www.enterprisedb.com very-rough-visibility

Re: MultiXact\SLRU buffers configuration

2021-03-24 Thread Thomas Munro
Hi Andrey, On Sat, Mar 13, 2021 at 1:44 AM Andrey Borodin wrote: > [v10] +intmultixact_offsets_slru_buffers = 8; +intmultixact_members_slru_buffers = 16; +intsubtrans_slru_buffers = 32; +intnotify_slru_buffers = 8; +intserial_slru_buffe

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-24 Thread Michael Paquier
On Wed, Mar 24, 2021 at 10:50:50AM -0400, Robert Haas wrote: > On Wed, Mar 24, 2021 at 5:56 AM Christoph Berg wrote: >> Maybe creating the tablespace directory from within the testsuite >> would suffice? >> >> CREATE TABLE foo (t text); >> COPY foo FROM PROGRAM 'mkdir @testtablespace@'; >> CREATE

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Daniel Gustafsson
> On 24 Mar 2021, at 04:54, Michael Paquier wrote: > > On Tue, Mar 23, 2021 at 12:38:50AM +0100, Daniel Gustafsson wrote: >> Thanks again for reviewing, another version which addresses the remaining >> issues will be posted soon but I wanted to get this out to give further >> reviews >> somethin

Re: Minimal logical decoding on standbys

2021-03-24 Thread Fabrízio de Royes Mello
On Wed, Mar 24, 2021 at 3:57 AM Drouvot, Bertrand wrote: > > Thanks for pointing out, fixed in v14 attached. > Thanks... now everything is working as expected... changed the status to Ready for Commiter: https://commitfest.postgresql.org/32/2968/ Regards, -- Fabrízio de Royes Mello Postg

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-24 Thread Daniel Gustafsson
Attached is a v2 which addresses the comments raised on the main NSS thread, as well as introduces named parameters for the server cert function to make the test code easier to read. -- Daniel Gustafsson https://vmware.com/ v2-0001-Refactor-SSL-testharness-for-multiple-library.pat

wal stats questions

2021-03-24 Thread Andres Freund
Hi, I got a few questions about the wal stats while working on the shmem stats patch: 1) What is the motivation to have both prevWalUsage and pgWalUsage, instead of just accumulating the stats since the last submission? There doesn't seem to be any comment explaining it? Computing differ

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-24 Thread Alvaro Herrera
On 2021-Mar-25, Daniel Gustafsson wrote: > Attached is a v2 which addresses the comments raised on the main NSS thread, > as > well as introduces named parameters for the server cert function to make the > test code easier to read. I don't like this patch. I think your SSL::Server::OpenSSL and

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-24 Thread Daniel Gustafsson
> On 25 Mar 2021, at 00:26, Alvaro Herrera wrote: > > On 2021-Mar-25, Daniel Gustafsson wrote: > >> Attached is a v2 which addresses the comments raised on the main NSS thread, >> as >> well as introduces named parameters for the server cert function to make the >> test code easier to read. >

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Jacob Champion
On Wed, 2021-03-24 at 14:10 -0400, Stephen Frost wrote: > * Jacob Champion (pchamp...@vmware.com) wrote: > > I could see this being a problem if two client certificate nicknames > > collide across multiple in-use databases, maybe? > > Right, in such a case either cert might get returned and it's p

Re: popcount

2021-03-24 Thread David Fetter
On Tue, Mar 23, 2021 at 10:51:08AM +0100, Peter Eisentraut wrote: > On 21.03.21 02:31, David Fetter wrote: > > > I have now read the entire internet on what a suitable name for this > > > function could be. I think the emerging winner is BIT_COUNT(), which > > > already exists in MySQL, and also i

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Justin Pryzby
On Thu, Mar 25, 2021 at 01:05:37AM +0100, Tomas Vondra wrote: > here's an updated patch. 0001 should address most of the today's review > items regarding comments etc. This is still an issue: postgres=# CREATE STATISTICS xt ON a FROM t JOIN t ON true; ERROR: schema "i" does not exist -- Justin

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-24 Thread Masahiro Ikeda
On 2021/03/24 18:36, Fujii Masao wrote: > > > On 2021/03/24 3:51, Andres Freund wrote: >> Hi, >> >> On 2021-03-23 15:50:46 +0900, Fujii Masao wrote: >>> This fact makes me wonder that if we collect the statistics about WAL >>> writing >>> from walreceiver as we discussed in other thread, the

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-24 Thread Masahiro Ikeda
On 2021/03/24 18:36, Fujii Masao wrote: > On 2021/03/23 15:50, Fujii Masao wrote: >> + * The statistics collector is started by the postmaster as soon as the >> + * startup subprocess finishes. >> >> This comment needs to be updated? Because the stats collector can >> be invoked when the startup pr

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-24 Thread Andres Freund
Hi, On 2021-03-24 18:36:14 +0900, Fujii Masao wrote: > Barring any objection, I'm thinking to commit this patch. To which branches? I am *strongly* opposed to backpatching this. > /* > - * Simple signal handler for exiting quickly as if due to a crash. > + * Simple signal handler for processe

Re: MultiXact\SLRU buffers configuration

2021-03-24 Thread Thomas Munro
On Thu, Mar 25, 2021 at 10:31 AM Thomas Munro wrote: > We already know that increasing the number of CLOG buffers above the > current number hurts as the linear search begins to dominate > (according to the commit message for 5364b357), and it doesn't seem > great to ship a new feature that melts

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-24 Thread Fujii Masao
On 2021/03/24 22:06, James Coleman wrote: That looks good to me. Thanks for working on this. Thanks! I pushed the patch. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: wal stats questions

2021-03-24 Thread Masahiro Ikeda
On 2021/03/25 8:22, Andres Freund wrote: > Hi, > > I got a few questions about the wal stats while working on the shmem > stats patch: Thanks for your reviews. > 1) What is the motivation to have both prevWalUsage and pgWalUsage, >instead of just accumulating the stats since the last sub

Re: making update/delete of inheritance trees scale better

2021-03-24 Thread Amit Langote
On Thu, Mar 25, 2021 at 4:22 AM Tom Lane wrote: > I wrote: > > Yeah, it's on my to-do list for this CF, but I expect it's going to > > take some concentrated study and other things keep intruding :-(. > > I've started to look at this seriously, Thanks a lot. > and I wanted to make a note > about

Proposal for col LIKE $1 with generic Plan

2021-03-24 Thread Andy Fan
Thanks to the get_index_clause_from_support, we can use index for WHERE a like 'abc%' case. However this currently only works on custom plan. Think about the case where the planning takes lots of time, custom plan will not give a good result. so I want to see if we should support this for a generic

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-24 Thread Julien Rouhaud
On Wed, Mar 24, 2021 at 01:02:00PM -0300, Alvaro Herrera wrote: > On 2021-Mar-24, Julien Rouhaud wrote: > > > From e08c9d5fc86ba722844d97000798de868890aba3 Mon Sep 17 00:00:00 2001 > > From: Bruce Momjian > > Date: Mon, 22 Mar 2021 17:43:23 -0400 > > Subject: [PATCH v20 2/3] Expose queryid in pg_

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-24 Thread Peter Smith
On Wed, Mar 24, 2021 at 11:31 PM Amit Kapila wrote: > > On Tue, Mar 23, 2021 at 3:31 PM Peter Smith wrote: > > > > On Tue, Mar 23, 2021 at 10:44 AM Peter Smith wrote: > > > > > > PSA patches to fix those. > > > > > > > Hi Amit. > > > > PSA a patch to allow the ALTER SUBSCRIPTION ... REFRESH PUBL

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-24 Thread Masahiro Ikeda
On 2021/03/23 16:10, Fujii Masao wrote: > > > On 2021/03/22 20:25, ikedamsh wrote: >> Agreed. Users can know whether the stats is for walreceiver or not. The >> pg_stat_wal view in standby server shows for the walreceiver, and in primary >> server it shows for the others. So, I updated the doc

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-24 Thread Noah Misch
On Thu, Mar 25, 2021 at 07:44:02AM +0900, Michael Paquier wrote: > On Wed, Mar 24, 2021 at 10:50:50AM -0400, Robert Haas wrote: > > On Wed, Mar 24, 2021 at 5:56 AM Christoph Berg wrote: > >> Maybe creating the tablespace directory from within the testsuite > >> would suffice? > >> > >> CREATE TABL

Re: multi-install PostgresNode

2021-03-24 Thread Michael Paquier
On Wed, Mar 24, 2021 at 03:33:51PM -0300, Alvaro Herrera wrote: > On 2021-Mar-24, Andrew Dunstan wrote: >> +# The install path set in get_new_node needs to be a directory containing >> +# bin and lib subdirectories as in a standard PostgreSQL installation, so >> this >> +# can't be used with insta

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-24 Thread Amul Sul
On Wed, Mar 24, 2021 at 8:09 PM Tom Lane wrote: > > Amul Sul writes: > > On Tue, Mar 23, 2021 at 8:59 PM Tom Lane wrote: > >> On closer inspection, I believe the true culprit is c6b92041d, > >> which did this: > >> - heap_sync(state->rs_new_rel); > >> + smgrimmedsync(

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Tomas Vondra
On 3/25/21 1:05 AM, Tomas Vondra wrote: > ... > > 0002 is an attempt to fix an issue I noticed today - we need to handle > type changes. Until now we did not have problems with that, because we > only had attnums - so we just reset the statistics (with the exception > of functional dependencies, on

Re: wal stats questions

2021-03-24 Thread Andres Freund
Hi, On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: > On 2021/03/25 8:22, Andres Freund wrote: > > 1) What is the motivation to have both prevWalUsage and pgWalUsage, > >instead of just accumulating the stats since the last submission? > >There doesn't seem to be any comment explaining

Re: Tying an object's ownership to datdba

2021-03-24 Thread Noah Misch
On Wed, Mar 24, 2021 at 11:57:37AM -0400, John Naylor wrote: > On Mon, Dec 28, 2020 at 12:32 AM Noah Misch wrote: > > [v2] > > Hi Noah, > > In the refactoring patch, there is a lingering comment reference to > roles_has_privs_of(). Aside from that, it looks good to me. A possible thing > to co

Re: shared-memory based stats collector

2021-03-24 Thread Kyotaro Horiguchi
Thank you for the the lot of help! At Mon, 22 Mar 2021 16:17:37 -0700, Andres Freund wrote in > Thanks! That change shouldn't be necessary on my branch - I did > something to fix this kind of problem too. I decided that there's no > point in doing hash table lookups for the database: It's not g

RE: Global snapshots

2021-03-24 Thread tsunakawa.ta...@fujitsu.com
From: Andrey V. Lepikhov > Current state of the patch set rebased on master, 5aed6a1fc2. > > It is development version. Here some problems with visibility still detected > in > two tests: > 1. CSN Snapshot module - TAP test on time skew. > 2. Clock SI implementation - TAP test on emulation of ba

Re: In-placre persistance change of a relation

2021-03-24 Thread Kyotaro Horiguchi
(I'm not sure when the subject was broken..) At Thu, 14 Jan 2021 17:32:17 +0900 (JST), Kyotaro Horiguchi wrote in > Commit bea449c635 conflicts with this on the change of the definition > of DropRelFileNodeBuffers. The change simplified this patch by a bit:p In this version, I got rid of the "

Re: In-placre persistance change of a relation

2021-03-24 Thread Kyotaro Horiguchi
At Thu, 25 Mar 2021 14:08:05 +0900 (JST), Kyotaro Horiguchi wrote in > (I'm not sure when the subject was broken..) > > At Thu, 14 Jan 2021 17:32:17 +0900 (JST), Kyotaro Horiguchi > wrote in > > Commit bea449c635 conflicts with this on the change of the definition > > of DropRelFileNodeBuffe

Decoding speculative insert with toast leaks memory

2021-03-24 Thread Ashutosh Bapat
Hi All, We saw OOM in a system where WAL sender consumed Gigabttes of memory which was never released. Upon investigation, we found out that there were many ReorderBufferToastHash memory contexts linked to ReorderBuffer context, together consuming gigs of memory. They were running INSERT ... ON CON

Re: Proposal: Save user's original authenticated identity for logging

2021-03-24 Thread Michael Paquier
On Wed, Mar 24, 2021 at 04:45:35PM +, Jacob Champion wrote: > With low-coverage test suites, I think it's useful to allow as little > strange behavior as possible -- in this case, printing authorization > before authentication could signal a serious bug -- but I don't feel > too strongly about

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-24 Thread Tom Lane
Amul Sul writes: > On Wed, Mar 24, 2021 at 8:09 PM Tom Lane wrote: >> static inline struct SMgrRelationData * >> RelationGetSmgr(Relation rel) >> { >> if (unlikely(rel->rd_smgr == NULL)) >> RelationOpenSmgr(rel); >> return rel->rd_smgr; >> } > A quick question: Can't it be a macr

Re: Replication slot stats misgivings

2021-03-24 Thread Masahiko Sawada
On Wed, Mar 24, 2021 at 7:06 PM Amit Kapila wrote: > > On Tue, Mar 23, 2021 at 10:54 PM Andres Freund wrote: > > > > On 2021-03-23 23:37:14 +0900, Masahiko Sawada wrote: > > > > > > > Maybe we can compare the slot name in the > > > > > received message to the name in the element of replSlotStats.

Re: psql lacking clearerr()

2021-03-24 Thread Kyotaro Horiguchi
At Wed, 24 Mar 2021 11:11:41 -0300, Alvaro Herrera wrote in > psql seems to never call clearerr() on its output file. So if it gets > an error while printing a result, it'll show > > could not print result table: Success > > after each and every result, even though the output file isn't in er

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-24 Thread Kyotaro Horiguchi
Sorry for the bug. At Thu, 25 Mar 2021 01:50:29 -0400, Tom Lane wrote in > Amul Sul writes: > > On Wed, Mar 24, 2021 at 8:09 PM Tom Lane wrote: > >> static inline struct SMgrRelationData * > >> RelationGetSmgr(Relation rel) > >> { > >> if (unlikely(rel->rd_smgr == NULL)) > >> Relat

<    1   2