Re: [PATCH] pg_permissions

2021-03-25 Thread Joel Jacobson
On Thu, Mar 25, 2021, at 17:51, Tom Lane wrote: > "Joel Jacobson" mailto:joel%40compiler.org>> writes: > > On Thu, Mar 25, 2021, at 16:16, Alvaro Herrera wrote: > >> Ah, of course -- the only way to obtain the acl columns is by going > >> through the catalogs individually, so it won't be possible.

Re: MultiXact\SLRU buffers configuration

2021-03-25 Thread Andrey Borodin
Hi Thomas, Gilles, all! Thanks for reviewing this! > 25 марта 2021 г., в 02:31, Thomas Munro написал(а): > > I don't think we should put "slru" (the name of the buffer replacement > algorithm, implementation detail) in the GUC names. +1 > +It defaults to 0, in this case CLOG size is t

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 14:02:49 +0900, Fujii Masao wrote in > > > On 2021/03/26 13:28, Kyotaro Horiguchi wrote: > >> "some contexts are omitted" > >> "n child contexts: total_bytes = ..." > > Sorry I missed that is already implemented. So my opnion is I agree > > with limiting with a fixed-numbe

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Fujii Masao
On 2021/03/26 13:28, Kyotaro Horiguchi wrote: "some contexts are omitted" "n child contexts: total_bytes = ..." Sorry I missed that is already implemented. So my opnion is I agree with limiting with a fixed-number, and preferablly sorted in descending order of... totalspace/nblocks? This

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 13:17:21 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 26 Mar 2021 12:49:16 +0900, Fujii Masao > wrote in > > On 2021/03/26 12:26, Fujii Masao wrote: > > > On 2021/03/26 12:01, Kyotaro Horiguchi wrote: > > >> At Thu, 25 Mar 2021 23:45:17 +0900, torikoshia > > >> wrote

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 12:49:16 +0900, Fujii Masao wrote in > On 2021/03/26 12:26, Fujii Masao wrote: > > On 2021/03/26 12:01, Kyotaro Horiguchi wrote: > >> At Thu, 25 Mar 2021 23:45:17 +0900, torikoshia > >> wrote in > >>> Attached new one. > > Regarding the argument max_children, isn't it better

Re: increase size of pg_commit_ts buffers

2021-03-25 Thread Thomas Munro
On Mon, Feb 15, 2021 at 11:56 PM Andrey Borodin wrote: > > 16 янв. 2021 г., в 03:07, Alvaro Herrera > > написал(а): > > Andrey Borodin already has a patch to change the behavior for > > multixact, which is something we should perhaps also do. I now notice > > that they're also reporting a bug i

Re: [UNVERIFIED SENDER] Re: Challenges preventing us moving to 64 bit transaction id (XID)?

2021-03-25 Thread Thomas Munro
On Fri, Mar 26, 2021 at 2:57 AM David Steele wrote: > On 1/22/21 6:46 PM, Finnerty, Jim wrote: > > First 3 patches derived from the original 64-bit xid patch set by Alexander > > Korotkov > > The patches no longer apply > (http://cfbot.cputube.org/patch_32_2960.log), so marked Waiting on Author.

Re: Logical Replication - improve error message while adding tables to the publication in check_publication_add_relation

2021-03-25 Thread Bharath Rupireddy
On Thu, Mar 11, 2021 at 8:26 PM Bharath Rupireddy wrote: > > On Wed, Mar 10, 2021 at 5:48 PM Euler Taveira wrote: > > > > On Wed, Mar 10, 2021, at 2:14 AM, Bharath Rupireddy wrote: > > > > While providing thoughts on [1], I observed that the error messages > > that are emitted while adding foreig

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 12:26:31 +0900, Fujii Masao wrote in > > > On 2021/03/26 12:01, Kyotaro Horiguchi wrote: > > At Thu, 25 Mar 2021 23:45:17 +0900, torikoshia > > wrote in > >> Attached new one. > > Regarding the argument max_children, isn't it better to set its > default value, > e.g., 100

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Fujii Masao
On 2021/03/26 12:26, Fujii Masao wrote: On 2021/03/26 12:01, Kyotaro Horiguchi wrote: At Thu, 25 Mar 2021 23:45:17 +0900, torikoshia wrote in Attached new one. Regarding the argument max_children, isn't it better to set its default value, e.g., 100 like MemoryContextStats() uses? +

Re: wal stats questions

2021-03-25 Thread Kyotaro Horiguchi
At Fri, 26 Mar 2021 10:32:23 +0900, Fujii Masao wrote in > > I may be misunderstanding or missing something, but the only place > > where pgWalUsage counters are increased is XLogInsertRecrod. That is, > > pgWalUsage counts wal insertions, not writes nor flushes. AFAICS > > Yes. And WalStats

Re: MultiXact\SLRU buffers configuration

2021-03-25 Thread Thomas Munro
Hi Andrey, all, I propose some changes, and I'm attaching a new version: I renamed the GUCs as clog_buffers etc (no "_slru_"). I fixed some copy/paste mistakes where the different GUCs were mixed up. I made some changes to the .conf.sample. I rewrote the documentation so that it states the cor

Re: [PATCH] add concurrent_abort callback for output plugin

2021-03-25 Thread Amit Kapila
On Thu, Mar 25, 2021 at 2:37 PM Markus Wanner wrote: > > here is another tidbit from our experience with using logical decoding. > The attached patch adds a callback to notify the output plugin of a > concurrent abort. I'll continue to describe the problem in more detail > and how this additional

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Fujii Masao
On 2021/03/26 12:01, Kyotaro Horiguchi wrote: At Thu, 25 Mar 2021 23:45:17 +0900, torikoshia wrote in Attached new one. Regarding the argument max_children, isn't it better to set its default value, e.g., 100 like MemoryContextStats() uses? + ereport(WARNING, +

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Kyotaro Horiguchi
At Thu, 25 Mar 2021 23:45:17 +0900, torikoshia wrote in > Attached new one. Thanks! -SELECT * FROM pg_get_backend_memory_contexts(); +SELECT * FROM pg_get_backend_memory_contexts(0, 0); However we can freely change the signature since it's new in 14, but I see the (void) signature as

Re: making update/delete of inheritance trees scale better

2021-03-25 Thread Tom Lane
Amit Langote writes: > On Fri, Mar 26, 2021 at 3:07 AM Tom Lane wrote: >> I think the reason is that the parser >> initially builds all INSERT ... SELECT cases with the SELECT as an >> explicit subquery, giving rise to a SubqueryScan node just below the >> ModifyTable, which will project exactly

Re: Replication slot stats misgivings

2021-03-25 Thread Amit Kapila
On Fri, Mar 26, 2021 at 1:17 AM Andres Freund wrote: > > Hi, > > On 2021-03-25 17:12:31 +0530, Amit Kapila wrote: > > On Thu, Mar 25, 2021 at 11:36 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Mar 24, 2021 at 7:06 PM Amit Kapila > > > wrote: > > > > > > > > > > > > Leaving aside restart c

Re: making update/delete of inheritance trees scale better

2021-03-25 Thread Amit Langote
On Fri, Mar 26, 2021 at 3:07 AM Tom Lane wrote: > Robert Haas writes: > > - Until I went back and found your earlier remarks on this thread, I > > was confused as to why you were replacing a JunkFilter with a > > ProjectionInfo. I think it would be good to try to add some more > > explicit commen

Re: Proposal for col LIKE $1 with generic Plan

2021-03-25 Thread Andy Fan
On Thu, Mar 25, 2021 at 10:15 AM Andy Fan wrote: > Thanks to the get_index_clause_from_support, we can use index for WHERE a > like > 'abc%' case. However this currently only works on custom plan. Think about > the > case where the planning takes lots of time, custom plan will not give a > good >

Re: wal stats questions

2021-03-25 Thread Fujii Masao
On 2021/03/26 10:08, Kyotaro Horiguchi wrote: At Thu, 25 Mar 2021 19:01:23 +0900, Fujii Masao wrote in On 2021/03/25 16:37, Kyotaro Horiguchi wrote: pgWalUsage was used without resetting and we (I) thought that that behavior should be preserved. On second thought, as Andres suggested, we

Re: Stronger safeguard for archive recovery not to miss data

2021-03-25 Thread Fujii Masao
On 2021/03/25 23:21, David Steele wrote: On 1/25/21 3:55 AM, Laurenz Albe wrote: On Mon, 2021-01-25 at 08:19 +, osumi.takami...@fujitsu.com wrote: I think you should pst another patch where the second, now superfluous, error message is removed. Updated. This patch showed no failure dur

Re: wal stats questions

2021-03-25 Thread Kyotaro Horiguchi
At Thu, 25 Mar 2021 19:01:23 +0900, Fujii Masao wrote in > On 2021/03/25 16:37, Kyotaro Horiguchi wrote: > > pgWalUsage was used without resetting and we (I) thought that that > > behavior should be preserved. On second thought, as Andres suggested, > > we can just reset pgWalUsage at sending s

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-25 Thread Fujii Masao
On 2021/03/26 9:25, Masahiro Ikeda wrote: On 2021/03/25 21:23, Fujii Masao wrote: On 2021/03/25 9:58, Andres Freund wrote: Outside of very narrow circumstances a normal exit shouldn't use _exit(1). Neither 1 no _exit() (as opposed to exit()) seem appropriate. This seems like a bad idea. S

Re: DETAIL for wrong scram password

2021-03-25 Thread Michael Paquier
On Thu, Mar 25, 2021 at 03:54:10PM +, Jacob Champion wrote: > It looks like the code paths that lead to a doomed authentication > already provide their own, more specific, logdetail (role doesn't > exist, role has no password, role doesn't have a SCRAM secret, etc.). Yes, you are right here.

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-25 Thread Masahiro Ikeda
On 2021/03/25 19:48, Fujii Masao wrote: > > > On 2021/03/25 9:31, Masahiro Ikeda wrote: >> >> >> On 2021/03/24 18:36, Fujii Masao wrote: >>> >>> >>> On 2021/03/24 3:51, Andres Freund wrote: Hi, On 2021-03-23 15:50:46 +0900, Fujii Masao wrote: > This fact makes me wonder that

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-25 Thread Masahiro Ikeda
On 2021/03/25 21:23, Fujii Masao wrote: > On 2021/03/25 9:58, Andres Freund wrote: >> Outside of very >> narrow circumstances a normal exit shouldn't use _exit(1). Neither 1 no >> _exit() (as opposed to exit()) seem appropriate. This seems like a bad >> idea. > > So you're thinking that the stat

Re: [HACKERS] Custom compression methods

2021-03-25 Thread Justin Pryzby
On Mon, Mar 22, 2021 at 10:41:33AM -0400, Robert Haas wrote: > trying to explain how TOAST works here, I added a link. It looks, > though, like that documentation also needs to be patched for this > change. I'll look into that, and your remaining patches, next. I added an Opened Item for any neces

Re: Proposal: Save user's original authenticated identity for logging

2021-03-25 Thread Michael Paquier
On Thu, Mar 25, 2021 at 06:51:22PM +, Jacob Champion wrote: > On Thu, 2021-03-25 at 14:41 +0900, Michael Paquier wrote: >> + port->authn_id = MemoryContextStrdup(TopMemoryContext, id); >> It may not be obvious that all the field is copied to TopMemoryContext >> because the Port requires that.

Re: [HACKERS] Custom compression methods

2021-03-25 Thread Robert Haas
On Thu, Mar 25, 2021 at 5:44 AM Dilip Kumar wrote: > Okay got it. Fixed as suggested. Committed with a bit of editing of the comments. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Support for NSS as a libpq TLS backend

2021-03-25 Thread Jacob Champion
On Fri, 2021-03-26 at 00:22 +0100, Daniel Gustafsson wrote: > > On 23 Mar 2021, at 20:04, Stephen Frost wrote: > > > > Eh, poor wording on my part. You're right, the question, reworded > > again, was "Would someone want to get the context returned by > > NSS_InitContext?". If we think there's a

Re: shared memory stats: high level design decisions: consistency, dropping

2021-03-25 Thread Magnus Hagander
On Thu, Mar 25, 2021 at 6:20 PM Andres Freund wrote: > > Hi, > > On 2021-03-24 14:42:11 +0100, Magnus Hagander wrote: > > On Sun, Mar 21, 2021 at 11:34 PM Andres Freund wrote: > > > > If I understand what you are proposing, all stats views would become > > > > completely volatile, without even wi

Re: [PATCH] add concurrent_abort callback for output plugin

2021-03-25 Thread Markus Wanner
On 25.03.21 21:21, Andres Freund wrote: ... the code added as part of 7259736a6e5b7c75 ... That's the streaming part, which can be thought of as a more general variant of the two-phase decoding in that it allows multiple "flush points" (invoking ReorderBufferProcessTXN). Unlike the PREPARE o

Re: [PATCH] More docs on what to do and not do in extension code

2021-03-25 Thread Bruce Momjian
On Thu, Mar 25, 2021 at 08:49:44AM -0400, David Steele wrote: > On 1/22/21 1:36 AM, Craig Ringer wrote: > > > > Would you mind attaching a revised version of the patch with your edits? > > Otherwise I'll go and merge them in once you've had your say on my > > comments inline below. > > Bharath, d

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-25 Thread Bruce Momjian
On Wed, Mar 24, 2021 at 11:20:49PM +0800, Julien Rouhaud wrote: > On Wed, Mar 24, 2021 at 08:13:40AM -0400, Bruce Momjian wrote: > > I have no local modifications. Please modify the patch I posted and > > repost your version, thanks. > > Ok! I used the last version of the patch you sent and addr

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-25 Thread Alvaro Herrera
On 2020-Nov-30, Justin Pryzby wrote: > On Tue, Nov 03, 2020 at 08:56:06PM -0300, Alvaro Herrera wrote: > > * On the first transaction (the one that marks the partition as > > detached), the partition is locked with ShareLock rather than > > ShareUpdateExclusiveLock. This means that DML is not al

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-25 Thread Alvaro Herrera
I added that test as promised, and I couldn't find any problems, so I have pushed it. Thanks! -- Álvaro Herrera Valdivia, Chile

Re: [PATCH] add concurrent_abort callback for output plugin

2021-03-25 Thread Andres Freund
Hi, On 2021-03-25 10:07:28 +0100, Markus Wanner wrote: > This leads to the possibility of the transaction getting aborted concurrent > to logical decoding. In that case, it is likely for the decoder to error on > a catalog scan that conflicts with the abort of the transaction. The > reorderbuffe

Re: [PATCH] Covering SPGiST index

2021-03-25 Thread Pavel Borisov
In a v14 I forgot to add the test. PFA v15 -- Best regards, Pavel Borisov Postgres Professional: http://postgrespro.com v15-0001-Covering-SP-GiST-index-support-for-INCLUDE-colum.patch Description: Binary data

Re: [PATCH] Covering SPGiST index

2021-03-25 Thread Pavel Borisov
> > On further contemplation, it occurs to me that if we make the switch > to "key values are stored per normal rules", then even if there is an > index with pass-by-value keys out there someplace, it would only break > on big-endian architectures. On little-endian, the extra space > occupied by t

Re: Replication slot stats misgivings

2021-03-25 Thread Andres Freund
Hi, On 2021-03-25 17:12:31 +0530, Amit Kapila wrote: > On Thu, Mar 25, 2021 at 11:36 AM Masahiko Sawada > wrote: > > > > On Wed, Mar 24, 2021 at 7:06 PM Amit Kapila wrote: > > > > > > > > > Leaving aside restart case, without some sort of such sanity checking, > > > if both drop (of old slot) a

Re: Error on failed COMMIT

2021-03-25 Thread David Steele
On 3/25/21 3:07 PM, Dave Cramer wrote: On Thu, 25 Mar 2021 at 12:04, David Steele > wrote: Test are failing on the cfbot for this patch and it looks like a new patch is needed from Dave, at the least, so marking Waiting on Author. Should we be considering

Re: Error on failed COMMIT

2021-03-25 Thread Dave Cramer
On Thu, 25 Mar 2021 at 12:04, David Steele wrote: > On 1/26/21 1:02 PM, Dave Cramer wrote: > > On Tue, 26 Jan 2021 at 12:46, Laurenz Albe > > wrote: > > > > I see your point from the view of the JDBC driver. > > > > It just feels hacky - somewhat similar

Re: shared memory stats: high level design decisions: consistency, dropping

2021-03-25 Thread Robert Haas
On Sun, Mar 21, 2021 at 12:14 PM Tom Lane wrote: > If I understand what you are proposing, all stats views would become > completely volatile, without even within-query consistency. That really > is not gonna work. As an example, you could get not-even-self-consistent > results from a join to a

Re: some pointless HeapTupleHeaderIndicatesMovedPartitions calls

2021-03-25 Thread Álvaro Herrera
On 2021-Feb-24, Michael Paquier wrote: > On Mon, Feb 22, 2021 at 05:15:57PM -0300, Álvaro Herrera wrote: > > I changed my mind on this after noticing that > > ItemPointerIndicatesMovedPartitions has a few callers; leaving the > > interface incomplete/asymmetric would be worse. So I propose to do

Re: Proposal: Save user's original authenticated identity for logging

2021-03-25 Thread Jacob Champion
On Thu, 2021-03-25 at 14:41 +0900, Michael Paquier wrote: > I got to look at the DN patch yesterday, so now's the turn of this > one. Nice work. Thanks! > + port->authn_id = MemoryContextStrdup(TopMemoryContext, id); > It may not be obvious that all the field is copied to TopMemoryContext > be

Re: making update/delete of inheritance trees scale better

2021-03-25 Thread Tom Lane
Robert Haas writes: > - Until I went back and found your earlier remarks on this thread, I > was confused as to why you were replacing a JunkFilter with a > ProjectionInfo. I think it would be good to try to add some more > explicit comments about that design choice, perhaps as header comments > f

document lock obtained for FKs during detach

2021-03-25 Thread Alvaro Herrera
I failed to document the lock acquired on tables that reference the partitioned table that we're detaching a partition from. This patch (proposed for backpatch to 12, where that feature was added) fixes that problem. Noticed the omission a week ago while working out some details of concurrent det

Re: shared memory stats: high level design decisions: consistency, dropping

2021-03-25 Thread Andres Freund
Hi, On 2021-03-21 14:53:50 -0700, Andres Freund wrote: > On 2021-03-21 11:41:30 -0400, Stephen Frost wrote: > > > 2) How to remove stats of dropped objects? > > > > > > [...] > > > > The current approach sounds pretty terrible and propagating that forward > > doesn't seem great. Guess here I'd di

Re: shared memory stats: high level design decisions: consistency, dropping

2021-03-25 Thread Andres Freund
Hi, On 2021-03-24 14:42:11 +0100, Magnus Hagander wrote: > On Sun, Mar 21, 2021 at 11:34 PM Andres Freund wrote: > > > If I understand what you are proposing, all stats views would become > > > completely volatile, without even within-query consistency. That really > > > is not gonna work. As a

Re: [PATCH] pg_permissions

2021-03-25 Thread Tom Lane
"Joel Jacobson" writes: > On Thu, Mar 25, 2021, at 16:16, Alvaro Herrera wrote: >> Ah, of course -- the only way to obtain the acl columns is by going >> through the catalogs individually, so it won't be possible. I think >> this could be fixed with some very simple, quick function pg_get_acl() >

Re: [bug fix] ALTER TABLE SET LOGGED/UNLOGGED on a partitioned table does nothing silently

2021-03-25 Thread David Steele
On 1/27/21 1:00 AM, Kyotaro Horiguchi wrote: At Wed, 27 Jan 2021 05:30:29 +, "tsunakawa.ta...@fujitsu.com" wrote in From: Kyotaro Horiguchi "CREATE TABLE" is not "CREATE LOGGED TABLE". We can assume that as "CREATE TABLE", where the default logged-ness varies according to context. Or it

Re: [PATCH] pg_permissions

2021-03-25 Thread Joel Jacobson
On Thu, Mar 25, 2021, at 16:16, Alvaro Herrera wrote: > On 2021-Mar-25, Joel Jacobson wrote: > > > pg_shdepend doesn't contain the aclitem info though, > > so it won't work for pg_permissions if we want to expose > > privilege_type, is_grantable and grantor. > > Ah, of course -- the only way to o

Re: [HACKERS] Custom compression methods

2021-03-25 Thread Robert Haas
On Wed, Mar 24, 2021 at 2:15 PM Tom Lane wrote: > But let's ignore the case of pg_upgrade and just consider a dump/restore. > I'd still say that unless you give --no-toast-compression then I would > expect the dump/restore to preserve the tables' old compression behavior. > Robert's argument that

Re: Error on failed COMMIT

2021-03-25 Thread David Steele
On 1/26/21 1:02 PM, Dave Cramer wrote: On Tue, 26 Jan 2021 at 12:46, Laurenz Albe > wrote: I see your point from the view of the JDBC driver. It just feels hacky - somewhat similar to what you say above: don't go through the normal transaction rollba

Re: DETAIL for wrong scram password

2021-03-25 Thread Jacob Champion
On Thu, 2021-03-25 at 16:41 +0900, Michael Paquier wrote: > On top of what's > proposed, would it make sense to have a second logdetail for the case > of a mock authentication? We don't log that yet, so I guess that it > could be useful for audit purposes? It looks like the code paths that lead to

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-03-25 Thread Alvaro Herrera
On 2021-Mar-23, Alvaro Herrera wrote: > I think a possible solution to this problem is that the "detach" flag in > pg_inherits is not a boolean anymore, but an Xid (or maybe two Xids). > Not sure exactly which Xid(s) yet, and I'm not sure what are the exact > rules, but the Xid becomes a marker th

Re: Is it useful to record whether plans are generic or custom?

2021-03-25 Thread torikoshia
On 2021-03-25 22:14, Fujii Masao wrote: On 2021/03/23 16:32, torikoshia wrote: On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! Thanks for updating the patch! PostgreSQL Patch Tester reported that the patched version failed to be compiled at Windows. Could you fix this iss

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-03-25 Thread Justin Pryzby
> diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c > index afc45429ba4..723eccca013 100644 > --- a/src/backend/commands/explain.c > +++ b/src/backend/commands/explain.c > @@ -1589,29 +1589,82 @@ ExplainNode(PlanState *planstate, List *ancestors, > double

Re: [PATCH] pg_permissions

2021-03-25 Thread Alvaro Herrera
On 2021-Mar-25, Joel Jacobson wrote: > pg_shdepend doesn't contain the aclitem info though, > so it won't work for pg_permissions if we want to expose > privilege_type, is_grantable and grantor. Ah, of course -- the only way to obtain the acl columns is by going through the catalogs individually,

Re: FETCH FIRST clause PERCENT option

2021-03-25 Thread David Steele
On 1/26/21 11:29 AM, Surafel Temesgen wrote: On Mon, Jan 25, 2021 at 2:39 PM Kyotaro Horiguchi mailto:horikyota@gmail.com>> wrote: Sorry for the dealy. I started to look this. Hi kyotaro, Thanks for looking into this but did we agree to proceed on this approach? I fear that it will b

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread torikoshia
On 2021-03-25 22:02, Fujii Masao wrote: On 2021/03/25 0:17, torikoshia wrote: On 2021-03-23 17:24, Kyotaro Horiguchi wrote: Thanks for reviewing and suggestions! The patched version failed to be compiled as follows. Could you fix this issue? Sorry, it included a header file that's not cont

Re: Stronger safeguard for archive recovery not to miss data

2021-03-25 Thread David Steele
On 1/25/21 3:55 AM, Laurenz Albe wrote: On Mon, 2021-01-25 at 08:19 +, osumi.takami...@fujitsu.com wrote: I think you should pst another patch where the second, now superfluous, error message is removed. Updated. This patch showed no failure during regression tests and has been aligned by

Re: [UNVERIFIED SENDER] Re: Challenges preventing us moving to 64 bit transaction id (XID)?

2021-03-25 Thread David Steele
On 1/22/21 6:46 PM, Finnerty, Jim wrote: First 3 patches derived from the original 64-bit xid patch set by Alexander Korotkov The patches no longer apply (http://cfbot.cputube.org/patch_32_2960.log), so marked Waiting on Author. I also removed the PG14 target since this is a fresh patch set

Re: Tying an object's ownership to datdba

2021-03-25 Thread John Naylor
On Thu, Mar 25, 2021 at 12:07 AM Noah Misch wrote: > > > In the refactoring patch, there is a lingering comment reference to roles_has_privs_of(). Aside from that, it looks good to me. A possible thing to consider is an assert that is_admin is not null where we expect that. > > Thanks. The next v

Re: Polyphase merge is obsolete

2021-03-25 Thread David Steele
On 3/25/21 9:41 AM, David Steele wrote: On 1/22/21 5:19 PM, Heikki Linnakangas wrote: On 22/10/2020 14:48, Heikki Linnakangas wrote: On 11/09/2017 13:37, Tomas Vondra wrote: I planned to do some benchmarking on this patch, but apparently the patch no longer applies. Rebase please? Here's a r

Re: Polyphase merge is obsolete

2021-03-25 Thread David Steele
On 1/22/21 5:19 PM, Heikki Linnakangas wrote: On 22/10/2020 14:48, Heikki Linnakangas wrote: On 11/09/2017 13:37, Tomas Vondra wrote: I planned to do some benchmarking on this patch, but apparently the patch no longer applies. Rebase please? Here's a rebase of this. Sorry to keep you waiting

Re: PoC/WIP: Extended statistics on expressions

2021-03-25 Thread Dean Rasheed
On Thu, 25 Mar 2021 at 00:05, Tomas Vondra wrote: > > here's an updated patch. 0001 The change to the way that CreateStatistics() records dependencies isn't quite right -- recordDependencyOnSingleRelExpr() will not create any dependencies if the expression uses only a whole-row Var. However, pull

Re: pg_rewind copies

2021-03-25 Thread David Steele
On 1/22/21 8:26 AM, Heikki Linnakangas wrote: On 16/12/2020 00:08, Cary Huang wrote: The following review has been posted through the commitfest application: make installcheck-world:  tested, passed Implements feature:   tested, passed Spec compliant:   tested, passed Documentation: 

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-25 Thread Andrew Dunstan
On 3/24/21 7:49 PM, Daniel Gustafsson wrote: >> On 25 Mar 2021, at 00:26, Alvaro Herrera wrote: >> >> On 2021-Mar-25, Daniel Gustafsson wrote: >> >>> Attached is a v2 which addresses the comments raised on the main NSS >>> thread, as >>> well as introduces named parameters for the server cert f

Re: Is it useful to record whether plans are generic or custom?

2021-03-25 Thread Fujii Masao
On 2021/03/23 16:32, torikoshia wrote: On 2021-03-05 17:47, Fujii Masao wrote: Thanks for your comments! Thanks for updating the patch! PostgreSQL Patch Tester reported that the patched version failed to be compiled at Windows. Could you fix this issue? https://ci.appveyor.com/project/post

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-25 Thread Fujii Masao
On 2021/03/25 11:50, Masahiro Ikeda wrote: On 2021/03/23 16:10, Fujii Masao wrote: On 2021/03/22 20:25, ikedamsh wrote: Agreed. Users can know whether the stats is for walreceiver or not. The pg_stat_wal view in standby server shows for the walreceiver, and in primary server it shows for

Re: Get memory contexts of an arbitrary backend process

2021-03-25 Thread Fujii Masao
On 2021/03/25 0:17, torikoshia wrote: On 2021-03-23 17:24, Kyotaro Horiguchi wrote: Thanks for reviewing and suggestions! The patched version failed to be compiled as follows. Could you fix this issue? mcxtfuncs.c:22:10: fatal error: utils/mcxtfuncs.h: No such file or directory #include "

Re: Additional Chapter for Tutorial - arch-dev.sgml

2021-03-25 Thread David Steele
On 1/22/21 4:15 AM, Heikki Linnakangas wrote: On 21/01/2021 14:38, Jürgen Purtz wrote: This supervisor process is called postmaster and listens at a specified TCP/IP port for incoming connections. Whenever he detects a request for a connection, he spawns a new backend process. It sounds weird

Re: [PATCH] More docs on what to do and not do in extension code

2021-03-25 Thread David Steele
On 1/22/21 1:36 AM, Craig Ringer wrote: Would you mind attaching a revised version of the patch with your edits? Otherwise I'll go and merge them in once you've had your say on my comments inline below. Bharath, do the revisions in [1] look OK to you? Bruce, Robert, can I have an opinion fr

Re: insensitive collations

2021-03-25 Thread Daniel Verite
Jim Finnerty wrote: > Currently nondeterministic collations are disabled at the database level. Deterministic ICU collations are also disabled. > The cited reason was because of the lack of LIKE support and because certain > catalog views use LIKE. But the catalogs shouldn't use the d

Re: [PATCH 1/1] Initial mach based shared memory support.

2021-03-25 Thread David Steele
On 1/19/21 6:50 AM, James Hilliard wrote: On Mon, Jan 18, 2021 at 11:12 PM Tom Lane wrote: James Hilliard writes: from my understanding due to mach semantics a number of the sanity checks we are doing for sysv shmem are probably unnecessary when using mach API's due to the mach port task bin

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-25 Thread Fujii Masao
On 2021/03/25 9:58, Andres Freund wrote: Hi, On 2021-03-24 18:36:14 +0900, Fujii Masao wrote: Barring any objection, I'm thinking to commit this patch. To which branches? I was thinking to push the patch to the master branch because this is not a bug fix. I am *strongly* opposed to bac

Re: insensitive collations

2021-03-25 Thread Daniel Verite
Jim Finnerty wrote: > For a UTF8 encoded, case-insensitive (nondeterministic), accent-sensitive > ICU > collation, a LIKE predicate can be used with a small transformation of the > predicate, and the pattern can contain multi-byte characters: > > from: > > SELECT * FROM locations WHERE l

Re: SQL/JSON: JSON_TABLE

2021-03-25 Thread David Steele
On 1/20/21 8:42 PM, Nikita Glukhov wrote: Thank you for review. Attached 45th version of the patches. "SQL/JSON functions" patch corresponds to v52 patch set posted in the separate thread. Another rebase needed (http://cfbot.cputube.org/patch_32_2902.log), marked Waiting on Author. I can se

Re: Replication slot stats misgivings

2021-03-25 Thread Amit Kapila
On Thu, Mar 25, 2021 at 11:36 AM Masahiko Sawada wrote: > > On Wed, Mar 24, 2021 at 7:06 PM Amit Kapila wrote: > > > > > > Leaving aside restart case, without some sort of such sanity checking, > > if both drop (of old slot) and create (of new slot) messages are lost > > then we will start accumu

Re: PoC/WIP: Extended statistics on expressions

2021-03-25 Thread Dean Rasheed
On Thu, 25 Mar 2021 at 00:05, Tomas Vondra wrote: > > Actually, I think we need that block at all - there's no point in > keeping the exact expression, because if there was a statistics matching > it it'd be matched by the examine_variable. So if we get here, we have > to just split it into the va

Re: shared-memory based stats collector

2021-03-25 Thread Kyotaro Horiguchi
At Mon, 15 Mar 2021 17:51:31 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 12 Mar 2021 22:20:40 -0800, Andres Freund wrote > in > > Horiguchi-san, is there a chance you could add a few tests (on master) > > that test/document the way stats are kept across "normal" restarts, and > > thrown

Re: row filtering for logical replication

2021-03-25 Thread Peter Eisentraut
On 22.03.21 03:15, Euler Taveira wrote: I attached a new patch set that addresses: * fix documentation; * rename PublicationRelationQual to PublicationRelationInfo; * remove the memset that was leftover from a previous patch set; * add new tests to improve coverage (INSERT/UPDATE/DELETE to exerc

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-25 Thread Amit Kapila
On Wed, Mar 24, 2021 at 3:59 PM houzj.f...@fujitsu.com wrote: > > > I have incorporated all your changes and additionally made few more changes > > (a) got rid of LogicalRepBeginPrepareData and instead used > > LogicalRepPreparedTxnData, (b) made a number of changes in comments and > > docs, (c) r

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-25 Thread Amul Sul
On Thu, Mar 25, 2021 at 12:10 PM Kyotaro Horiguchi wrote: > > Sorry for the bug. > > At Thu, 25 Mar 2021 01:50:29 -0400, Tom Lane wrote in > > Amul Sul writes: > > > On Wed, Mar 24, 2021 at 8:09 PM Tom Lane wrote: > > >> static inline struct SMgrRelationData * > > >> RelationGetSmgr(Relation re

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-25 Thread Fujii Masao
On 2021/03/25 9:31, Masahiro Ikeda wrote: On 2021/03/24 18:36, Fujii Masao wrote: On 2021/03/24 3:51, Andres Freund wrote: Hi, On 2021-03-23 15:50:46 +0900, Fujii Masao wrote: This fact makes me wonder that if we collect the statistics about WAL writing from walreceiver as we discussed

Re: wal stats questions

2021-03-25 Thread Fujii Masao
On 2021/03/25 16:37, Kyotaro Horiguchi wrote: At Wed, 24 Mar 2021 21:07:26 -0700, Andres Freund wrote in Hi, On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: On 2021/03/25 8:22, Andres Freund wrote: 1) What is the motivation to have both prevWalUsage and pgWalUsage, instead of just

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-03-25 Thread e . sokolova
Thank you all for your feedback and reforms. I attach a new version of the patch with the some changes and fixes. Here's a list of the major changes: 1) New format of extra statistics. This is now contained in a line separate from the main statistics. Julien Rouhaud писал 2021-02-01 08:28: On

Re: [PATCH] pg_permissions

2021-03-25 Thread Joel Jacobson
On Tue, Mar 23, 2021, at 21:39, Alvaro Herrera wrote: >I wonder if these views should be defined on top of pg_shdepend instead >of querying every single catalog. That would make for much shorter >queries. +1 pg_shdepend doesn't contain the aclitem info though, so it won't work for pg_permissions

Re: proposal: unescape_text function

2021-03-25 Thread Peter Eisentraut
On 10.03.21 14:52, David Steele wrote: I thought about it a little bit more, and  the prefix specification has not too much sense (more if we implement this functionality as function "unistr"). I removed the optional argument and renamed the function to "unistr". The functionality is the same

Re: [HACKERS] Custom compression methods

2021-03-25 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 10:57 PM Robert Haas wrote: > > On Wed, Mar 24, 2021 at 12:14 PM Dilip Kumar wrote: > > Actually, we are already doing this, I mean ALTER TABLE .. ALTER > > COLUMN .. SET COMPRESSION is already updating the compression method > > of the index attribute. So 0003 doesn't ma

[PATCH] add concurrent_abort callback for output plugin

2021-03-25 Thread Markus Wanner
Hi, here is another tidbit from our experience with using logical decoding. The attached patch adds a callback to notify the output plugin of a concurrent abort. I'll continue to describe the problem in more detail and how this additional callback solves it. Streamed transactions as well as

Re: [PATCH] Provide more information to filter_prepare

2021-03-25 Thread Markus Wanner
On 22.03.21 09:50, Markus Wanner wrote: thank you for reconsidering this patch.  I updated it to include the required adjustments to the documentation.  Please review. I tweaked the wording in the docs a bit, resulting in a v3 of this patch. Regards Markus From: Markus Wanner Date: Tue, 2 Ma

Re: multi-install PostgresNode

2021-03-25 Thread Peter Eisentraut
On 25.03.21 04:47, Michael Paquier wrote: On Wed, Mar 24, 2021 at 03:33:51PM -0300, Alvaro Herrera wrote: On 2021-Mar-24, Andrew Dunstan wrote: +# The install path set in get_new_node needs to be a directory containing +# bin and lib subdirectories as in a standard PostgreSQL installation, so t

Re: proposal: schema variables - doc

2021-03-25 Thread Pavel Stehule
ed > > >> >> In the CREATE VARIABLE page the example is: >> >> CREATE VARIABLE var1 AS integer; >> SELECT var1; >> >> I suggest to make that >> >> CREATE VARIABLE var1 AS date; >> LET var1 = (select current_date); >> SELECT var1; >> >> So that the example immediately shows an application of functionality. >> > > done > > Thank you for the documentation review. > > Updated patch attached > > Regards > > Pavel > > fresh update with merged Eric's changes in documentation Regards Pavel > >> >> Thanks, >> >> Erik Rijkers >> >> >> >> >> >> >> >> >> >> >> >> >> >> > >> > Pavel >> > schema-variables-20210325.patch.gz Description: application/gzip

Re: wal stats questions

2021-03-25 Thread Kyotaro Horiguchi
At Wed, 24 Mar 2021 21:07:26 -0700, Andres Freund wrote in > Hi, > > On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: > > On 2021/03/25 8:22, Andres Freund wrote: > > > 1) What is the motivation to have both prevWalUsage and pgWalUsage, > > >instead of just accumulating the stats since th

Re: DETAIL for wrong scram password

2021-03-25 Thread Michael Paquier
On Tue, Mar 02, 2021 at 05:48:05PM +, Jacob Champion wrote: > What would you think about adding the additional detail right after > verify_client_proof() fails? I.e. Agreed. Having that once all the code paths have been taken and the client proof has been verified looks more solid. On top of

Re: fix typo in reorderbuffer.c

2021-03-25 Thread Michael Paquier
On Wed, Mar 24, 2021 at 10:52:14AM +, houzj.f...@fujitsu.com wrote: > Sorry for the late reply and yes, " Combo CID(s)" looks better. > Attaching patch which replaces all styles "Combocid(s)" with " Combo CID(s)". I have double-checked the code, adjusted things a bit to adapt with some of the

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-25 Thread Peter Smith
On Thu, Mar 25, 2021 at 1:40 PM Peter Smith wrote: > > On Wed, Mar 24, 2021 at 11:31 PM Amit Kapila wrote: > > > > On Tue, Mar 23, 2021 at 3:31 PM Peter Smith wrote: > > > > > > On Tue, Mar 23, 2021 at 10:44 AM Peter Smith > > > wrote: > > > > > > > > PSA patches to fix those. > > > > > > > >

  1   2   >