Re: shared-memory based stats collector - v70

2022-04-06 Thread Andres Freund
Hi, On 2022-04-06 15:32:39 -0700, David G. Johnston wrote: > On Wednesday, April 6, 2022, Andres Freund wrote: > > > > > > > I'd go for > > pgstat_reset_slru_counter() -> pgstat_reset_slru() > > pgstat_reset_subscription_counter() -> pgstat_reset_subscription() > > pgstat_reset_subscription_coun

Re: PATCH: add "--config-file=" option to pg_rewind

2022-04-06 Thread Michael Paquier
On Wed, Apr 06, 2022 at 02:32:44PM +0200, Gunnar "Nick" Bluth wrote: > Ta! Sorry, been ridiculously busy these days, and I do confess that I've > been struggling a bit with those tests before ;-) No problem. Double-checked this morning and applied, then. -- Michael signature.asc Description: PG

Re: Mark all GUC variable as PGDLLIMPORT

2022-04-06 Thread Michael Paquier
On Wed, Apr 06, 2022 at 12:57:29AM +0700, John Naylor wrote: > For these two patches, I'd say a day or two after feature freeze is a > reasonable goal. Yeah. For patches as invasive as the PGDLLIMPORT business and the frontend error refactoring, I am also fine to have two exceptions with the free

Re: shared-memory based stats collector - v70

2022-04-06 Thread David G. Johnston
On Wed, Apr 6, 2022 at 4:12 PM Andres Freund wrote: > > On 2022-04-06 15:32:39 -0700, David G. Johnston wrote: > > On Wednesday, April 6, 2022, Andres Freund wrote: > > > > > > I like having the SQL function paired with a matching implementation in > > this scheme. > > It would have gotten thing

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-04-06 Thread Michael Paquier
On Wed, Apr 06, 2022 at 07:16:43PM +, Jacob Champion wrote: > I assumed that we would follow the existing model of "(de)serialize a > whole struct", rather than pulling it apart into many separate keys. If > it got too complicated then we could consider introducing a > SerializedParallelProcInf

Re: shared-memory based stats collector - v70

2022-04-06 Thread Andres Freund
Hi, On 2022-04-06 17:01:17 -0700, David G. Johnston wrote: > On Wed, Apr 6, 2022 at 4:12 PM Andres Freund wrote: > > The fact there is just the one outlier here suggests that this is indeed the > better option. FWIW, the outlier also uses pgstat_reset(), just with a small wrapper doing the trans

Re: simplifying foreign key/RI checks

2022-04-06 Thread Amit Langote
There were rebase conflicts with the recently committed execPartition.c/h changes. While fixing them, I thought maybe find_leaf_part_for_key() doesn't quite match in style with its neighbors in execPartition.h, so changed it to ExecGetLeafPartitionForKey(). -- Amit Langote EDB: http://www.enterp

Re: How about a psql backslash command to show GUCs?

2022-04-06 Thread Tom Lane
"Jonathan S. Katz" writes: > I am a bit torn between "\dcp" (or \dsetting / \dconfig? we don't > necessarily need for it to be super short) and "\sc". Certainly with > pattern matching the interface for the "\d" commands would fit that > pattern. "\sc" would make sense for a thorough introspect

Re: How about a psql backslash command to show GUCs?

2022-04-06 Thread David G. Johnston
On Wed, Apr 6, 2022 at 6:16 PM Tom Lane wrote: > "Jonathan S. Katz" writes: > > I am a bit torn between "\dcp" (or \dsetting / \dconfig? we don't > > necessarily need for it to be super short) and "\sc". Certainly with > > pattern matching the interface for the "\d" commands would fit that > > p

Re: How about a psql backslash command to show GUCs?

2022-04-06 Thread Jonathan S. Katz
On 4/6/22 9:18 PM, David G. Johnston wrote: On Wed, Apr 6, 2022 at 6:16 PM Tom Lane I agree that \d-something makes the most sense from a functionality standpoint.  But I don't want to make the name too long, even if we do have tab completion to help. \dconf maybe? I don't ha

Re: Last day of commitfest

2022-04-06 Thread Greg Stark
I won't touch the Ready for Committer stuff until after the end of the commitfest anyways. I did put those two in that state already. Right now I'm trying to get a bit ahead of the game by going through the "Waiting on Author" patches. The documented process[*] is that they get Returned with Feedb

Re: shared-memory based stats collector - v70

2022-04-06 Thread Kyotaro Horiguchi
At Wed, 6 Apr 2022 09:04:09 -0700, Andres Freund wrote in > > +* Don't define an INVALID value so switch() statements can warn if some > > +* cases aren't covered. But define the first member to 1 so that > > +* uninitialized values can be detected more easily. > > > > FWIW, I like t

Re: Should pg_dumpall dump ALTER SYSTEM settings?

2022-04-06 Thread Robert Haas
On Wed, Apr 6, 2022 at 2:26 PM Tom Lane wrote: > Thoughts? I'm a little bit skeptical about this proposal, mostly because it seems like it has the end result that values that are configured in postgresql.conf and postgresql.auto.conf end up being handled differently: one file has to be copied by

Re: Last day of commitfest

2022-04-06 Thread Julien Rouhaud
Hi, On Wed, Apr 06, 2022 at 09:32:31PM -0400, Greg Stark wrote: > I won't touch the Ready for Committer stuff until after the end of the > commitfest anyways. I did put those two in that state already. > > Right now I'm trying to get a bit ahead of the game by going through > the "Waiting on Autho

Re: shared-memory based stats collector - v70

2022-04-06 Thread Andres Freund
Hi, On 2022-04-07 10:36:30 +0900, Kyotaro Horiguchi wrote: > At Wed, 6 Apr 2022 09:04:09 -0700, Andres Freund wrote > in > > > + * Don't define an INVALID value so switch() statements can warn if some > > > + * cases aren't covered. But define the first member to 1 so that > > > + * uninitia

Re: Window Function "Run Conditions"

2022-04-06 Thread David Rowley
On Wed, 6 Apr 2022 at 00:59, Andy Fan wrote: > > On Tue, Apr 5, 2022 at 7:49 PM David Rowley wrote: >> Yeah, there is more performance to be had than even what you've done >> there. There's no reason really for spool_tuples() to do >> tuplestore_puttupleslot() when we're not in run mode. > > > Y

Re: Skipping logical replication transactions on subscriber side

2022-04-06 Thread Amit Kapila
On Wed, Apr 6, 2022 at 10:01 AM Amit Kapila wrote: > > On Wed, Apr 6, 2022 at 9:25 AM Masahiko Sawada wrote: > > > > On Wed, Apr 6, 2022 at 12:21 PM Noah Misch wrote: > > > > Right. I've attached an updated patch. > > > > Thanks, this looks good to me as well. Noah, would you like to commit it?

Re: How about a psql backslash command to show GUCs?

2022-04-06 Thread Tom Lane
"Jonathan S. Katz" writes: > +1 for \dconf Here's a draft patch using \dconf. No tests or docs yet. regards, tom lane diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c index 079f4a1a76..bbf4a5a44e 100644 --- a/src/bin/psql/command.c +++ b/src/bin/psql/command

RE: Data is copied twice when specifying both child and parent table in publication

2022-04-06 Thread wangw.f...@fujitsu.com
On Thur, Mar 10, 2021 at 10:08 AM houzj.f...@fujitsu.com wrote: > Hi, > > When reviewing some logical replication related features. I noticed another > possible problem if the subscriber subscribes multiple publications which > publish parent and child table. > > For example: > > pub > creat

Re: How about a psql backslash command to show GUCs?

2022-04-06 Thread David Rowley
I also find myself querying pg_settings all too often. More typing than I'd like. On Thu, 7 Apr 2022 at 06:40, Tom Lane wrote: > I do agree that \show might be a bad choice, the reason being that > the adjacent \set command is for psql variables not GUCs; if we > had a \show I'd sort of expect it

Re: Window Function "Run Conditions"

2022-04-06 Thread Zhihong Yu
On Wed, Apr 6, 2022 at 7:36 PM David Rowley wrote: > On Wed, 6 Apr 2022 at 00:59, Andy Fan wrote: > > > > On Tue, Apr 5, 2022 at 7:49 PM David Rowley > wrote: > >> Yeah, there is more performance to be had than even what you've done > >> there. There's no reason really for spool_tuples() to do

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2022-04-06 Thread David Rowley
On Wed, 6 Apr 2022 at 03:40, Yura Sokolov wrote: > I'm looking on patch and don't get some moments. > > `GrantLockLocal` allocates `LOCALLOCKOWNER` and links it into > `locallock->locallockowners`. It links it regardless `owner` could be > NULL. But then `RemoveLocalLock` does `Assert(locallockown

Re: Handle infinite recursion in logical replication setup

2022-04-06 Thread Peter Smith
FYI, here is a test script that is using the current patch (v6) to demonstrate a way to share table data between different numbers of nodes (up to 5 of them here). The script starts off with just 2-way sharing (nodes N1, N2), then expands to 3-way sharing (nodes N1, N2, N3), then 4-way sharing (no

Re: Add proper planner support for ORDER BY / DISTINCT aggregates

2022-04-06 Thread David Rowley
On Thu, 31 Mar 2022 at 06:36, Greg Stark wrote: > > This patch is now failing in the sqljson regression test. It looks > like it's just the ordering of the elements in json_arrayagg() calls > which may actually be a faulty test that needs more ORDER BY clauses > rather than any issues with the cod

Re: Logical replication timeout problem

2022-04-06 Thread Amit Kapila
On Wed, Apr 6, 2022 at 6:30 PM wangw.f...@fujitsu.com wrote: > > On Wed, Apr 6, 2022 at 1:59 AM Amit Kapila wrote: > On Wed, Apr 6, 2022 at 4:32 AM Amit Kapila wrote: > > > Thanks for your comments. > > > typedef void (*LogicalOutputPluginWriterUpdateProgress) (struct > > LogicalDecodingContext

Re: suboverflowed subtransactions concurrency performance optimize

2022-04-06 Thread Michael Paquier
On Mon, Mar 07, 2022 at 10:17:41PM +0800, Julien Rouhaud wrote: > On Mon, Mar 07, 2022 at 01:27:40PM +, Simon Riggs wrote: >> The patch was posted because TransactionLogFetch() has a cache, yet >> SubTransGetTopmostTransaction() does not, yet the argument should be >> identical in both cases. >

Re: Change the csv log to 'key:value' to facilitate the user to understanding and processing of logs

2022-04-06 Thread Michael Paquier
On Tue, Mar 15, 2022 at 10:33:42AM -0400, Jan Wieck wrote: > Also, the CSV format, while human readable to a degree, wasn't meant for > direct, human consumption. It was meant to be read by programs and at the > time, CSV made the most sense. FWIW, I have noticed that this patch was still listed i

Re: pg_dump new feature: exporting functions only. Bad or good idea ?

2022-04-06 Thread Michael Paquier
On Sat, Mar 26, 2022 at 09:53:19AM +0100, Laetitia Avrot wrote: > I think it's time to sum up what we want to do: > > - We'd like to use switches to export objects according to a pattern. > - For each object type we will have an --object=PATTERN flag and a > --exclude-object=PATTERN > - Having a s

Re: Supply restore_command to pg_rewind via CLI argument

2022-04-06 Thread Michael Paquier
On Thu, Nov 04, 2021 at 01:55:43PM +0100, Daniel Gustafsson wrote: >> On 14 Sep 2021, at 16:05, Andrey Borodin wrote: >>> Do we actually need --target-restore-command at all? It seems to me >>> that we have all we need with --restore-target-wal, and that's not >>> really instinctive to pass down a

RE: Handle infinite recursion in logical replication setup

2022-04-06 Thread kuroda.hay...@fujitsu.com
Dear Peter, > FYI, here is a test script that is using the current patch (v6) to > demonstrate a way to share table data between different numbers of > nodes (up to 5 of them here). Thanks for sharing your script! It's very helpful for us. While reading your script, however, I had a question abo

Re: Handle infinite recursion in logical replication setup

2022-04-06 Thread Peter Smith
On Thu, Apr 7, 2022 at 4:03 PM kuroda.hay...@fujitsu.com wrote: > > Dear Peter, > > > FYI, here is a test script that is using the current patch (v6) to > > demonstrate a way to share table data between different numbers of > > nodes (up to 5 of them here). > > Thanks for sharing your script! It's

Re: Extensible Rmgr for Table AMs

2022-04-06 Thread Jeff Davis
On Tue, 2022-04-05 at 15:12 +0530, Bharath Rupireddy wrote: > Yes, I meant this. If we do this, then a global variable > current_max_rmid can be maintained (by RegisterCustomRmgr) and the > other functions RmgrStartup, RmgrCleanup and RegisterCustomRmgr can > avoid for-loops with full range RM_MAX_

Re: API stability [was: pgsql: Fix possible recovery trouble if TRUNCATE overlaps a checkpoint.]

2022-04-06 Thread Michael Paquier
On Tue, Apr 05, 2022 at 03:16:20PM -0400, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 5, 2022 at 10:32 AM Tom Lane wrote: >>> My point is that we want that to happen in HEAD, but it's not okay >>> for it to happen in a minor release of a stable branch. > >> I understand, but I am not su

Re: Extensible Rmgr for Table AMs

2022-04-06 Thread Andres Freund
Hi, On 2022-04-06 23:15:27 -0700, Jeff Davis wrote: > I'm happy with this patch and I committed it. That caused breakage with WAL_DEBUG enabled. Fixed that. But it was half-broken before, at least since 70e81861fad, because 'rmid' didn't refer to the current record's rmid anymore, but to rmid fro

Re: [PATCH] Add native windows on arm64 support

2022-04-06 Thread Michael Paquier
On Fri, Mar 25, 2022 at 11:38:44AM +1300, Thomas Munro wrote: > Yeah. I think we should commit this patch, but decree that > Windows/aarch64 support is experimental only for now. That allows a > build farm animal to be set up. Then we add a bit of extra logging > and see how it does running our

Re: Proposal: More structured logging

2022-04-06 Thread Michael Paquier
On Mon, Jan 31, 2022 at 05:46:29PM -0500, Greg Stark wrote: > It looks strange to me that the errorTag struct has a "const char > *tagname" but a "char *tagvalue". I think this is a side effect of the > code and not actually a sensible way to define the struct. Surely they > should both be const?

Re: Extensible Rmgr for Table AMs

2022-04-06 Thread Andres Freund
Hi, On 2022-04-06 23:35:05 -0700, Andres Freund wrote: > Causes plenty new warnings here: And my machine isn't alone. There's also: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=lapwing&dt=2022-04-07%2006%3A40%3A14 rmgrdesc.c:44:1: error: missing braces around initializer [-Werror=mis

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-06 Thread Michael Paquier
On Tue, Apr 05, 2022 at 11:57:14AM +0800, Julien Rouhaud wrote: > This is a minor detail and the rest of the patch looks good to me, so I'm > marking the patch as Ready for Committer! @@ -440,10 +442,14 @@ BufFileLoadBuffer(BufFile *file) + if (track_io_timing) + INSTR_TIME_SET_CURRENT(io_

<    1   2