Re: unlogged sequences

2022-04-04 Thread Peter Eisentraut
On 04.04.22 01:58, David G. Johnston wrote: "Because pg_dump is used to transfer data to newer versions of PostgreSQL, the output of pg_dump can be expected to load into PostgreSQL server versions newer than pg_dump's version." [1] That is what I'm getting on about when talking about migration

Re: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2022-04-04 Thread Julien Rouhaud
Hi, On Mon, Apr 04, 2022 at 09:59:04AM +0300, Andrei Zubkov wrote: > > Minor rephrasing: > > > > s/evicted and returned back/evicted and stored again/? > > s/with except of all/with the exception of all/ > > s/is now returns/now returns/ > > Agreed, commit message updated. > > > - code: > > > > +#

Re: Removing unneeded self joins

2022-04-04 Thread Andrey V. Lepikhov
On 4/1/22 20:27, Greg Stark wrote: Sigh. And now there's a patch conflict in a regression test expected output: sysviews.out Please rebase. Incidentally, make sure to check the expected output is actually correct. It's easy to "fix" an expected output to accidentally just memorialize an incorrec

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-04 Thread Kyotaro Horiguchi
At Fri, 1 Apr 2022 14:51:58 -0400, Robert Haas wrote in > On Fri, Apr 1, 2022 at 12:22 AM Kyotaro Horiguchi > wrote: > > By the way, may I ask how do we fix this? The existing recovery code > > already generates just-to-be-delete files in a real directory in > > pg_tblspc sometimes, and elsewis

collect_corrupt_items_vacuum.patch

2022-04-04 Thread Daniel Shelepanov
Hi hackers!   I’ve been working on this [ https://www.postgresql.org/message-id/flat/cfcca574-6967-c5ab-7dc3-2c82b6723b99%40mail.ru ] bug. Finally, I’ve come up with the patch you can find attached. Basically what is does is rises a PROC_IN_VACUUM flag and resets it afterwards. I know this see

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-04 Thread Kyotaro Horiguchi
At Mon, 04 Apr 2022 17:29:48 +0900 (JST), Kyotaro Horiguchi wrote in > I haven't found how the patch caused creation of a relation file that > is to be removed soon. However, I find that v19 patch fails by maybe > due to some change in Cluster.pm. It takes a bit more time to check > that.. I

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2022-04-04 Thread Pavel Borisov
> > This patch was broken by d16773cdc86210493a2874cb0cf93f3883fcda73 "Add > macros in hash and btree AMs to get the special area of their pages" > > If it's really just a few macros it should be easy enough to merge but > it would be good to do a rebase given the number of other commits > since Fe

Re: Defer selection of asynchronous subplans until the executor initialization stage

2022-04-04 Thread Andrey V. Lepikhov
On 4/3/22 15:29, Etsuro Fujita wrote: On Sun, Mar 13, 2022 at 6:39 PM Etsuro Fujita wrote: On Wed, Sep 15, 2021 at 3:40 PM Alexander Pyhalov wrote: The patch looks good to me and seems to work as expected. I’m planning to commit the patch. I polished the patch a bit: * Reordered a bit of

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > On Mon, Apr 4, 2022 at 8:01 AM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 10:28:30AM +0900, Masahiko Sawada wrote: > > > > On Sun, Apr 3, 2022 at 9:45 AM Noah Misch wrote: >

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-04-04 Thread Dilip Kumar
On Sun, Apr 3, 2022 at 9:52 PM Andres Freund wrote: > > Hi, > > On 2022-03-29 11:55:05 -0400, Robert Haas wrote: > > I committed v6 instead. > > Coverity complains that this patch added GetDatabasePath() calls without > freeing its return value. Normally that'd be easy to dismiss, due to memory >

Re: Use generation context to speed up tuplesorts

2022-04-04 Thread David Rowley
On Sat, 2 Apr 2022 at 02:25, Justin Pryzby wrote: > > On Fri, Apr 01, 2022 at 10:00:08PM +1300, David Rowley wrote: > > 0002: > > This modifies the tuplesort API so that instead of having a > > randomAccess bool flag, > > Per cirrus, this missed updating one line for dtrace. Thanks. I've pushed

Re: logical decoding and replication of sequences

2022-04-04 Thread Amit Kapila
On Mon, Apr 4, 2022 at 11:42 AM Masahiko Sawada wrote: > > On Sat, Apr 2, 2022 at 8:52 PM Tomas Vondra > wrote: > > It's not related to this issue but I think that non-transactional > sequence changes could be resent in case of the subscriber crashes > because it doesn’t update replication origin

Re: Reword docs of feature "Remove temporary files after backend crash"

2022-04-04 Thread Daniel Gustafsson
> On 1 Apr 2022, at 15:57, Robert Haas wrote: > On Fri, Apr 1, 2022 at 9:42 AM Daniel Gustafsson wrote: >> This has been sitting the CF for quite some time, time to make a decision on >> it. I think it makes sense, having detailed docs around debugging is rarely >> a >> bad thing. Does anyone

Re: generic plans and "initial" pruning

2022-04-04 Thread Amit Langote
Thanks for the review. On Sun, Apr 3, 2022 at 8:33 PM Alvaro Herrera wrote: > I noticed a definitional problem in 0001 that's also a bug in some > conditions -- namely that the bitmapset "validplans" is never explicitly > initialized to NIL. In the original coding, the BMS was always returned >

Re: CLUSTER on partitioned index

2022-04-04 Thread Justin Pryzby
On Sat, Apr 02, 2022 at 07:11:47PM +0200, Alvaro Herrera wrote: > Thanks, pushed. Thank you for revisiting it, and thanks to Zhihong Yu for earlier review. I'll look into your outstanding questions later this week. -- Justin

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-04-04 Thread Robert Haas
On Sat, Apr 2, 2022 at 1:32 PM Joe Conway wrote: > I saw that Robert added documentation and it already reads correctly I > believe, except possibly an unrelated issue: > > 8<-- > >A role which replication whose privileges users are required to > possess >in orde

Re: pg_rewind copies

2022-04-04 Thread Peter Eisentraut
On 01.04.22 11:00, Daniel Gustafsson wrote: One small comment on the patch: + snprintf(srcpath, sizeof(srcpath), "%s/%s", datadir, path); This should IMO check the returnvalue of snprintf to ensure it wasn't truncated. While the risk is exceedingly small, a truncated filename might match ano

GSoC: pgmoneta, storage API

2022-04-04 Thread Mariam Fahmy
Hi, Hope you’re having a good day! I am Mariam Fahmy, A senior computer and systems engineering student at faculty of engineering, AinShams university. I am interested in working with pgmoneta during GSoC 2022. Here is a link to the draft proposal for implementing storage engine in pgmoneta: http

Re: shared-memory based stats collector - v68

2022-04-04 Thread Thomas Munro
On Mon, Apr 4, 2022 at 4:16 PM Andres Freund wrote: > Please take a look! A few superficial comments: > [PATCH v68 01/31] pgstat: consistent function header formatting. > [PATCH v68 02/31] pgstat: remove some superflous comments from pgstat.h. +1 > [PATCH v68 03/31] dshash: revise sequential s

Re: PostgreSQL shutdown modes

2022-04-04 Thread Robert Haas
On Sat, Apr 2, 2022 at 9:39 AM wrote: > I've waited for April 2nd to submit this comment, but it seemed to me > that the > suggestion about the first-pass checkpoint in 'slow' mode is a > no-foolin' good one. Yeah. While the patch itself is mostly in jest, everything I wrote in the email is unfor

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread Ashutosh Bapat
I didn't find this in https://commitfest.postgresql.org/37/. Is this somewhere in the commitfest? On Fri, Apr 1, 2022 at 12:46 PM vignesh C wrote: > > On Thu, Mar 31, 2022 at 2:52 PM Amit Kapila wrote: > > > > On Thu, Mar 31, 2022 at 9:14 AM Amit Kapila wrote: > > > > > > On Wed, Mar 30, 2022 a

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-04 Thread Fabien COELHO
Hello Tatsuo-san, interval_start num_transactions sum_latency sum_latency_2 min_latency max_latency sum_lag sum_lag_2 min_lag max_lag skipped failures serialization_failures deadlock_failures retried retries I would suggest to reorder the last chunk to: ... retried retries failures serf

Re: GSoC: pgmoneta, storage API

2022-04-04 Thread Jesper Pedersen
Hi Mariam, On 4/4/22 09:16, Mariam Fahmy wrote: Hope you’re having a good day! I am Mariam Fahmy, A senior computer and systems engineering student at faculty of engineering, AinShams university. I am interested in working with pgmoneta during GSoC 2022. Here is a link to the draft proposal fo

Re: use has_privs_of_role() for pg_hba.conf

2022-04-04 Thread Joshua Brindle
On Fri, Apr 1, 2022 at 6:06 PM Nathan Bossart wrote: > > Hi hackers, > > 6198420 ensured that has_privs_of_role() is used for predefined roles, > which means that the role inheritance hierarchy is checked instead of mere > role membership. However, inheritance is still not respected for > pg_hba.

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread David Steele
On 3/28/22 10:09 PM, Nathan Bossart wrote: On Mon, Mar 28, 2022 at 04:30:27PM -0400, Stephen Frost wrote: On a once-over of the rest of the code, I definitely like how much we're able to simplify things in this area and remove various hacks in things like pg_basebackup and pg_rewind where we pr

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Robert Haas
On Sun, Apr 3, 2022 at 10:10 PM Peter Geoghegan wrote: > I meant to tell the authors of verify_heapam() (also CC'd) that it > really helped with my recent VACUUM project. While the assertions that > I wrote in vacuumlazy.c might catch certain bugs like this, > verify_heapam() is much more effectiv

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-04 Thread Robert Haas
On Fri, Apr 1, 2022 at 5:03 PM Thomas Munro wrote: > Another idea would be to call a new function DropPendingWritebacks(), > and also tell all the SMgrRelation objects to close all their internal > state (ie the fds + per-segment objects) but not free the main > SMgrRelationData object, and for go

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread Nathan Bossart
On Mon, Apr 04, 2022 at 09:56:26AM -0400, David Steele wrote: > Minor typo in the docs: > > + * capable of doing an online backup, but exclude then just in case. > > Should be: > > capable of doing an online backup, but exclude them just in case. fixed -- Nathan Bossart Amazon Web Servic

Re: use has_privs_of_role() for pg_hba.conf

2022-04-04 Thread Nathan Bossart
On Mon, Apr 04, 2022 at 09:36:13AM -0400, Joshua Brindle wrote: > Good catch, I think this is a logical followup to the previous > has_privs_of_role patch. > > Reviewed and +1 Thanks! I created a commitfest entry for this: https://commitfest.postgresql.org/38/3609/ -- Nathan Bossart A

Re: basebackup/lz4 crash

2022-04-04 Thread Robert Haas
On Thu, Mar 31, 2022 at 3:19 AM Dipesh Pandit wrote: > Patch attached. Committed. Thanks. -- Robert Haas EDB: http://www.enterprisedb.com

Re: JSON constructors and window functions

2022-04-04 Thread Andrew Dunstan
On 4/3/22 22:46, Andrew Dunstan wrote: > On 4/3/22 20:11, Andres Freund wrote: >> Hi, >> >> On 2022-04-03 18:56:39 -0400, Andrew Dunstan wrote: >>> Haven't found the issue yet :-( It happens on the second call for the >>> partition to  json_check_unique_key(). >>> >>> Here's a more idiomatic and

Re: Add non-blocking version of PQcancel

2022-04-04 Thread Jelte Fennema
Hereby what I consider the final version of this patch. I don't have any changes planned myself (except for ones that come up during review). Things that changed since the previous iteration: 1. postgres_fdw now uses the non-blocking cancellation API (including test). 2. Added some extra sleeps to

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Tom Lane
Mark Dilger writes: > If we want to backtrack to v8, that's fine. I can rebase that, port > some of the other changes from v14 to it, and repost it as v15. Are you working on that? I've set aside time this week to hopefully get this over the finish line, but I don't want to find out that I've b

Re: PATCH: add "--config-file=" option to pg_rewind

2022-04-04 Thread Greg Stark
Just reposting the previous patches to straighten out the cfbot. --- /usr/lib/python3/dist-packages/patroni/postgresql/__init__.py 2022-02-18 13:16:15.0 + +++ __init__.py 2022-04-03 19:17:29.952665383 + @@ -798,7 +798,8 @@ return True def get_guc_value(self, name): -

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread Nathan Bossart
I noticed a couple of other things that can be removed. Since we no longer wait on exclusive backup mode during smart shutdown, we can change connsAllowed (in postmaster.c) to a boolean and remove CAC_SUPERUSER. We can also remove a couple of related notes in the documentation. I've done all thi

Re: JSON constructors and window functions

2022-04-04 Thread Tom Lane
Andrew Dunstan writes: > On 4/3/22 22:46, Andrew Dunstan wrote: >> On 4/3/22 20:11, Andres Freund wrote: >>> I don't think you're allowed to free stuff in a finalfunc - we might reuse >>> the >>> transition state for further calls to the aggregate. >> Doh! Of course! I'll fix it in the morning.

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-04-04 Thread Dilip Kumar
On Mon, Apr 4, 2022 at 2:25 PM Kyotaro Horiguchi wrote: > > At Mon, 04 Apr 2022 17:29:48 +0900 (JST), Kyotaro Horiguchi > wrote in > > I haven't found how the patch caused creation of a relation file that > > is to be removed soon. However, I find that v19 patch fails by maybe > > due to some c

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 8:36 AM, Tom Lane wrote: > > Mark Dilger writes: >> If we want to backtrack to v8, that's fine. I can rebase that, port >> some of the other changes from v14 to it, and repost it as v15. > > Are you working on that? I've set aside time this week to hopefully > get this

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Masahiko Sawada
On Mon, Apr 4, 2022 at 1:30 PM Julien Rouhaud wrote: > > Hi, > > On Tue, Mar 01, 2022 at 11:35:32AM +0900, Masahiko Sawada wrote: > > On Wed, Jan 19, 2022 at 5:52 PM Julien Rouhaud wrote: > > > > > > It seems that the regression tests aren't entirely stable, per cfbot: > > > https://cirrus-ci.com

Re: JSON constructors and window functions

2022-04-04 Thread Greg Stark
Are we missing regression tests using these functions as window functions? Hm. I suppose it's possible to write a general purpose regression test that loops over all aggregate functions and runs them as window functions and aggregates over the same data sets and compares results. At least for the

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Mar 21, 2022, at 10:03 PM, Thomas Munro wrote: > > On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger > wrote: >> (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl >> when testing v1-0001. I'm not sure yet what that is about.) > > Doesn't look like 0001 has anything to

Re: JSON constructors and window functions

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 11:43:31 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 4/3/22 22:46, Andrew Dunstan wrote: > >> On 4/3/22 20:11, Andres Freund wrote: > >>> I don't think you're allowed to free stuff in a finalfunc - we might > >>> reuse the > >>> transition state for further calls to

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Peter Geoghegan
On Mon, Apr 4, 2022 at 7:02 AM Robert Haas wrote: > Yeah, I was very excited about verify_heapam(). There is a lot more > stuff that we could check, but a lot of those things would be much > more expensive to check. If anything I understated the value of verify_heapam() with this kind of work bef

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 9:27 AM, Peter Geoghegan wrote: > > I'd really like it if amcheck had HOT chain verification. That's the > other area where catching bugs passively with assertions and whatnot > is clearly not good enough. I agree, and was hoping to get around to this in the postgres 15 d

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Julien Rouhaud
On Tue, Apr 05, 2022 at 12:51:12AM +0900, Masahiko Sawada wrote: > On Mon, Apr 4, 2022 at 1:30 PM Julien Rouhaud wrote: > > > > Hmm, but AFAICS the json format would be stable as the counters are always > > shown even if zero. So just doing the json format first and then the text > > format shoul

Re: JSON constructors and window functions

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 11:54:23 -0400, Greg Stark wrote: > Are we missing regression tests using these functions as window functions? So far, yes. ISTM that 4eb97988796 should have at least included the crashing statement as a test... The statement can be simpler too: SELECT json_objectagg(k : v wit

Re: Pluggable toaster

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 12:13 PM Nikita Malakhov wrote: > I'm really sorry. Messed up some code while merging rebased branches with > previous (v1) > patches issued in December and haven't noticed that seg fault because of > corrupted code > while running check-world. > I've fixed messed code in

Re: A test for replay of regression tests

2022-04-04 Thread Tom Lane
I wrote: > I think the fundamental instability here is that this TAP test is > setting shared_buffers small enough to allow the syncscan logic > to kick in where it does not in normal testing. Maybe we should > just disable syncscan in this test script? Did that, we'll see how much it helps.

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > I just got a crash in this test again. Are you still interested? I still > have the logs. No core file appears to have been generated. > The test failure is > not ok 5 - psql query died successfully after SIGQUIT Hmm ... I can see one problem with that test: ok( pump_un

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-04-04 Thread David Steele
On 4/4/22 11:42 AM, Nathan Bossart wrote: I noticed a couple of other things that can be removed. Since we no longer wait on exclusive backup mode during smart shutdown, we can change connsAllowed (in postmaster.c) to a boolean and remove CAC_SUPERUSER. We can also remove a couple of related no

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
I wrote: > Usually the test would succeed anyway because of matching the > second or third regex alternative, but I wonder if there is > some other spelling of libpq's complaint that shows up > occasionally. It'd be nice if we could see the contents of > $killme_stderr upon failure. OK, now I'm c

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 10:41 AM, Tom Lane wrote: > > Is our CI setup failing to capture stderr from TAP tests?? I'm looking into the way our TAP test infrastructure assigns port numbers to nodes, and whether that is reasonable during parallel test runs with nodes stopping and starting again.

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
I wrote: > Is our CI setup failing to capture stderr from TAP tests?? Oh, I'm barking up the wrong tree. This test must have been run against HEAD between 6da65a3f9 (23 Feb) and 2beb4acff (31 Mar), when pump_until indeed didn't print this essential information :-( If you just got this failure, c

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-05 01:16:04 +1200, Thomas Munro wrote: > On Mon, Apr 4, 2022 at 4:16 PM Andres Freund wrote: > > Please take a look! > > A few superficial comments: > > > [PATCH v68 01/31] pgstat: consistent function header formatting. > > [PATCH v68 02/31] pgstat: remove some superflous comment

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 10:51 AM, Tom Lane wrote: > > Oh, I'm barking up the wrong tree. This test must have been run > against HEAD between 6da65a3f9 (23 Feb) and 2beb4acff (31 Mar), when > pump_until indeed didn't print this essential information :-( > > If you just got this failure, could you

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > The test logs are attached. Server log looks as-expected: 2022-04-04 09:11:51.087 PDT [2084] 013_crash_restart.pl LOG: statement: SELECT pg_sleep(3600); 2022-04-04 09:11:51.094 PDT [2083] 013_crash_restart.pl WARNING: terminating connection because of unexpected SIGQUIT

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 11:07 AM, Tom Lane wrote: > > I was hoping to see regress_log_013_crash_restart, though. regress_log_013_crash_restart Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 10:02:37 -0400, Robert Haas wrote: > It does a good job, I think, checking all the things that a human being > could potentially spot just by looking at an individual page. I think there's a few more things that'd be good to check. For example amcheck doesn't verify that HOT cha

Re: JSON constructors and window functions

2022-04-04 Thread Andrew Dunstan
On 4/4/22 12:33, Andres Freund wrote: > Hi, > > On 2022-04-04 11:54:23 -0400, Greg Stark wrote: >> Are we missing regression tests using these functions as window functions? > So far, yes. > > ISTM that 4eb97988796 should have at least included the crashing statement as > a test... The statement

Re: JSON constructors and window functions

2022-04-04 Thread Andrew Dunstan
On 4/4/22 11:43, Tom Lane wrote: > Andrew Dunstan writes: >> On 4/3/22 22:46, Andrew Dunstan wrote: >>> On 4/3/22 20:11, Andres Freund wrote: I don't think you're allowed to free stuff in a finalfunc - we might reuse the transition state for further calls to the aggregate. >>> Do

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 2:16 PM Andres Freund wrote: > On 2022-04-04 10:02:37 -0400, Robert Haas wrote: > > It does a good job, I think, checking all the things that a human being > > could potentially spot just by looking at an individual page. > > I think there's a few more things that'd be good

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 10:44 AM, Mark Dilger wrote: > > I'm writing a parallel test just for this. Will get back to you. Ok, that experiment didn't accomplish anything, beyond refreshing my memory regarding Cluster.pm preferring sockets over ports. — Mark Dilger EnterpriseDB: http://www.ente

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: > # Running: pg_ctl kill QUIT 2083 > ok 4 - killed process with SIGQUIT > # pump_until: process terminated unexpectedly when searching for > "(?^m:WARNING: terminating connection because of crash of another server > process|server closed the connection unexpectedly|connecti

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
I wrote: > The "terminating connection" warning absolutely should get through, ... oh, no, that's not guaranteed at all, since it's sent from quickdie(). So scratch that. Maybe we'd better add "could not send data to server" to the regex? regards, tom lane

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-03 21:15:16 -0700, Andres Freund wrote: > - collect who reviewed earlier revisions I found reviews by - Tomas Vondra - Arthur Zakirov - Antonin Houska There's also reviews by Fujii and Alvaro, but afaics just for parts that were separately committed. Greetings, Andres Freund

Re: Pluggable toaster

2022-04-04 Thread Greg Stark
Thanks for the review Robert. I think that gives some pretty actionable advice on how to improve the patch and it doesn't seem likely to get much more in this cycle. I'll mark the patch Returned with Feedback. Hope to see it come back with improvements in the next release.

Re: Higher level questions around shared memory stats

2022-04-04 Thread Andres Freund
Hi, On 2022-03-30 14:08:41 -0700, Andres Freund wrote: > On 2022-03-30 14:42:23 -0400, Robert Haas wrote: > > On Tue, Mar 29, 2022 at 5:01 PM Andres Freund wrote: > > > I can think of these different times: > > > > > > - Last time stats were removed due to starting up in crash recovery > > > - La

pushdown of joinquals beyond group by/distinct on

2022-04-04 Thread Arne Roland
Hi, can someone point out to me, why we don't consider pushdowns of the joinqual for these queries beyond the distinct on? When the qual matches the distinct clause, it should be possible to generate both parametrized and non parametrized subplans for the same query. The same should hold true

Re: Pluggable toaster

2022-04-04 Thread Nikita Malakhov
Hi, Thank you very much for your review, I'd like to get it much earlier. I'm currently working on cleaning up code, and would try to comment as much as possible. This patch set is really a large set of functionality, it was divided into 4 logically complete parts, but anyway these parts contain a

Re: Pluggable toaster

2022-04-04 Thread Robert Haas
On Mon, Apr 4, 2022 at 4:05 PM Nikita Malakhov wrote: > - Is 'git apply' not a valid way to apply such patches? I have found that it never works. This case is no exception: [rhaas pgsql]$ git apply ~/Downloads/1_toaster_interface_v4.patch /Users/rhaas/Downloads/1_toaster_interface_v4.patch:253:

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 12:05 PM, Tom Lane wrote: > > I wrote: >> The "terminating connection" warning absolutely should get through, > > ... oh, no, that's not guaranteed at all, since it's sent from quickdie(). > So scratch that. Maybe we'd better add "could not send data to server" > to the

Re: Mingw task for Cirrus CI

2022-04-04 Thread Andres Freund
Hi, On 2022-03-30 17:26:18 -0700, Andres Freund wrote: > Hi, > > On 2022-03-22 19:00:42 +0300, Melih Mutlu wrote: > > Rebased it. > > I also removed the temp installation task and > > used NoDefaultCurrentDirectoryInExePath env variable instead. > > Hm. But you're still using a separate build dire

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Sun, Apr 3, 2022 at 9:16 PM Andres Freund wrote: > > Please take a look! > > I didn't take the time to fixup all the various odd typos in the general code comments; none of them reduced comprehension appreciably. I may do so when/if I do another pass. I did skim over the entire patch set and

Re: New Object Access Type hooks

2022-04-04 Thread Tom Lane
Mark Dilger writes: >> On Apr 4, 2022, at 12:05 PM, Tom Lane wrote: >> So scratch that. Maybe we'd better add "could not send data to server" >> to the regex? > If it fails in pqsecure_raw_write(), you get either "server closed the > connection unexpectedly" or "could not send data to server".

Re: New Object Access Type hooks

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 1:47 PM, Tom Lane wrote: > > Yeah, it's plausible to get a failure on either the write or read side > depending on timing. > > Perhaps libpq should be trying harder to make those cases look alike, but > this test is about server behavior not libpq behavior, so I'm incline

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 13:45:40 -0700, David G. Johnston wrote: > I didn't take the time to fixup all the various odd typos in the general > code comments; none of them reduced comprehension appreciably. I may do so > when/if I do another pass. Cool. > My first encounter with pg_stat_exists_stat()

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 2:06 PM Andres Freund wrote: > > > My first encounter with pg_stat_exists_stat() didn't draw my attention as > > being problematic so I'd say we just stick with it. As a SQL user > reading: > > WHERE exists (...) is somewhat natural; using "have" or back-to-back > > stat_s

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Tom Lane
Mark Dilger writes: > [ v15 patch ] Thanks. As I'm working through this, I'm kind of inclined to drop the has_parameter_privilege() variants that take an OID as object identifier. This gets back to the fact that I don't think pg_parameter_acl OIDs have any outside use; we wouldn't even have the

Re: psql - add SHOW_ALL_RESULTS option

2022-04-04 Thread Peter Eisentraut
On 02.04.22 15:26, Fabien COELHO wrote: Again, after the SendQuery refactoring extraction. I'm doing this locally, so don't feel obliged to send more of these. ;-) Good for me :-) This has been committed. I reduced some of your stylistic changes in order to keep the surface area of this

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 14:25:57 -0700, David G. Johnston wrote: > > You mentioned this as a restriction above - I'm not seeing it as such? I'd > > like to write out stats more often in the future (e.g. in the > > checkpointer), > > but then it'd not be written out with this function... > > > > > Yeah,

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Andrew Dunstan
On 4/4/22 17:26, Tom Lane wrote: > Mark Dilger writes: >> [ v15 patch ] > Thanks. As I'm working through this, I'm kind of inclined to drop > the has_parameter_privilege() variants that take an OID as object > identifier. This gets back to the fact that I don't think > pg_parameter_acl OIDs ha

Re: shared-memory based stats collector - v68

2022-04-04 Thread David G. Johnston
On Mon, Apr 4, 2022 at 2:54 PM Andres Freund wrote: > > > As the existing function only handles functions and relations why not > just > > perform a specific Kind check for them? Generalizing to assert on > whether > > or not the function works on fixed or variable Kinds seems beyond its > > pre

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-04 Thread Thomas Munro
On Tue, Apr 5, 2022 at 2:18 AM Robert Haas wrote: > On Fri, Apr 1, 2022 at 5:03 PM Thomas Munro wrote: > > Another idea would be to call a new function DropPendingWritebacks(), > > and also tell all the SMgrRelation objects to close all their internal > > state (ie the fds + per-segment objects)

Re: shared-memory based stats collector - v68

2022-04-04 Thread Andres Freund
Hi, On 2022-04-04 15:24:24 -0700, David G. Johnston wrote: > Replacing the existing assert(!kind->fixed_amount) with > assert(!kind->accessed_across_databases) produces the same result as the > later presently implies the former. I wasn't proposing to replace, but to add... > Now I start to dis

Re: pushdown of joinquals beyond group by/distinct on

2022-04-04 Thread David Rowley
On Tue, 5 Apr 2022 at 07:40, Arne Roland wrote: > can someone point out to me, why we don't consider pushdowns of the joinqual > for these queries beyond the distinct on? > > When the qual matches the distinct clause, it should be possible to generate > both parametrized and non parametrized sub

Re: shared-memory based stats collector - v67

2022-04-04 Thread Andres Freund
Hi, On 2022-03-23 10:42:03 -0700, Andres Freund wrote: > On 2022-03-23 17:27:50 +0900, Kyotaro Horiguchi wrote: > > + /* > > +* When starting with crash recovery, reset pgstat data - it might not > > be > > +* valid. Otherwise restore pgstat data. It's safe to do this here, > > +* b

Re: [HACKERS] WIP aPatch: Pgbench Serialization and deadlock errors

2022-04-04 Thread Tatsuo Ishii
Hi Fabien, > Hello Tatsuo-san, > >> interval_start num_transactions sum_latency sum_latency_2 min_latency >> max_latency >> sum_lag sum_lag_2 min_lag max_lag skipped >> failures serialization_failures deadlock_failures retried retries > > I would suggest to reorder the last chunk to: > >...

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Michael Paquier
On Sun, Apr 03, 2022 at 11:53:03AM -0700, Andres Freund wrote: > It seems $subject would have a chance of catching some of these bugs, as well > as exposing amcheck to a database with a bit more varied content? Makes sense to me to extend that. > Depending on the cost it might make sense to do th

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 2:26 PM, Tom Lane wrote: > > Thanks. As I'm working through this, I'm kind of inclined to drop > the has_parameter_privilege() variants that take an OID as object > identifier. This gets back to the fact that I don't think > pg_parameter_acl OIDs have any outside use; we

Re: Window Function "Run Conditions"

2022-04-04 Thread David Rowley
Thanks for having a look at this. On Wed, 30 Mar 2022 at 11:16, Andres Freund wrote: > On 2022-03-29 15:11:52 +1300, David Rowley wrote: > > One thing which I'm not sure about with the patch is how I'm handling > > the evaluation of the runcondition in nodeWindowAgg.c. Instead of > > having Exec

Re: Extensible Rmgr for Table AMs

2022-04-04 Thread Jeff Davis
On Sun, 2022-04-03 at 21:33 -0700, Andres Freund wrote: > I still think the easiest and fastest would be to just make RmgrTable > longer, > and not const. When registering, copy the caller provided struct into > the > respective RmgrData element. Yes, we'd waste a bit of space at the > end of the

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Tom Lane
Mark Dilger writes: > On Apr 4, 2022, at 2:26 PM, Tom Lane wrote: >> So I think that instead of what you've got here, we should >> (1) apply the map_old_guc_names[] mapping, which is constant >> (for any one PG release anyway) >> (2) smash to lower case >> (3) verify validity per valid_variable_n

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-04-04 Thread Mark Dilger
> On Apr 4, 2022, at 5:12 PM, Tom Lane wrote: > > Wrote it already, no need for you to do it. Thanks! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Noah Misch
On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > > How about a comment like: "It has to be kept at 8-byte alignment > > > boundary so as to be accessed directly v

Re: Extensible Rmgr for Table AMs

2022-04-04 Thread Jeff Davis
On Mon, 2022-04-04 at 11:15 +0530, Bharath Rupireddy wrote: > 1) Why can't rmid be chosen by the extensions in sequential order > from > (129 till 255), say, to start with a columnar extension can choose > 129, another extension can register 130 and so on right? I'm not sure what you mean by "chos

Re: Run pg_amcheck in 002_pg_upgrade.pl and 027_stream_regress.pl?

2022-04-04 Thread Andres Freund
Hi, On 2022-04-05 08:46:06 +0900, Michael Paquier wrote: > On Sun, Apr 03, 2022 at 11:53:03AM -0700, Andres Freund wrote: > > It seems $subject would have a chance of catching some of these bugs, as > > well > > as exposing amcheck to a database with a bit more varied content? > > Makes sense to

Re: Handle infinite recursion in logical replication setup

2022-04-04 Thread Peter Smith
Here are my comments for the latest patch v6-0001. (I will post my v6-0002 review comments separately) PATCH v6-0001 comments == 1.1 General - Option name I still feel like the option name is not ideal. Unfortunately, this is important because any name change would impact lo

Re: Skipping logical replication transactions on subscriber side

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 9:21 AM Noah Misch wrote: > > On Mon, Apr 04, 2022 at 06:55:45PM +0900, Masahiko Sawada wrote: > > On Mon, Apr 4, 2022 at 3:26 PM Noah Misch wrote: > > > On Mon, Apr 04, 2022 at 08:20:08AM +0530, Amit Kapila wrote: > > > > How about a comment like: "It has to be kept at 8-b

Re: Add LZ4 compression in pg_dump

2022-04-04 Thread Michael Paquier
On Fri, Apr 01, 2022 at 03:06:40PM +, gkokola...@pm.me wrote: > I understand the itch. Indeed when LZ4 is added as compression method, this > block changes slightly. I went with the minimum amount changed. Please find > in 0001 of the attached this variable renamed as $gzip_program_exist. I >

Re: Showing I/O timings spent reading/writing temp buffers in EXPLAIN

2022-04-04 Thread Masahiko Sawada
On Tue, Apr 5, 2022 at 1:31 AM Julien Rouhaud wrote: > > On Tue, Apr 05, 2022 at 12:51:12AM +0900, Masahiko Sawada wrote: > > On Mon, Apr 4, 2022 at 1:30 PM Julien Rouhaud wrote: > > > > > > Hmm, but AFAICS the json format would be stable as the counters are always > > > shown even if zero. So j

  1   2   >