Re: strange case of "if ((a & b))"

2021-10-06 Thread Masahiko Sawada
On Thu, Oct 7, 2021 at 1:37 PM Michael Paquier wrote: > > On Thu, Oct 07, 2021 at 01:27:34PM +0900, Masahiko Sawada wrote: > > On Thu, Oct 7, 2021 at 11:44 AM Justin Pryzby wrote: > >> I'm not quite sure how you managed to search for it - well done ;) > > > > I could not find the recent thread, t

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-10-06 Thread Amit Kapila
On Wed, Oct 6, 2021 at 12:41 PM Masahiko Sawada wrote: > > Hi all, > > A customer reported that during parallel index vacuum, the oldest xmin > doesn't advance. Normally, the calculation of oldest xmin > (ComputeXidHorizons()) ignores xmin/xid of processes having > PROC_IN_VACUUM flag in MyProc->s

Re: Delegating superuser tasks to new security roles (Was: Granting control of SUSET gucs to non-superusers)

2021-10-06 Thread Bharath Rupireddy
On Mon, Sep 27, 2021 at 11:45 PM Mark Dilger wrote: > > Thanks for looking! I have pulled together a new patch set which applies > cleanly against current master. Hi Mark, thanks for this work. I'm late to be here in this thread, please note that I didn't go through the entire thread as it is q

Re: pgsql: Document XLOG_INCLUDE_XID a little better

2021-10-06 Thread Amit Kapila
On Sat, Oct 2, 2021 at 4:16 PM Dilip Kumar wrote: > > On Fri, Oct 1, 2021 at 6:24 PM Alvaro Herrera wrote: > > > > On 2021-Oct-01, Amit Kapila wrote: > > > I think a straight standalone variable (probably a static boolean in > > xloginsert.c) might be less confusing. > > I have written two patche

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2021-10-06 Thread Bharath Rupireddy
On Wed, Oct 6, 2021 at 10:26 PM Jeremy Schneider wrote: > > On 10/5/21 17:43, Bruce Momjian wrote: > > On Tue, Oct 5, 2021 at 03:30:07PM -0700, Jeremy Schneider wrote: > >> Specifically exposing pg_waldump functionality in SQL could speed up > >> finding bugs in the PG logical replication code. W

Re: strange case of "if ((a & b))"

2021-10-06 Thread Michael Paquier
On Thu, Oct 07, 2021 at 01:27:34PM +0900, Masahiko Sawada wrote: > On Thu, Oct 7, 2021 at 11:44 AM Justin Pryzby wrote: >> I'm not quite sure how you managed to search for it - well done ;) > > I could not find the recent thread, though :) Hm. It looks like there are more occurences of "false :

RE: Transactions involving multiple postgres foreign servers, take 2

2021-10-06 Thread k.jami...@fujitsu.com
Hi Fujii-san and Sawada-san, Thank you very much for your replies. > >> I noticed that this thread and its set of patches have been marked with > "Returned with Feedback" by yourself. > >> I find the feature (atomic commit for foreign transactions) very > >> useful and it will pave the road for h

Re: strange case of "if ((a & b))"

2021-10-06 Thread Masahiko Sawada
On Thu, Oct 7, 2021 at 11:44 AM Justin Pryzby wrote: > > On Thu, Oct 07, 2021 at 11:18:24AM +0900, Masahiko Sawada wrote: > > On Sat, Sep 11, 2021 at 2:44 PM Michael Paquier wrote: > > > > > > On Thu, Sep 09, 2021 at 02:14:50PM +0900, Kyotaro Horiguchi wrote: > > > > Maybe I'm missing something,

Re: Lost logs with csvlog redirected to stderr under WIN32 service

2021-10-06 Thread Michael Paquier
On Wed, Oct 06, 2021 at 09:33:24PM -0500, Chris Bandy wrote: > I don't have a windows machine to test, but this refactor looks good to me. Thanks for the review! I did test this on Windows, only MSVC builds. >> +/* Write to CSV log, if enabled */ >> +if ((Log_destination & LOG_DESTINATIO

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2021-10-06 Thread Masahiko Sawada
On Thu, Sep 23, 2021 at 5:44 PM Drouvot, Bertrand wrote: > > Hi, > > On 7/29/21 10:25 AM, Masahiko Sawada wrote: > > Thank you for reporting the issue. > > > > I could reproduce this issue by the steps you shared. > > Thanks for looking at it! > > > > >> Currently, the system relies on processing

Question about client_connection_check_interval

2021-10-06 Thread kuroda.hay...@fujitsu.com
Dear Hackers, While reading source codes about timeouts and GUC and I found that strange behavior about client_connection_check_interval. Currently we did not an assign_hook about client_connection_check_interval, that means a timeout will not turn on immediately if users change the GUC from zero

Re: a comment in joinrel.c: compute_partition_bounds()

2021-10-06 Thread Amit Langote
Fujita-san, On Wed, Oct 6, 2021 at 5:41 PM Etsuro Fujita wrote: > On Fri, Sep 24, 2021 at 4:20 PM Etsuro Fujita wrote: > > On Fri, Sep 24, 2021 at 3:34 PM Amit Langote > > wrote: > > > I think there's a word missing in the following comment: > > > > > > /* > > > * See if the p

RE: Allow escape in application_name

2021-10-06 Thread kuroda.hay...@fujitsu.com
Dear Fujii-san, Thank you for reviewing! > + else if (*p == '-' ? isdigit(p[1]) : isdigit(p[0])) > + { > + char *endptr; > + padding = strtol(p, &endptr, 10); > > Maybe isdigit() and strtol() work differently depending on locale set

Re: strange case of "if ((a & b))"

2021-10-06 Thread Justin Pryzby
On Thu, Oct 07, 2021 at 11:18:24AM +0900, Masahiko Sawada wrote: > On Sat, Sep 11, 2021 at 2:44 PM Michael Paquier wrote: > > > > On Thu, Sep 09, 2021 at 02:14:50PM +0900, Kyotaro Horiguchi wrote: > > > Maybe I'm missing something, but I can see several instances of the > > > "eval-bool ? true : f

Re: Lost logs with csvlog redirected to stderr under WIN32 service

2021-10-06 Thread Chris Bandy
On 10/6/21 12:10 AM, Michael Paquier wrote: > I have thought about various ways to > fix that, and finished with a solution where we handle csvlog first, > and fallback to stderr after so as there is only one code path for > stderr, as of the attached. This reduces a bit the confusion around > the

Re: strange case of "if ((a & b))"

2021-10-06 Thread Masahiko Sawada
On Sat, Sep 11, 2021 at 2:44 PM Michael Paquier wrote: > > On Thu, Sep 09, 2021 at 02:14:50PM +0900, Kyotaro Horiguchi wrote: > > Maybe I'm missing something, but I can see several instances of the > > "eval-bool ? true : false" pattern after fd0625c7a9 that are not in > > the latest 0002. > > Yep

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Andres Freund
Hi, On 2021-10-06 09:47:54 -0700, Andres Freund wrote: > I'll also try to figure out print a bit more detail about timing for each tap > test, looks like I need to figure out how to pass PROVE_TEST='--timer' through > the buildfarm. Shouldn't be too hard. Turns out that the buildfarm already adds

Re: wrapping CF 2021-09

2021-10-06 Thread Michael Paquier
On Wed, Oct 06, 2021 at 10:55:53AM +0200, Daniel Gustafsson wrote: > Indeed, thanks for all your work this CF! +1. Thanks, Jaime! -- Michael signature.asc Description: PGP signature

Re: ALTER INDEX .. RENAME allows to rename tables/views as well

2021-10-06 Thread Michael Paquier
On Wed, Oct 06, 2021 at 06:43:25PM -0400, Tom Lane wrote: > ... as, no doubt, are a lot of applications that this will gratuitously > break. We've long had a policy that ALTER TABLE will work on relations > that aren't tables, so long as the requested operation is sensible. Yeah, that was my firs

Re: More business with $Test::Builder::Level in the TAP tests

2021-10-06 Thread Michael Paquier
On Wed, Oct 06, 2021 at 07:33:22AM -0400, Andrew Dunstan wrote: > We should probably state a requirement for this somewhere. Maybe in > src/test/perl/README. AIUI, the general rule is that any subroutine that > directly or indirectly calls ok() and friends should increase the level. > Such subrouti

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 2:28 PM Pavel Borisov wrote: >> It is the most consistent with the general design of the system, for >> reasons that are pretty deeply baked into the system. I'm reminded of >> the fact that REINDEX CONCURRENTLY's completion became blocked due to >> similar trepidations. Und

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2021-10-06 Thread Fujii Masao
On 2021/10/07 4:58, Jelte Fennema wrote: Ugh forgot to attach the patch. Here it is. Thanks for working on this patch! @@ -4546,10 +4684,21 @@ PQrequestCancel(PGconn *conn) return false; } - - r = internal_cancel(&conn->raddr, conn->be_pid, conn->be_key, Since PQreques

Re: [Patch] ALTER SYSTEM READ ONLY

2021-10-06 Thread Jaime Casanova
On Tue, Oct 05, 2021 at 04:11:58PM +0530, Amul Sul wrote: >On Mon, Oct 4, 2021 at 1:57 PM Rushabh Lathia > wrote: > > > > I tried to apply the patch on the master branch head and it's failing > > with conflicts. > > > > Thanks, Rushabh, for the quick check, I have attached a rebased version f

Re: ALTER INDEX .. RENAME allows to rename tables/views as well

2021-10-06 Thread Alvaro Herrera
On 2021-Oct-06, Bossart, Nathan wrote: > On 10/6/21, 3:44 PM, "Tom Lane" wrote: > > The situation for "ALTER some-other-relation-kind" is a bit more > > confused, because some cases throw errors and some don't; but I really > > doubt that tightening things up here will earn you anything but > >

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 3:47 PM Mark Dilger wrote: > I totally agree that the amcheck functions cannot prove the absence of > corruption. > > I prefer not to even use language about proving the presence of corruption > when discussing pg_amcheck. I agree that it doesn't usually help. But sometim

Re: Role Self-Administration

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 1:48 PM, Stephen Frost wrote: > > This specific syntax, including the CASCADE bit, has, at minimum, at least > been contemplate by the SQL folks sufficiently to be described in one > specific way. I don’t have a copy of 2016 handy, unfortunately, and so I’m > not sure

Re: plperl: update ppport.h and fix configure version check

2021-10-06 Thread Andrew Dunstan
On 10/5/21 10:30 AM, Tom Lane wrote: > Andrew Dunstan writes: >> Seems OK. Note that the Msys DTK perl currawong uses to build with is >> ancient (5.6.1). That's going to stay as it is until it goes completely >> out of scope in about 13 months. The perl it builds plperl against is >> much more

Re: ALTER INDEX .. RENAME allows to rename tables/views as well

2021-10-06 Thread Bossart, Nathan
On 10/6/21, 3:44 PM, "Tom Lane" wrote: > "Bossart, Nathan" writes: >> Here's a patch that ERRORs if the object type and statement type do >> not match. Interestingly, some of the regression tests were relying >> on this behavior. > > ... as, no doubt, are a lot of applications that this will gra

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 3:20 PM, Peter Geoghegan wrote: > >> I think the disagreements are about something else. > > Informally speaking, you could say that pg_amcheck and amcheck verify > relations. More formally speaking, both amcheck (whether called by > pg_amcheck or some other thing) can on

Re: ALTER INDEX .. RENAME allows to rename tables/views as well

2021-10-06 Thread Tom Lane
"Bossart, Nathan" writes: > On 10/6/21, 1:52 PM, "Bruce Momjian" wrote: >> I can confirm this bug in git head, and I think it should be fixed. > Here's a patch that ERRORs if the object type and statement type do > not match. Interestingly, some of the regression tests were relying > on this be

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 2:45 PM Mark Dilger wrote: > I think the disagreements are about something else. Informally speaking, you could say that pg_amcheck and amcheck verify relations. More formally speaking, both amcheck (whether called by pg_amcheck or some other thing) can only prove the prese

Re: using extended statistics to improve join estimates

2021-10-06 Thread Tomas Vondra
On 10/6/21 23:03, Zhihong Yu wrote: Hi, +       conditions2 = statext_determine_join_restrictions(root, rel, mcv); + +       /* if the new statistics covers more conditions, use it */ +       if (list_length(conditions2) > list_length(conditions1)) +       { +           mcv = stat; It seems con

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 2:45 PM, Mark Dilger wrote: > > and db3 is in recovery. > they're scattered across different databases, some in recovery, some not. What I mean here is that, since pg_amcheck might run for many hours, and database may start in recovery but then exit recovery, or may b

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 1:49 PM, Peter Geoghegan wrote: > >> The analogy here is: are we trying to verify that the relations are >> valid? Or are we just trying to verify that they are as valid as we >> can expect them to be? > > I think that we do the latter (or something much closer to the lat

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Pavel Borisov
> > > It is the most consistent with the general design of the system, for > reasons that are pretty deeply baked into the system. I'm reminded of > the fact that REINDEX CONCURRENTLY's completion became blocked due to > similar trepidations. Understandably so. I may mistake, but I recall the fac

Re: using extended statistics to improve join estimates

2021-10-06 Thread Zhihong Yu
On Wed, Oct 6, 2021 at 12:33 PM Tomas Vondra wrote: > Hi, > > attached is an improved version of this patch, addressing some of the > points mentioned in my last message: > > 1) Adds a couple regression tests, testing various join cases with > expressions, additional conditions, etc. > > 2) Adds

Re: ALTER INDEX .. RENAME allows to rename tables/views as well

2021-10-06 Thread Bruce Momjian
I can confirm this bug in git head, and I think it should be fixed. --- On Mon, Oct 4, 2021 at 10:23:23AM +, Onder Kalaci wrote: > Hi hackers, > > I realized a subtle behavior with ALTER INDEX .. RENAME. It seems like

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 1:15 PM Robert Haas wrote: > > Where I might go further than you or Mark (not sure) is on this: I > > also think that it's the caller's job to not call the functions with > > temp relations, or (in the case of the index verification stuff) with > > !indisready or !indisvalid

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, On Wed, Oct 6, 2021 at 16:28 Robert Haas wrote: > On Wed, Oct 6, 2021 at 3:29 PM Stephen Frost wrote: > > Does that mean that we also get to change what a specific set of > > commands, which are all well-defined in the standard, do even when that > > goes against what an SQL complian

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, On Wed, Oct 6, 2021 at 16:01 Mark Dilger wrote: > > On Oct 6, 2021, at 11:48 AM, Stephen Frost wrote: > > > > In the spec, under , there is a 'General Rules' > > section (as there is with most statements) and in that section it says > > that for every authorization identifier (that i

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2021-10-06 Thread Robert Haas
On Mon, Oct 4, 2021 at 12:44 PM Shruthi Gowda wrote: > Thanks for the inputs, Robert. In the v4 patch, an unused OID (i.e, 4) > is fixed for the template0 and the same is removed from unused oid > list. > > In addition to the review comment fixes, I have removed some code that > is no longer neede

Re: parallelizing the archiver

2021-10-06 Thread Magnus Hagander
On Tue, Oct 5, 2021 at 5:32 AM Bossart, Nathan wrote: > On 10/4/21, 8:19 PM, "Stephen Frost" wrote: > > It's also been discussed, at least around the water cooler (as it were > > in pandemic times- aka our internal slack channels..) that the existing > > archive command might be reimplemented as

Re: Role Self-Administration

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 3:29 PM Stephen Frost wrote: > Does that mean that we also get to change what a specific set of > commands, which are all well-defined in the standard, do even when that > goes against what an SQL compliant implementation would do? I really > don't think so. If this was *n

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 3:56 PM Peter Geoghegan wrote: > I agree, with the stipulation that the caller (in this case > pg_amcheck) is required to know certain basic things about the > relation in order to get useful behavior. For example, if you use > bt_index_check() with a GIN index, you're going

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 12:36 PM Mark Dilger wrote: > The user may not know that the system has changed. > > For example, if I see errors in the logs suggesting corruption in a relation > named "mark" and run pg_amcheck --relation=mark, I expect that to check the > relation. If that relation is

Re: storing an explicit nonce

2021-10-06 Thread Bruce Momjian
On Wed, Oct 6, 2021 at 03:17:00PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Tue, Oct 5, 2021 at 04:29:25PM -0400, Bruce Momjian wrote: > > > On Tue, Sep 28, 2021 at 12:30:02PM +0300, Ants Aasma wrote: > > > > On Mon, 27 Sept 2021 at 23:34, Bruc

Re: Role Self-Administration

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 11:48 AM, Stephen Frost wrote: > > In the spec, under , there is a 'General Rules' > section (as there is with most statements) and in that section it says > that for every authorization identifier (that is, some privilege, be it > a GRANT of SELECT rights on an object, or

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2021-10-06 Thread Jelte Fennema
Ugh forgot to attach the patch. Here it is. From: Jelte Fennema Sent: Wednesday, October 6, 2021 21:56 To: Zhihong Yu Cc: pgsql-hack...@postgresql.org Subject: Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings We

Re: [EXTERNAL] Re: PQcancel does not use tcp_user_timeout, connect_timeout and keepalive settings

2021-10-06 Thread Jelte Fennema
We actually ran into an issue caused by this in production, where a PQcancel connection was open on the client for a 2+ days because the server had restarted at the wrong moment in the cancel handshake. The client was now indefinitely waiting for the server to send an EOF back, and because keepa

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 12:33 PM Robert Haas wrote: > To me, it doesn't matter which specific option we're talking about. If > I tell pg_amcheck to pass a certain flag to the underlying functions, > then it should do that. If the behavior needs to be changed, it should > be changed in those underly

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 12:28 PM, Peter Geoghegan wrote: > > I think that what I've said boils down to this: > > * pg_amcheck shouldn't attempt to verify temp relations, on the > grounds that this is fundamentally not useful, and not something that > could ever be sensibly interpreted as "just d

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 2:56 PM Peter Geoghegan wrote: > --heapallindexed doesn't complicate things for us at all. It changes > nothing about the locking considerations. It's just an additive thing, > some extra checks with the same basic underlying requirements. Maybe > you meant to say --parent-c

Re: using extended statistics to improve join estimates

2021-10-06 Thread Tomas Vondra
Hi, attached is an improved version of this patch, addressing some of the points mentioned in my last message: 1) Adds a couple regression tests, testing various join cases with expressions, additional conditions, etc. 2) Adds support for expressions, so the join clauses don't need to reference

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 6, 2021 at 2:48 PM Stephen Frost wrote: > > What I'm saying above is that the command explicitly listed there > > *isn't* 'DROP ROLE A DB', even though that is something which the spec > > *could* have done, had they wished to.

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 11:55 AM Peter Geoghegan wrote: > I am pretty sure that I agree with you about all these details. We > need to tease them apart some more. I think that what I've said boils down to this: * pg_amcheck shouldn't attempt to verify temp relations, on the grounds that this is f

Re: storing an explicit nonce

2021-10-06 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Tue, Oct 5, 2021 at 04:29:25PM -0400, Bruce Momjian wrote: > > On Tue, Sep 28, 2021 at 12:30:02PM +0300, Ants Aasma wrote: > > > On Mon, 27 Sept 2021 at 23:34, Bruce Momjian wrote: > > > We are still working on our TDE patch. Right now th

Re: Role Self-Administration

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 2:48 PM Stephen Frost wrote: > What I'm saying above is that the command explicitly listed there > *isn't* 'DROP ROLE A DB', even though that is something which the spec > *could* have done, had they wished to. Given that they didn't, it seems > very clear that making such

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 11:32 AM Robert Haas wrote: > All of the decisions we're talking about here really have to do with > determining the user's intent. I think that if the user says > pg_amcheck --all, there's a good argument that they don't want us to > check unlogged relations on a standby wh

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Andres Freund
Hi, On 2021-10-06 12:58:34 -0400, Tom Lane wrote: > Andres Freund writes: > > One thing I think would really help is having the total time for each run > > visible in an animals run history. That way we could pinpoint regressions > > reasonably efficiently, right now that's not easily possible wi

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Oct 6, 2021, at 11:09 AM, Stephen Frost wrote: > > After all, it says > > "REOKVE R FROM A DB", not "DROP ROLE A CASCADE". > > Wait, are you arguing what DROP ROLE A CASCADE should do based on what the > spec says REVOKE R F

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 1:57 PM Mark Dilger wrote: > > To me #2 sounds like a tautology. It could almost be phrased as > > "pg_amcheck does not check the things that it cannot check". > > I totally disagree. It is uncomfortable to me that `pg_amcheck > --parent-check` will now silently not perfor

Re: Role Self-Administration

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 11:09 AM, Stephen Frost wrote: > > After all, it says > "REOKVE R FROM A DB", not "DROP ROLE A CASCADE". Wait, are you arguing what DROP ROLE A CASCADE should do based on what the spec says REVOKE R FROM A DB should do? If so, I'd say that's irrelevant. I'm not propo

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 10:57 AM Mark Dilger wrote: > > Clearly pg_amcheck never checked all relations, because it never > > checked indexes that are not B-Tree indexes. I'm pretty sure that I > > can poke big holes in almost any positivist statement like that with > > little effort. > > There is a

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Oct 6, 2021, at 10:20 AM, Stephen Frost wrote: > > > > Consistency is not having dangling pointers around to things which no > > longer exist- FK reference kind of things. Object management is about > > actual *removal* of fu

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 10:39 AM, Peter Geoghegan wrote: > >> The differences in the upcoming version are >> >> 1) --all no longer means "all relations" but rather "all checkable relations" > > Clearly pg_amcheck never checked all relations, because it never > checked indexes that are not B-Tre

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 10:19 AM Mark Dilger wrote: > > A return value of 0 cannot be said to indicate that the database is > > not corrupt; > > Nor can a non-zero value be said to indicate that the database is corrupt. I never said otherwise. I think it's perfectly fine that there are multiple no

Re: Role Self-Administration

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 10:20 AM, Stephen Frost wrote: > > Consistency is not having dangling pointers around to things which no > longer exist- FK reference kind of things. Object management is about > actual *removal* of full blown objects like roles, tables, etc. DROP > TABLE ... CASCADE doe

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2021-10-06 Thread Alvaro Herrera
On 2021-Oct-06, Jeremy Schneider wrote: > Well this whole conversation is just theoretical anyway until the code > is shared. :) But if Bharath is writing functions to decode WAL, then > wouldn't we just have pg_waldump use these same functions in order to > avoid duplicating code? Actually, a

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Oct 6, 2021, at 9:01 AM, Stephen Frost wrote: > > I can see how what you describe as the behavior you'd like to see of > > DROP ROLE ... CASCADE could be useful... However, at least in the > > latest version of the standard th

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2021-10-06 Thread Bruce Momjian
On Wed, Oct 6, 2021 at 09:56:34AM -0700, Jeremy Schneider wrote: > On 10/5/21 17:43, Bruce Momjian wrote: > > On Tue, Oct 5, 2021 at 03:30:07PM -0700, Jeremy Schneider wrote: > >> Specifically exposing pg_waldump functionality in SQL could speed up > >> finding bugs in the PG logical replication

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 10:16 AM, Peter Geoghegan wrote: > > A return value of 0 cannot be said to indicate that the database is > not corrupt; Nor can a non-zero value be said to indicate that the database is corrupt. These invocations will still return a non-zero exit status: pg_amch

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Peter Geoghegan
On Wed, Oct 6, 2021 at 9:25 AM Mark Dilger wrote: > Thanks for reviewing! > > I expect to post a new version shortly. Not sure how much it matters, but I have some thoughts on the return value of pg_amcheck. (I'm mostly going into this now because it seems related to how we discuss these issues g

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 12:47 PM Andres Freund wrote: > There's probably some of that. > > The fact that the tap test infrastructure does all communication with the > server via psql each only execute only a single query is a problem - > connection startup is expensive. Ageed. safe_psql() is a poo

Re: storing an explicit nonce

2021-10-06 Thread Bruce Momjian
On Wed, Oct 6, 2021 at 11:17:59AM -0400, Robert Haas wrote: > If you enable checksums or set wal_log_hints=on, then you might incur > a some write-ahead log records that would otherwise be avoided, and > those records will include full page images. This can happen once per > page per checkpoint cy

Re: storing an explicit nonce

2021-10-06 Thread Bruce Momjian
On Wed, Oct 6, 2021 at 12:54:49PM -0400, Bruce Momjian wrote: > On Wed, Oct 6, 2021 at 11:01:25AM -0400, Robert Haas wrote: > > On Tue, Oct 5, 2021 at 4:29 PM Bruce Momjian wrote: > > > On Tue, Sep 28, 2021 at 12:30:02PM +0300, Ants Aasma wrote: > > > > On Mon, 27 Sept 2021 at 23:34, Bruce Momji

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Tom Lane
Andres Freund writes: > One thing I think would really help is having the total time for each run > visible in an animals run history. That way we could pinpoint regressions > reasonably efficiently, right now that's not easily possible without writing > nontrivial queries to the buildfarm databas

Re: [PATCH] Added TRANSFORM FOR for COMMENT tab completion

2021-10-06 Thread Suraj Khamkar
Hello, I reviewed your patch. At a first glance, I have the below comments. 1. The below change crosses the 80-character limit in a line. Please maintain the same. - "LARGE OBJECT", "TABLESPACE", "TEXT SEARCH", "ROLE"); + "LARGE OBJECT", "TABLESPACE", "TEXT SEARCH", "TRANSFORM FOR"

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2021-10-06 Thread Jeremy Schneider
On 10/5/21 17:43, Bruce Momjian wrote: > On Tue, Oct 5, 2021 at 03:30:07PM -0700, Jeremy Schneider wrote: >> Specifically exposing pg_waldump functionality in SQL could speed up >> finding bugs in the PG logical replication code. We found and fixed a >> few over this past year, but there are more

Re: storing an explicit nonce

2021-10-06 Thread Bruce Momjian
On Wed, Oct 6, 2021 at 11:01:25AM -0400, Robert Haas wrote: > On Tue, Oct 5, 2021 at 4:29 PM Bruce Momjian wrote: > > On Tue, Sep 28, 2021 at 12:30:02PM +0300, Ants Aasma wrote: > > > On Mon, 27 Sept 2021 at 23:34, Bruce Momjian wrote: > > > We are still working on our TDE patch. Right now the f

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Andres Freund
Hi, On 2021-10-06 12:09:36 -0400, Robert Haas wrote: > Is the problem here that we're adding a lot of new new test cases? Or > is the problem that valgrind runs are getting slower for the same > number of test cases? I don't know precisely. It's probably a combination of several factors. I do thi

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-10-06 Thread Bossart, Nathan
On 10/6/21, 12:13 AM, "Masahiko Sawada" wrote: > A customer reported that during parallel index vacuum, the oldest xmin > doesn't advance. Normally, the calculation of oldest xmin > (ComputeXidHorizons()) ignores xmin/xid of processes having > PROC_IN_VACUUM flag in MyProc->statusFlags. But since

Re: Pre-allocating WAL files

2021-10-06 Thread Bossart, Nathan
On 10/6/21, 5:20 AM, "Maxim Orlov" wrote: > We've looked through the code and everything looks good except few minor > things: > 1). Using dedicated bg worker seems not optimal, it introduces a lot of > redundant code for little single action. > We'd join initial proposal of Andres to implem

Re: [Proposal] Global temporary tables

2021-10-06 Thread Andrew Bille
On master with the v54 patches applied the following script leads to crash: export ASAN_OPTIONS=detect_leaks=0:abort_on_error=1:disable_coredump=0:strict_string_checks=1:check_initialization_order=1:strict_init_order=1 initdb -D data pg_ctl -w -t 5 -D data -l server.log start psql -c "create global

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 8:14 AM, Pavel Borisov wrote: > > We've looked through the initial patch and the exclusion of temporary tables > from pg_amcheck seems the right thing. Also it is not the matter anyone > disagrees here, and we propose to commit it alone. Thanks for reviewing! I expect

Re: Speed up transaction completion faster after many relations are accessed in a transaction

2021-10-06 Thread Yura Sokolov
I've made some remarks in related thread: https://www.postgresql.org/message-id/flat/0A3221C70F24FB45833433255569204D1FB976EF@G01JPEXMBYT05 The new status of this patch is: Waiting on Author

Re: Role Self-Administration

2021-10-06 Thread Mark Dilger
> On Oct 6, 2021, at 9:01 AM, Stephen Frost wrote: > > I can see how what you describe as the behavior you'd like to see of > DROP ROLE ... CASCADE could be useful... However, at least in the > latest version of the standard that I'm looking at, when a > DROP ROLE ... CASCADE is executed, wh

Re: Use simplehash.h instead of dynahash in SMgr

2021-10-06 Thread Yura Sokolov
Good day, David and all. В Вт, 05/10/2021 в 11:07 +1300, David Rowley пишет: > On Mon, 4 Oct 2021 at 20:37, Jaime Casanova > wrote: > > Based on your comments I will mark this patch as withdrawn at midday > > of > > my monday unless someone objects to that. > > I really think we need a hash tabl

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 1:57 AM Andres Freund wrote: > After a recent migration of the skink and a few other animals (sorry for the > false reports on BF, I forgot to adjust a path), I looked at the time it takes > to complete a valgrind run: > > 9.6: Consumed 4h 53min 18.518s CPU time > 10: Consum

Re: Role Self-Administration

2021-10-06 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On Oct 5, 2021, at 10:20 AM, Stephen Frost wrote: > > On Tue, Oct 5, 2021 at 13:17 Mark Dilger > > wrote: > > > On Oct 5, 2021, at 10:14 AM, Stephen Frost wrote: > > > > > > What does the “ownership” concept actually buy us th

Re: Running tests under valgrind is getting slower at an alarming pace

2021-10-06 Thread Tom Lane
Andres Freund writes: > After a recent migration of the skink and a few other animals (sorry for the > false reports on BF, I forgot to adjust a path), I looked at the time it takes > to complete a valgrind run: > 9.6: Consumed 4h 53min 18.518s CPU time > 10: Consumed 5h 32min 50.839s CPU time >

Re: storing an explicit nonce

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 11:22 AM Stephen Frost wrote: > Seems like it'd be easier to achieve that by having something that looks > very close to how write() looks, but just happens to have the option to > run the data through a stream cipher and maybe does better error > handling for us. Making th

Re: Role Self-Administration

2021-10-06 Thread Robert Haas
On Tue, Oct 5, 2021 at 3:41 PM Mark Dilger wrote: > If roles have owners, then DROP ROLE bob CASCADE drops bob, bob's objects, > roles owned by bob, their objects and any roles they own, recursively. Roles > which bob merely has admin rights on are unaffected, excepting that they are > adminis

Re: storing an explicit nonce

2021-10-06 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Oct 5, 2021 at 1:24 PM Robert Haas wrote: > > On Mon, Oct 4, 2021 at 10:00 PM Stephen Frost wrote: > > > I do want to point out, as I think I did when we discussed this but want > > > to be sure it's also captured here- I don't th

Re: storing an explicit nonce

2021-10-06 Thread Robert Haas
On Wed, Oct 6, 2021 at 9:35 AM Bruce Momjian wrote: > The above text isn't very clear. What I am saying is that currently > torn pages can be tolerated by hint bit writes because only a single > byte is changing. If we use a block cipher like AES-XTS, later 16-byte > encrypted blocks would be ch

Re: BUG #17212: pg_amcheck fails on checking temporary relations

2021-10-06 Thread Pavel Borisov
Hi, hackers! We've looked through the initial patch and the exclusion of temporary tables from pg_amcheck seems the right thing. Also it is not the matter anyone disagrees here, and we propose to commit it alone. Supplementary things/features might be left for further discussion but refusing to ch

Re[2]: [PATCH] Tracking statements entry timestamp in pg_stat_statements

2021-10-06 Thread Мельников Антон Андреевич
Hi, Andrey!   I’ve tried to apply your patch v2-0001 on current master, but i failed. There were git apply errors at: pg_stat_statements.out:941 pg_stat_statements.sql:385   Best Regards, Anton Melnikov Postgres Professional: http://www.postgrespro.com The Russian Postgres Company  

Re: Compressing temporary files

2021-10-06 Thread Tomas Vondra
Hi, On 9/11/21 2:31 PM, Andrey Borodin wrote: > Hi hackers! > > There's a lot of compression discussions nowadays. And that's cool! > Recently Naresh Chainani in private discussion shared with me the > idea to compress temporary files on disk. And I was thrilled to find > no evidence of implemen

Re: storing an explicit nonce

2021-10-06 Thread Robert Haas
On Tue, Oct 5, 2021 at 4:29 PM Bruce Momjian wrote: > On Tue, Sep 28, 2021 at 12:30:02PM +0300, Ants Aasma wrote: > > On Mon, 27 Sept 2021 at 23:34, Bruce Momjian wrote: > > We are still working on our TDE patch. Right now the focus is on refactoring > > temporary file access to make the TDE patc

Re: storing an explicit nonce

2021-10-06 Thread Robert Haas
On Tue, Oct 5, 2021 at 1:55 PM Antonin Houska wrote: > I'm just trying to make our changes to buffile.c less invasive. Or do you mean > that this module should be reworked regardless the encryption? I wasn't thinking of buffile.c specifically. I think improving that might be a really good idea, a

Re: Compressing temporary files

2021-10-06 Thread Robert Haas
On Sat, Sep 11, 2021 at 8:31 AM Andrey Borodin wrote: > I've prototyped Random Access Compressed File for fun[0]. The code is very > dirty proof-of-concept. > I compress Buffile by one block at a time. There are directory pages to store > information about the size of each compressed block. If a

  1   2   >