Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-06 Thread Mark Dilger
don't see a specific security concern with that, though I'd be happy to consider arguments to the contrary. I believe there are also some issues with this patch relating to pg_dump/pg_restore and to pg_upgrade, which EDB is working to address. We intend to repost something soon. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-06 Thread Mark Dilger
I'm inclined to at least try doing it as you propose. Thanks for the suggestion! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-07 Thread Mark Dilger
take a stab at a proof-of-concept patch unless you suggest a better approach. Thoughts? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-07 Thread Mark Dilger
option, we could document that it works in a non-inherited way. We'd just be saying the current hard-coded behavior is an option which can be revoked rather than something you're stuck with. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-07 Thread Mark Dilger
> On Mar 7, 2022, at 12:03 PM, Mark Dilger wrote: > > Right, but with a reflexive self-admin-option, we could document that it > works in a non-inherited way. We'd just be saying the current hard-coded > behavior is an option which can be revoked rather than something

Re: role self-revocation

2022-03-07 Thread Mark Dilger
upgrade to the patched version of HEAD from each of versions 11, 12, 13 and 14 doesn't turn up anything untoward. The change I used (for reference) is attached: v1-0001-WIP-Removing-role-self-administration.patch.WIP Description: Binary data — Mark Dilger EnterpriseDB: http://www.ente

Re: role self-revocation

2022-03-10 Thread Mark Dilger
nd if CREATEROLE has been made more configurable, wouldn't the bot only be able to grant that ldap user the limited set of privileges that the bot's database user has been granted ADMIN OPTION for? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-10 Thread Mark Dilger
OLE itself could be reimagined as pg_create_role, and then users could be granted into this role with or without admin option, meaning they could/couldn't further give it away. I think that would be a necessary component to Joshua's "bot" use-case, since the bot must

Re: role self-revocation

2022-03-11 Thread Mark Dilger
pt whatever backwards incompatibility that entails. This is not a green-field development project. The constant spinning around with regard to how much compatibility we need to preserve is giving me vertigo. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-11 Thread Mark Dilger
he command. Changing the CREATEROLE, CREATEDB, REPLICATION, and BYPASSRLS attributes into pg_create_role, pg_create_db, pg_replication, and pg_bypassrls, the creator could only give them to the created role if the creator has G on the roles. If we do this, we could keep the historical privilege bits and their syntax support for backward compatibility, or we could rip them out, but the decision between those two options is independent of the rest of the design. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-11 Thread Mark Dilger
or what?) constitute a competing idea to the idea of role ownership, and greater detail about how each of those steps translate into specific behavior changes in postgres. Your initial five-step email seems to be claiming that #1 by itself is competitive, but to me it seems at least #1 and #2 woul

Re: role self-revocation

2022-03-11 Thread Mark Dilger
to distill technical details, high level conversation can be hard to grok. I have the sense that you aren't going to submit a patch, so I wanted this thread to contain enough detail for somebody else to do so. Thanks. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: role self-revocation

2022-03-14 Thread Mark Dilger
> On Mar 14, 2022, at 7:38 AM, Stephen Frost wrote: > > So ... do you feel like that's now the case? Or were you looking for > more? I don't have any more questions at the moment. Thanks! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg14 psql broke \d datname.nspname.relname

2022-03-15 Thread Mark Dilger
> On Mar 15, 2022, at 12:27 PM, Robert Haas wrote: > > - Justin Pryzby, who originally discovered the problem, prefers the > same behavior that I prefer long-term, but thinks Tom's behavior is > better than doing nothing. > - Mark Dilger, Isaac Moreland, Garick Haml

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-16 Thread Mark Dilger
merely into GRANT and REVOKE. > If we do have > a need for a hook editorializing on GUC value settings, that would > have to be an independent API --- but I have heard no calls for > the ability to have such a hook, and I don't think that this patch > is the place to introduce one. Well, the call for it was in this thread, but I'm ok with yanking out that part of the patch if it seems half-baked. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-17 Thread Mark Dilger
rop the DAC portion for postgres 15 and revisit the issue for postgres 16? I think it's getting late in the development cycle to attempt what Tom described upthread, and I'd hate to see the rest of this patch punted. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-17 Thread Mark Dilger
> On Mar 17, 2022, at 9:29 AM, Joshua Brindle > wrote: > > I hope this patch can be rolled > into the contribution from Mark. Working on it Thanks for the patch! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

New Object Access Type hooks

2022-03-17 Thread Mark Dilger
patch Description: Binary data [1] https://www.postgresql.org/message-id/flat/664799.1647456444%40sss.pgh.pa.us#c9721c2da88d59684ac7ac5fc36f09c1 — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-18 Thread Mark Dilger
ed in a different order, something which isn't necessarily wrong, but should not be done unknowingly. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-21 Thread Mark Dilger
he .control file exists because the test defines a loadable module which defines the hooks. The test_oat_hooks.h file was extraneous, and has been removed in v2. v2-0001-Add-String-object-access-hooks.patch Description: Binary data v2-0002-Add-regression-tests-of-Object-Access-Type-hooks.patch

Re: New Object Access Type hooks

2022-03-21 Thread Mark Dilger
o fix the function names in the comments. Joshua, Do you care to create a new version of this, perhaps based on the v2-0001 patch I just posted? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-21 Thread Mark Dilger
ion-tests-of-Object-Access-Type-hooks.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg14 psql broke \d datname.nspname.relname

2022-03-21 Thread Mark Dilger
> On Mar 21, 2022, at 6:12 PM, Andres Freund wrote: > > Needs another one: http://cfbot.cputube.org/patch_37_3367.log > > Marked as waiting-on-author. v6-0001-Reject-patterns-with-too-many-parts-or-wrong-db.patch Description: Binary data — Mark Dilger Ent

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
> On Mar 21, 2022, at 10:03 PM, Thomas Munro wrote: > > On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger > wrote: >> (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl >> when testing v1-0001. I'm not sure yet what that is about.) > >

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
==~_~== pgsql.build/src/test/modules/test_rls_hooks/log/initdb.log ==~_~===-=-===~_~== — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
heckPerms_hook, but since it is causing problems, I can submit a patch with that removed. Give me a couple minutes.... — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
> On Mar 22, 2022, at 9:11 AM, Mark Dilger wrote: > > Give me a couple minutes v1-0001-Fix-build-farm-failures-in-test_oat_hooks.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
> On Mar 22, 2022, at 9:33 AM, Tom Lane wrote: > > Andrew Dunstan writes: >> On 3/22/22 11:26, Mark Dilger wrote: >>> culicidae is complaining: >>> 2022-03-22 14:53:27.198 UTC [2167008][not initialized][:0] FATAL: >>> test_oat_hooks must be load

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
ot;To" line of that email was to you and Andrew. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: New Object Access Type hooks

2022-03-22 Thread Mark Dilger
and to enabled for > others? Wouldn't we get differing numbers of NOTICE messages depending on how many parallel workers there are? Or would you propose setting the number of workers to a small, fixed value? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

room for improvement in amcheck btree checking?

2020-12-01 Thread Mark Dilger
error shown above. [1] https://commitfest.postgresql.org/31/2670/ [2] https://www.postgresql.org/docs/13/amcheck.html [3] https://www.postgresql.org/message-id/flat/CAH2-WznaU6HcahLV4Hg-DnhEmW8DuSdYfn3vfWXoj3Me9jq%3DsQ%40mail.gmail.com#0691475da5e9163d21b13fc415095801 — Mark Dilger EnterpriseD

Re: room for improvement in amcheck btree checking?

2020-12-01 Thread Mark Dilger
> On Dec 1, 2020, at 4:38 PM, Peter Geoghegan wrote: > > On Tue, Dec 1, 2020 at 12:39 PM Mark Dilger > wrote: >> I found it surprising that even when precisely zero of the tids in the index >> exist in the table the index checks all come back clean. The hea

Re: Test harness for regex code (to allow importing Tcl's test suite)

2021-01-03 Thread Mark Dilger
;t in test/modules, it might be nice to expose test_regex from SQL with a slightly different interface that doesn't throw on regex compilation error? Maybe something in contrib? It might be useful to some users to validate regular expressions. I'm just asking... I don't have any problem with how you have it here. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: macOS SIP, next try

2021-01-04 Thread Mark Dilger
wWAQ%40mail.gmail.com > https://www.postgresql.org/message-id/flat/6a4d6124-41f0-756b-0811-c5c5def7ef4b%402ndquadrant.com See also https://www.postgresql.org/message-id/flat/18012hGLG6HJ9pQDkHAMYuwQKg%40sparkpost.com — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: new heapcheck contrib module

2021-01-07 Thread Mark Dilger
hich has an xmin that is less than the >> pg_clas.relfrozenxid value by 1. You are proposing that because I have >> the latter problem I don't want you to check for the former one. But >> I, John Q. Smartuser, do not want you to second-guess what I told you >> on the command line that I wanted. :-) > > Even if your user is just average, they still have one major advantage > over the architects of pg_amcheck: actual knowledge of the problem in > front of them. There is a switch for skipping index checks on corrupt tables. By default, the indexes will be checked. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: new heapcheck contrib module

2021-02-23 Thread Mark Dilger
> On Feb 17, 2021, at 12:56 PM, Robert Haas wrote: > > On Wed, Feb 17, 2021 at 1:46 PM Mark Dilger > wrote: >> It will reconnect and retry a command one time on error. That should cover >> the case that the connection to the database was merely lost. If the secon

Re: proposal: psql –help reflecting service or URI usage

2021-02-28 Thread Mark Dilger
Other client applications follow the same pattern as psql, so if this change were adopted, it should apply to all of them. Your proposal seems like something that would have been posted to the list before, possibly multiple times. Any chance you could dig up past conversations on this subject and post links here for context? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: proposal: psql –help reflecting service or URI usage

2021-03-01 Thread Mark Dilger
> On Feb 28, 2021, at 10:10 AM, Paul Förster wrote: > > Hi Mark, > >> On 28. Feb, 2021, at 17:54, Mark Dilger wrote: >> >> "definited" is a typo. > > yes, definitely a typo, sorry. Thanks for pointing this out. > >> Should this s

Re: Add --tablespace option to reindexdb

2021-03-01 Thread Mark Dilger
do here, beware that you are using similar language in the --help, so consider changing that, too. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Add --tablespace option to reindexdb

2021-03-01 Thread Mark Dilger
h verbose output'); + [ 'reindexdb', '--concurrently', '-v', '-t', 'test1', 'postgres' ], + qr/statement: REINDEX \(VERBOSE\) TABLE CONCURRENTLY public\.test1;/, + 'reindex concurrently with verbose output');

Re: Add --tablespace option to reindexdb

2021-03-01 Thread Mark Dilger
;command_checks_all( [ 'reindexdb','--concurrently', '-i', $toast_index, '--tablespace', $tbspace, 'postgres' ], + 1, + [ ], + [ qr/cannot move system relation/ ], 'reindex toast index concurrently with tablespace'); # connection strings — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Cross-reference comments on signal handling logic

2021-03-01 Thread Mark Dilger
t.c contains handlers that normal user backends to not use, then how can it be that most backends use one of the handlers in the file? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-03-01 Thread Mark Dilger
> On Feb 9, 2021, at 10:43 AM, Pavel Borisov wrote: > > вт, 9 февр. 2021 г. в 01:46, Mark Dilger : > > > > On Feb 8, 2021, at 2:46 AM, Pavel Borisov wrote: > > > > 0002 - is a temporary hack for testing. It will allow inserting duplicates > > in

Re: Use extended statistics to estimate (Var op Var) clauses

2021-03-01 Thread Mark Dilger
> On Nov 12, 2020, at 5:14 PM, Tomas Vondra > wrote: > > <0001-Support-estimation-of-clauses-of-the-form-V-20201113.patch> Your patch no longer applies. Can we get a new version please? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-03-01 Thread Mark Dilger
he tid values in the test output, but it does pass. > Thanks for your attention to the patch Thanks for the patch! — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-03-01 Thread Mark Dilger
nning uniqueness checks on unique indexes. It seems fairly normal that a user would want that. Perhaps the option should default to 'true' if unspecified? But having no option at all seems to run contrary to how the other functionality is structured. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: new heapcheck contrib module

2021-03-01 Thread Mark Dilger
> On Mar 1, 2021, at 1:14 PM, Robert Haas wrote: > > On Wed, Feb 24, 2021 at 1:55 PM Mark Dilger > wrote: >> [ new patches ] > > Regarding 0001: > > There seem to be whitespace-only changes to the comment for select_loop(). > > I w

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

2021-03-01 Thread Mark Dilger
so I did a bit of testing. I think the following should not error, but does: +SELECT regexp_positions('foObARbEqUEbAz', $re$(?=beque)$re$, 'i'); +ERROR: range lower bound must be less than or equal to range upper bound — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Add --tablespace option to reindexdb

2021-03-02 Thread Mark Dilger
> On Mar 1, 2021, at 10:04 PM, Michael Paquier wrote: > > On Mon, Mar 01, 2021 at 10:12:54AM -0800, Mark Dilger wrote: >> Your check verifies that reindexing a system table on a new >> tablespace fails, but does not verify what the failure was. I >> wonder if you

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

2021-03-02 Thread Mark Dilger
matches but you do at least have where > they are relative to non-empty matches. I agree the contents of the match don't matter, because they are always empty. But the position matters. You could intend to split a string in multiple places using lookaheads and lookbehinds to determi

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
I don't find anything, probably because I'm not searching for the right keywords. Anybody have a link to that discussion? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
> On Mar 2, 2021, at 8:51 AM, Pantelis Theodosiou wrote: > > > > On Tue, Mar 2, 2021 at 3:28 PM Mark Dilger > wrote: > > > > On Mar 2, 2021, at 5:20 AM, Joel Jacobson wrote: > > > > it's currently not possible to create an empty range wi

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
n years too late. Range semantics are already relied upon in our code, but also in the code of others. It might be very hard to debug code that was correct when written but broken by this proposed change. The problem is not just with lower() and upper(), but with equality testin

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
> On Mar 2, 2021, at 10:08 AM, Joel Jacobson wrote: > > On Tue, Mar 2, 2021, at 19:01, Mark Dilger wrote: >> The problem is not just with lower() and upper(), but with equality testing >> (mentioned upthread), since code may rely on two different "positions" (

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
> On Mar 2, 2021, at 10:16 AM, Chapman Flack wrote: > > On 03/02/21 13:01, Mark Dilger wrote: >> The problem is not just with lower() and upper(), but with equality testing >> (mentioned upthread), since code may rely on two different "positions" >> (

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
s against something else, it's not going to work. Positions will randomly vanish from the results of that join, which is going to be really surprising. I'm sure there are other examples of Tom's general point about compares-equal-but-not-equal datatypes. — Mark Dilger E

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
> On Mar 2, 2021, at 11:42 AM, Mark Dilger wrote: > > > >> On Mar 2, 2021, at 11:34 AM, Joel Jacobson wrote: >> >> Yes. It's random, since equality isn't changed, the sort operation cannot >> tell the difference, and nor could a user who isn

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
> On Mar 2, 2021, at 12:04 PM, Joel Jacobson wrote: > > On Tue, Mar 2, 2021, at 20:57, Mark Dilger wrote: >> I didn't phrase that clearly enough. I'm thinking about whether you include >> the bounds information in the hash function. The current i

Re: [PATCH] Support empty ranges with bounds information

2021-03-02 Thread Mark Dilger
n it originally got written. (Not to say that it *should* have worked that way, just that part of me wanted it.) — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-04 Thread Mark Dilger
mething like that. I don't think it's very fun to have it error out for each relation that doesn't have at least ten blocks, nor is it fun to have those relations skipped by error'ing out before checking any blocks, as they might be the corrupt relations you are looking for. But using --startblock and --endblock for this is not a natural fit, as evidenced by how I was trying to "fix things up" for the user, so I'll punt on this usage until some future version, when I might add a sampling option. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: proposal: psql –help reflecting service or URI usage

2021-03-08 Thread Mark Dilger
> On Mar 6, 2021, at 5:55 AM, Paul Förster wrote: > > Hi Mark, > > sorry for the delay. > >> On 01. Mar, 2021, at 17:02, Mark Dilger wrote: >> >> The output from --help should fit in a terminal window with only 80 >> characters width. For ex

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

2021-03-08 Thread Mark Dilger
his string should read as {"[14,14)"}, and you have been forced to add one to avoid that collapsing down to "empty", but I'd rather you found a different datatype rather than abuse the definition of int4range. It seems that you may have reached a similar conclusion down-thread? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: proposal: psql –help reflecting service or URI usage

2021-03-08 Thread Mark Dilger
> On Mar 8, 2021, at 8:40 AM, Paul Förster wrote: > > Hi Mark, > >> On 08. Mar, 2021, at 16:39, Mark Dilger wrote: >> >> Fortunately, the man pages and html docs are generated from the same >> sources. Those sources are written in sgml, and the

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

2021-03-08 Thread Mark Dilger
to go along with the regexp_positions, and then do it that way? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] regexp_positions ( string text, pattern text, flags text ) → setof int4range[]

2021-03-08 Thread Mark Dilger
> On Mar 8, 2021, at 9:20 AM, Joel Jacobson wrote: > > On Mon, Mar 8, 2021, at 18:11, Mark Dilger wrote: >> > On Mar 8, 2021, at 9:05 AM, Joel Jacobson wrote: >> > >> > If a N+1 dimension array could easily be unnested to a N dimension array, >

Re: pg_amcheck contrib application

2021-03-08 Thread Mark Dilger
> On Mar 8, 2021, at 8:26 AM, Robert Haas wrote: > > On Thu, Mar 4, 2021 at 5:39 PM Mark Dilger > wrote: >> I think Robert mistook why I was doing that. I was thinking about a >> different usage pattern. If somebody thinks a subset of relations have been >>

Re: Lowering the ever-growing heap->pd_lower

2021-03-09 Thread Mark Dilger
-id/CAEze2Wjf42g8Ho%3DYsC_OvyNE_ziM0ZkXg6wd9u5KVc2nTbbYXw%40mail.gmail.com > For a prior discussion on this topic: https://www.postgresql.org/message-id/2e78013d0709130606l56539755wb9dbe17225ffe90a%40mail.gmail.com — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Lowering the ever-growing heap->pd_lower

2021-03-09 Thread Mark Dilger
used (rather than garbage collected by this patch) prior to such hypothetical code using an outdated TID? I'm not expressing a view here, just asking questions. [1] https://www.postgresql.org/message-id/2e78013d0709130832t31244e79k9488a3e4eb00d64c%40mail.gmail.com — Mark Dilger Enterpri

Re: pg_amcheck contrib application

2021-03-11 Thread Mark Dilger
that was at the moment.) I have no argument with changing the location of this tool before it gets committed, but I wonder if we should do that now, or wait until some future time when contrib gets reorganized? I can't quite tell which you prefer from your comments above. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-11 Thread Mark Dilger
one. pg_amcheck is not designed to detect corruption directly, but rather to open one or more connections to the database and execute sql queries which employ the contrib/amcheck sql functions. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-11 Thread Mark Dilger
> On Mar 11, 2021, at 9:10 AM, Andrey Borodin wrote: > > > >> 11 марта 2021 г., в 20:30, Mark Dilger >> написал(а): >> >> >> pg_amcheck does not need a local data directory to check a remote database >> server, though it does need to con

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
entries for value %u". > > -- > Robert Haas > EDB: http://www.enterprisedb.com This does nothing to change the verbiage from contrib/amcheck, but it should address the problems discussed here in pg_amcheck's regression tests. v1-0001-Fixing-portability-issues-in-pg_amcheck-regressio.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
> On Mar 12, 2021, at 2:55 PM, Robert Haas wrote: > > On Fri, Mar 12, 2021 at 5:24 PM Mark Dilger > wrote: >> This does nothing to change the verbiage from contrib/amcheck, but it should >> address the problems discussed here in pg_amcheck's regression tests. &g

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
ar before finishing all relations in foo, but with a fudge factor, pg_amcheck can report that the processing has moved on to database bar, etc. You wouldn't want the parens to jump around when the long database names get processed. So it keeps the parens in the same location, space

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
> On Mar 12, 2021, at 3:24 PM, Mark Dilger wrote: > > and the second deals with an apparent problem with IPC::Run shell expanding > an asterisk on some platforms but not others. That second one, if true, > seems like a problem with scope beyond the pg_amcheck project

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
> On Mar 12, 2021, at 5:16 PM, Robert Haas wrote: > > Gah, tests are so annoying. :-) There is another problem of non-portable option ordering in the tests. v4-0001-pg_amcheck-Keep-trying-to-fix-the-tests.patch Description: Binary data — Mark Dilger Enterpris

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
ne here. > > Pushed your fix. > > regards, tom lane Thanks! Was just responding to your other email, but now I don't have to send it. Sorry for painting so many farm animals red this evening. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
gt; > hoverfly does configure with PERL=perl64. /usr/bin/prove is from the 32-bit > Perl, so I suspect the TAP suites get 32-bit Perl that way. (There's no > "prove64".) This test should unpack the field as two 32-bit values, not a > 64-bit value, to avoid requiring more from the Perl installation. I will post a modified test in a bit that avoids using Q/q. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
> On Mar 12, 2021, at 10:22 PM, Tom Lane wrote: > > Mark Dilger writes: >> On Mar 12, 2021, at 10:16 PM, Noah Misch wrote: >>> hoverfly does configure with PERL=perl64. /usr/bin/prove is from the 32-bit >>> Perl, so I suspect the TAP suites get 3

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
> On Mar 12, 2021, at 10:28 PM, Mark Dilger > wrote: > > > >> On Mar 12, 2021, at 10:22 PM, Tom Lane wrote: >> >> Mark Dilger writes: >>> On Mar 12, 2021, at 10:16 PM, Noah Misch wrote: >>>> hoverfly does configure with PERL=perl64.

Re: pg_amcheck contrib application

2021-03-12 Thread Mark Dilger
> On Mar 12, 2021, at 10:36 PM, Tom Lane wrote: > > Mark Dilger writes: >> On Mar 12, 2021, at 10:22 PM, Tom Lane wrote: >>> Coping with both endiannesses might be painful. > >> Not too bad if the bigint value is zero, as both the low and high 32bit

Re: pg_amcheck contrib application

2021-03-13 Thread Mark Dilger
patch uses the value DEADF9F9 as it seems a little easier to remember than other permutations of those nibbles. v6-0001-pg_amcheck-continuing-to-fix-portability-problems.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

amcheck hardening

2021-03-13 Thread Mark Dilger
h Description: Binary data [3] https://www.postgresql.org/message-id/CA%2BTgmob2c0eM8%2B5kzkXaqdc9XbBCkHmtihSOSk-jCzzT4BFhFQ%40mail.gmail.com — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-13 Thread Mark Dilger
> On Mar 13, 2021, at 10:46 AM, Noah Misch wrote: > > On Fri, Mar 12, 2021 at 05:04:09PM -0800, Mark Dilger wrote: >>> On Mar 12, 2021, at 3:24 PM, Mark Dilger >>> wrote: >>> >>> and the second deals with an apparent problem with IPC::Run shell

Re: amcheck hardening

2021-03-13 Thread Mark Dilger
> On Mar 13, 2021, at 11:06 AM, Peter Geoghegan wrote: > > On Sat, Mar 13, 2021 at 10:35 AM Mark Dilger > wrote: >> The testing strategy I'm using is to corrupt heap and btree pages in schema >> "public" of the "regression" database created

Re: REINDEX backend filtering

2021-03-14 Thread Mark Dilger
on you went a different direction with this feature? + printf(_(" --outdated only process indexes having outdated depencies\n")); typo. + bool outdated; /* depends on at least on deprected collation? */ typo. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [PATCH] Improve amcheck to also check UNIQUE constraint in btree index.

2021-03-15 Thread Mark Dilger
oblems. > > You "break the warranty" by updating pg_index, even compared to > updating other system catalogs. In particular, you break the > "indcheckxmin wait -- wait for xmin to be old before using index" > stuff in get_relation_info(). So it seems worse than

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
icu" and "libc" that it currently accepts, I wonder if it might accept "test" or similar, and then you could create a test in src/test/modules that compiles a "test" provider, creates a database with indexes dependent on something from that provider, stops the

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
returns a different version string and has genuinely different collation rules between versions, thereby breaking the index until it is updated. Is that being tested? — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 10:34 AM, Julien Rouhaud wrote: > > On Mon, Mar 15, 2021 at 10:13:55AM -0700, Mark Dilger wrote: >> >> >>> On Mar 15, 2021, at 9:52 AM, Julien Rouhaud wrote: >>> >>> But there are also the tests in collate.icu.utf8.

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 10:50 AM, Julien Rouhaud wrote: > > On Mon, Mar 15, 2021 at 10:40:25AM -0700, Mark Dilger wrote: >> I'm saying that your patch seems to call down to >> get_collation_actual_version() via get_collation_version_for_oid()

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
sible that pg_statistic really is corrupt here, and that this is not a bug in pg_amcheck? It's not like we've been checking for corruption in the build farm up till now. I notice that this test, as well as test 005_opclass_damage.pl, neglects to turn off autovacuum for th

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:10 AM, Julien Rouhaud wrote: > > On Mon, Mar 15, 2021 at 10:56:50AM -0700, Mark Dilger wrote: >> >> >>> On Mar 15, 2021, at 10:50 AM, Julien Rouhaud wrote: >>> >>> On Mon, Mar 15, 2021 at 10:40:25AM -0700, Mark Dil

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:11 AM, Mark Dilger > wrote: > > I will submit a patch that turns off autovacuum for the test node shortly. v5-0001-Turning-off-autovacuum-during-corruption-tests.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com Th

Re: REINDEX backend filtering

2021-03-15 Thread Mark Dilger
> On Mar 15, 2021, at 11:32 AM, Mark Dilger > wrote: > > If you had a real, not fake, collation provider which actually provided a > collation with an actual version number, stopped the server, changed the > behavior of the collation as well as its version number,

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
reason to assume that is the issue. If we still see the complaint on tern or hornet after committing the patch to turn off autovacuum, we will be able to rule out the theory that the toast was removed by autovacuum. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
v6-0002-Fixing-a-confusing-amcheck-corruption-message.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-15 Thread Mark Dilger
ing-a-confusing-amcheck-corruption-message.patch Description: Binary data v7-0003-Extend-pg_amcheck-test-suite.patch Description: Binary data v7-0004-Add-extra-check-of-toast-pointer-in-amcheck.patch Description: Binary data — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-16 Thread Mark Dilger
is propogating corruptions into pg_statistic, and also the theory that it is architecture dependent. I'll investigate further. — Mark Dilger EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: pg_amcheck contrib application

2021-03-16 Thread Mark Dilger
> On Mar 16, 2021, at 9:07 AM, Tom Lane wrote: > > Mark Dilger writes: >> I think autovacuum simply triggers the bug, and is not the cause of the bug. >> If I turn autovacuum off and instead do an ANALYZE in each test database >> rather than performing the corru

  1   2   3   4   5   6   7   8   9   10   >