Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-13 Thread Dilip Kumar
On Wed, May 12, 2021 at 5:55 PM Amul Sul wrote: > Thanks for the updated patch, while going through I noticed this comment. + /* + * WAL prohibit state changes not allowed during recovery except the crash + * recovery case. + */ + PreventCommandDuringRecovery("pg_prohibit_wal()"); Why do we nee

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 12:33:47 +0800, Julien Rouhaud wrote in > Le jeu. 13 mai 2021 à 12:26, Kyotaro Horiguchi a > écrit : > > > At Thu, 13 May 2021 12:11:12 +0900 (JST), Kyotaro Horiguchi < > > horikyota@gmail.com> wrote in > > pg_stat_statemnts defines its own query-id provider function in

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Michael Paquier
On Tue, May 11, 2021 at 04:42:27PM +0900, Michael Paquier wrote: > Whatever the solution chosen, the thing I can see we agree on here is > that we need to do something, at least in the shape of an on/off > switch to have an escape path in case of problems. Peter, could we > get something by beta1

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 04:15:30PM +0900, Kyotaro Horiguchi wrote: > > > > what if you want to have some other extensions like pg_stat_kcache or > > pg_store_plans that need a query_id but don't really care if > > pg_stat_statements is enabled or not? should they all declare their own > > Thanks

Re: Inherited UPDATE/DELETE vs async execution

2021-05-13 Thread Etsuro Fujita
On Thu, May 13, 2021 at 3:32 PM Amit Langote wrote: > On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita wrote: > > Here is a rebased version of the patch. I'm planning to apply this > > tommorow. > > + /* > +* Finally, unset the async-capable flag if it is set. > +*/ > > Would it make sen

Re: Executor code - found an instance of a WHILE that should just be an IF

2021-05-13 Thread David Rowley
On Mon, 10 May 2021 at 23:49, David Rowley wrote: > > On Mon, 10 May 2021 at 21:16, Greg Nancarrow wrote: > > During debugging I noticed some code in ExecResult() where a WHILE > > loop is being used with an unconditional RETURN at the end of the > > block (which is intentional, looking at the hi

Re: RFC: Logging plan of the running query

2021-05-13 Thread torikoshia
Thank you all for your positive comments. On 2021-05-12 21:55, Matthias van de Meent wrote: Great idea. One feature I'd suggest would be adding a 'format' option as well, if such feature would be feasable. Thanks for the comment! During the development of pg_log_backend_memory_contexts(), I

Re: RFC: Logging plan of the running query

2021-05-13 Thread torikoshia
On 2021-05-13 01:08, Laurenz Albe wrote: On Wed, 2021-05-12 at 18:03 +0530, Bharath Rupireddy wrote: Since it also shows up the full query text and the plan in the server log as plain text, there are chances that the sensitive information might be logged into the server log which is a risky thin

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Wed, May 12, 2021 at 4:54 PM torikoshia wrote: > > Hi, > > During the discussion about memory contexts dumping[1], there > was a comment that exposing not only memory contexts but also > query plans and untruncated query string would be useful. > > I also feel that it would be nice when thinkin

Re: [PATCH] distinct aggregates within a window function WIP

2021-05-13 Thread Eugen Konkov
I resolve my problem https://stackoverflow.com/a/67167595/4632019: Could it be possible PG will use `filter` trick when DISTINCT is used: `sum (distinct suma)`? This will benefit to not write second SELECT https://www.postgresql.org/message-id/CAN1PwonqojSAP_N91zO5Hm7Ta4Mdib-2YuUaEd0NP6Fn6Xutz

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 2:54 PM Amul Sul wrote: > > On Thu, May 13, 2021 at 12:36 PM Dilip Kumar wrote: > > > > On Wed, May 12, 2021 at 5:55 PM Amul Sul wrote: > > > > > > > Thanks for the updated patch, while going through I noticed this comment. > > > > + /* > > + * WAL prohibit state changes

Re: RFC: Logging plan of the running query

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 2:44 PM Dilip Kumar wrote: > +1 for the idea. I did not read the complete patch but while reading > through the patch, I noticed that you using elevel as LOG for printing > the stack trace. But I think the backend whose pid you have passed, > the connected client to that

Re: RFC: Logging plan of the running query

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy wrote: > On Thu, May 13, 2021 at 2:44 PM Dilip Kumar wrote: > > +1 for the idea. I did not read the complete patch but while reading > > through the patch, I noticed that you using elevel as LOG for printing > > the stack trace. But I think the

Re: RFC: Logging plan of the running query

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 1:56 PM torikoshia wrote: > > On 2021-05-13 01:08, Laurenz Albe wrote: > > On Wed, 2021-05-12 at 18:03 +0530, Bharath Rupireddy wrote: > >> Since it also shows up the full query text and the plan > >> in the server log as plain text, there are chances that the sensitive > >

subscriptioncheck failure

2021-05-13 Thread Amit Kapila
I noticed $SUBJECT in curculio. https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=2021-05-11%2018%3A30%3A23 The failure test report: t/020_messages.pl(Wstat: 7424 Tests: 1 Failed: 0) Non-zero exit status: 29 Parse errors: Bad plan. You planned 5 tests but ran

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 3:06 PM Bharath Rupireddy wrote: > > On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy > wrote: > > On Thu, May 13, 2021 at 2:44 PM Dilip Kumar wrote: > > > +1 for the idea. I did not read the complete patch but while reading > > > through the patch, I noticed that you u

Re: [HACKERS] logical decoding of two-phase transactions

2021-05-13 Thread Peter Smith
On Mon, May 10, 2021 at 1:31 PM vignesh C wrote: > > On Thu, Apr 29, 2021 at 2:23 PM Peter Smith wrote: > > > > Please find attached the latest patch set v74* > > > > Differences from v73* are: > > > > * Rebased to HEAD @ 2 days ago. > > > > * v74 addresses most of the feedback comments from Vign

Re: RFC: Logging plan of the running query

2021-05-13 Thread torikoshia
On 2021-05-13 18:36, Bharath Rupireddy wrote: On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy wrote: On Thu, May 13, 2021 at 2:44 PM Dilip Kumar wrote: > +1 for the idea. I did not read the complete patch but while reading > through the patch, I noticed that you using elevel as LOG for pri

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-13 Thread Amit Kapila
On Thu, May 13, 2021 at 11:15 AM osumi.takami...@fujitsu.com wrote: > > On Thursday, April 29, 2021 2:31 PM Amit Kapila > wrote: > > I am not so sure about it because I think we don't have any example of > > user_catalog_tables in the core code. This is the reason I was kind of > > looking > >

Re: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-13 Thread Amit Kapila
On Tue, Apr 20, 2021 at 8:36 AM Amit Langote wrote: > > On Sat, Apr 17, 2021 at 1:30 PM Amit Kapila wrote: > > On Fri, Apr 16, 2021 at 11:24 PM Andres Freund wrote:> > > > This made me take a brief look at pgoutput.c - maybe I am missing > > > something, but how is the following not a memory le

Re: subscriptioncheck failure

2021-05-13 Thread vignesh C
On Thu, May 13, 2021 at 3:12 PM Amit Kapila wrote: > > I noticed $SUBJECT in curculio. > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=curculio&dt=2021-05-11%2018%3A30%3A23 > > The failure test report: > t/020_messages.pl(Wstat: 7424 Tests: 1 Failed: 0) > Non-zero exit

Re: RFC: Logging plan of the running query

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 3:20 PM Dilip Kumar wrote: > > On Thu, May 13, 2021 at 3:06 PM Bharath Rupireddy > wrote: > > > > On Thu, May 13, 2021 at 2:57 PM Bharath Rupireddy > > wrote: > > > On Thu, May 13, 2021 at 2:44 PM Dilip Kumar wrote: > > > > +1 for the idea. I did not read the complete p

Re: Executor code - found an instance of a WHILE that should just be an IF

2021-05-13 Thread Michael Paquier
On Thu, May 13, 2021 at 08:20:36PM +1200, David Rowley wrote: > Since there's no bug fix here, I thought that there's not much point > in backpatching this. Indeed. I would not bother with a back-patch either. > Does anyone object to making this small change in master? No objections from here.

Re: Inherited UPDATE/DELETE vs async execution

2021-05-13 Thread Etsuro Fujita
On Thu, May 13, 2021 at 5:00 PM Etsuro Fujita wrote: > > On Thu, May 13, 2021 at 3:32 PM Amit Langote wrote: > > On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita > > wrote: > > > Here is a rebased version of the patch. I'm planning to apply this > > > tommorow. > > > > + /* > > +* Finally,

Re: subscriptioncheck failure

2021-05-13 Thread Michael Paquier
On Thu, May 13, 2021 at 04:14:55PM +0530, vignesh C wrote: > +$node_publisher->wait_for_catchup('tap_sub'); > + > # Ensure a transactional logical decoding message shows up on the slot > $node_subscriber->safe_psql('postgres', "ALTER SUBSCRIPTION tap_sub > DISABLE"); > > # wait for the replic

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy wrote: > > I'm saying that - currently, queries are logged with LOG level when > the log_statement GUC is set. The queries might be sent to the > non-superuser clients. So, your point of "sending the plan to those > clients is not a good idea from

Re: RFC: Logging plan of the running query

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > wrote: > > > > I'm saying that - currently, queries are logged with LOG level when > > the log_statement GUC is set. The queries might be sent to the > > non-superuser clients. So, your poi

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy wrote: > > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > > wrote: > > > > > > I'm saying that - currently, queries are logged with LOG level when > > > the log_statement GUC is set

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 04:27:47PM +0900, Michael Paquier wrote: > On Tue, May 11, 2021 at 04:42:27PM +0900, Michael Paquier wrote: > > Whatever the solution chosen, the thing I can see we agree on here is > > that we need to do something, at least in the shape of an on/off > > switch to have an es

Re: Inherited UPDATE/DELETE vs async execution

2021-05-13 Thread Amit Langote
On Thu, May 13, 2021 at 8:10 PM Etsuro Fujita wrote: > On Thu, May 13, 2021 at 5:00 PM Etsuro Fujita wrote: > > On Thu, May 13, 2021 at 3:32 PM Amit Langote > > wrote: > > > On Wed, May 12, 2021 at 6:45 PM Etsuro Fujita > > > wrote: > > > > Here is a rebased version of the patch. I'm plannin

Re: subscriptioncheck failure

2021-05-13 Thread vignesh C
On Thu, May 13, 2021 at 4:41 PM Michael Paquier wrote: > > On Thu, May 13, 2021 at 04:14:55PM +0530, vignesh C wrote: > > +$node_publisher->wait_for_catchup('tap_sub'); > > + > > # Ensure a transactional logical decoding message shows up on the slot > > $node_subscriber->safe_psql('postgres', "A

Re: Executor code - found an instance of a WHILE that should just be an IF

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 08:06:18PM +0900, Michael Paquier wrote: > On Thu, May 13, 2021 at 08:20:36PM +1200, David Rowley wrote: > > Since there's no bug fix here, I thought that there's not much point > > in backpatching this. > > Indeed. I would not bother with a back-patch either. > > > Does

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 5:18 PM Dilip Kumar wrote: > > On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy > wrote: > > > > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > > > > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > > > wrote: > > > > > > > > I'm saying that - currently, q

Re: JSON doc example (matchiness)

2021-05-13 Thread Alexander Korotkov
On Mon, May 10, 2021 at 3:41 AM Michael Paquier wrote: > On Sun, May 09, 2021 at 11:17:47PM +0300, Alexander Korotkov wrote: > > I propose backpatching this to 12 when jsonpath was introduced. It > > seems useful to have this docs improvement every release supporting > > jsonpath. > > > > Objecti

Re: parallel vacuum - few questions on docs, comments and code

2021-05-13 Thread Masahiko Sawada
On Thu, May 13, 2021 at 3:25 PM Amit Kapila wrote: > > On Wed, May 12, 2021 at 6:37 PM Bharath Rupireddy > wrote: > > > > On Wed, May 12, 2021 at 11:10 AM Amit Kapila > > wrote: > > > > > > On Tue, May 11, 2021 at 5:38 PM Bharath Rupireddy > > > wrote: > > > > > > > > I was going through the p

Re: subscriptioncheck failure

2021-05-13 Thread vignesh C
On Thu, May 13, 2021 at 4:41 PM Michael Paquier wrote: > > On Thu, May 13, 2021 at 04:14:55PM +0530, vignesh C wrote: > > +$node_publisher->wait_for_catchup('tap_sub'); > > + > > # Ensure a transactional logical decoding message shows up on the slot > > $node_subscriber->safe_psql('postgres', "A

Explicit NULL dereference (src/backend/commands/tablecmds.c)

2021-05-13 Thread Ranier Vilela
Hi, Per Coverity. CID 1453114 (#1 of 1): Explicit null dereferenced (FORWARD_NULL) 53. var_deref_model: Passing null pointer child_expr to strcmp, which dereferences it. It is agreed that asserts should be used for error conditions that can never occur in the release. But with errors that can occ

Re: Explicit NULL dereference (src/backend/commands/tablecmds.c)

2021-05-13 Thread Tom Lane
Ranier Vilela writes: > It is agreed that asserts should be used for error conditions that can > never occur in the release. > But with errors that can occur, using assert does not make sense. On what grounds do you claim that those asserts are wrong? Coverity's opinion counts for just about not

Re: alter subscription drop publication fixes

2021-05-13 Thread vignesh C
On Wed, May 12, 2021 at 10:15 PM Bharath Rupireddy wrote: > > On Wed, May 12, 2021 at 9:55 PM vignesh C wrote: > > While I was reviewing one of the logical decoding features, I found a > > few issues in alter subscription drop publication. > > Thanks! > > > Alter subscription drop publication doe

Re: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-13 Thread Greg Nancarrow
On Thu, May 13, 2021 at 11:25 AM Pengchengliu wrote: > > Hi Andres, > Thanks for you replay. Er, it's Greg who has replied so far (not Andres). > > And If you still cannot reproduce it in 2 minitus. Could you run pgbench > longer time, such as 30 or 60 minutes. > Actually, I did run it, mu

Re: Enhanced error message to include hint messages for redundant options error

2021-05-13 Thread vignesh C
On Thu, May 13, 2021 at 4:58 AM Alvaro Herrera wrote: > > You can avoid duplicating the ereport like this: > > + ereport(ERROR, > + (errcode(ERRCODE_SYNTAX_ERROR), > +errmsg("option \"%s\" specified more than > once", def

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 12:03:42PM +0800, Julien Rouhaud wrote: > On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: > I don't know what to say. So here is a summary of the complaints that I'm > aware of: > > - > https://www.postgresql.org/message-id/1953aec168224b95b0c962a622bef0794

Re: alter subscription drop publication fixes

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 7:43 PM vignesh C wrote: > I have separated the Drop publication documentation contents. There > are some duplicate contents but the readability is slightly better. > Thoughts? -ALTER SUBSCRIPTION name DROP PUBLICATION publication_name [, ...] [ WITH ( set_publication_opti

Re: PG 14 release notes, first draft

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 02:46:58PM +0900, Amit Langote wrote: > How about writing the 2nd line instead as: > > Updates/deletes on partitioned tables can now use execution-time > partition pruning. > > We don't seem to use the term "run-time pruning" anywhere else in the > documentation, and "prun

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
Pavel Borisov writes: > [ v4-0001-When-there-are-INCLUDEd-columns-in-SpGist-check-t.patch ] I don't like this patch one bit --- it's basically disabling a fairly important SPGiST feature as soon as there are included columns. What's more, it's not really giving us any better defense against the i

Re: parallel vacuum - few questions on docs, comments and code

2021-05-13 Thread Bharath Rupireddy
On Thu, May 13, 2021 at 7:00 PM Masahiko Sawada wrote: > > > Yeah, I get it. Even if users don't specify a parallel option there > > > are chances that parallelism is picked. So the parallel degree is the > > > final number of workers that are chosen by the server for vacuuming > > > indexes. And,

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Maciek Sakrejda
On Thu, May 13, 2021 at 7:42 AM Bruce Momjian wrote: > > On Thu, May 13, 2021 at 12:03:42PM +0800, Julien Rouhaud wrote: > > On Wed, May 12, 2021 at 11:33:32PM -0400, Bruce Momjian wrote: > > I don't know what to say. So here is a summary of the complaints that I'm > > aware of: > > > > - > > ht

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 10:41:43AM -0400, Bruce Momjian wrote: > > Well, now that we have clear warnings when it is misconfigured, > especially when querying the pg_stat_statements view, are these > complaints still valid? I'm personally fine with it, and I can send a new version with just the wa

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > The warning makes it clear that there's something wrong, but not how > to fix it I'm confused, are we talking about the new warning in v2 as suggested by Pavel? As it should make things quite clear: +SELECT count(*) FROM pg_sta

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > Well, now that we have clear warnings when it is misconfigured, > > especially when querying the pg_stat_statements view, are these > > complaints still valid? Also, how is modifying > > shared_preload_libraries zero-config, but

Re: OOM in spgist insert

2021-05-13 Thread Alvaro Herrera
On 2021-May-13, Tom Lane wrote: > BTW, another nasty thing I discovered while testing this is that > the CHECK_FOR_INTERRUPTS() at line 2146 is useless, because > we're holding a buffer lock there so InterruptHoldoffCount > 0. > So once you get into this loop you can't even cancel the query. > See

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 11:35:13PM +0800, Julien Rouhaud wrote: > On Thu, May 13, 2021 at 10:41:43AM -0400, Bruce Momjian wrote: > > > > Well, now that we have clear warnings when it is misconfigured, > > especially when querying the pg_stat_statements view, are these > > complaints still valid? >

Re: Granting control of SUSET gucs to non-superusers

2021-05-13 Thread Mark Dilger
> On May 12, 2021, at 12:58 PM, Robert Haas wrote: > > - Group things by which section of postgresql.conf they're in, and > then further restrict some of them as security-sensitive. This is > reasonably close to what you've got, but not exactly what you've got. > One issue is that it risks sep

Re: OOM in spgist insert

2021-05-13 Thread Alvaro Herrera
On 2021-May-13, Alvaro Herrera wrote: > The btree code modified was found to be an actual problem in production > when a btree is corrupted in such a way that vacuum would get an > infinite loop. I don't remember the exact details but I think we saw > vacuum running for a couple of weeks, and had

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Maciek Sakrejda
On Thu, May 13, 2021 at 8:38 AM Julien Rouhaud wrote: > > On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > > > The warning makes it clear that there's something wrong, but not how > > to fix it > > I'm confused, are we talking about the new warning in v2 as suggested by > Pave

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
Alvaro Herrera writes: > On 2021-May-13, Tom Lane wrote: >> BTW, another nasty thing I discovered while testing this is that >> the CHECK_FOR_INTERRUPTS() at line 2146 is useless, because >> we're holding a buffer lock there so InterruptHoldoffCount > 0. >> So once you get into this loop you can't

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 09:30:55AM -0700, Maciek Sakrejda wrote: > On Thu, May 13, 2021 at 8:38 AM Julien Rouhaud wrote: > > > > On Thu, May 13, 2021 at 08:32:50AM -0700, Maciek Sakrejda wrote: > > > > > > The warning makes it clear that there's something wrong, but not how > > > to fix it > > > >

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Andrew Dunstan
On 5/13/21 12:18 AM, Fujii Masao wrote: > > > > If we do this, compute_query_id=auto seems to be similar to > huge_pages=try. > When huge_pages=try, whether huge pages is actually used is defined > depending > outside PostgreSQL (i.e, OS setting in this case). Neither > pg_settings.setting nor >

amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Justin Pryzby
Per sqlsmith. postgres=# select amvalidate(123); ERROR: cache lookup failed for operator class 123 postgres=# \errverbose ERROR: XX000: cache lookup failed for operator class 123 LOCATION: amvalidate, amapi.c:125 The usual expectation is that sql callable functions should return null rather t

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 12:45:07PM -0400, Andrew Dunstan wrote: > > On 5/13/21 12:18 AM, Fujii Masao wrote: > > > > > > > > If we do this, compute_query_id=auto seems to be similar to > > huge_pages=try. > > When huge_pages=try, whether huge pages is actually used is defined > > depending > > outs

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
Andrew Dunstan writes: > The only thing that bugs me is that we're pretty damn late in the > process to be engaging in this amount of design. Indeed. I feel that this feature was forced in before it was really ready. regards, tom lane

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
Alvaro Herrera writes: > (Looking again, the nbtpage.c hunk might have been made obsolete by > c34787f91058 and other commits). OK. Here's a revision that adopts your idea, except that I left out the nbtpage.c change since you aren't sure of that part. I added a macro that allows spgdoinsert to

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Andrew Dunstan writes: > > The only thing that bugs me is that we're pretty damn late in the > > process to be engaging in this amount of design. > > Indeed. I feel that this feature was forced in before it was really > ready. I'm coming arou

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 01:17:16PM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > The only thing that bugs me is that we're pretty damn late in the > > process to be engaging in this amount of design. > > Indeed. I feel that this feature was forced in before it was really > ready. The user

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Christoph Berg
Re: Bruce Momjian > Well, now that we have clear warnings when it is misconfigured, > especially when querying the pg_stat_statements view, are these > complaints still valid? Also, how is modifying > shared_preload_libraries zero-config, but modifying > shared_preload_libraries and compute_query

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 01:33:27PM -0400, Stephen Frost wrote: > I'm coming around to have a similar feeling. While having an > alternative query ID might be useful, I have a hard time seeing it as > likely to be a hugely popular feature that is worth a lot of users > complaining (as we've seen al

Re: Granting control of SUSET gucs to non-superusers

2021-05-13 Thread Stephen Frost
Greetings, * Mark Dilger (mark.dil...@enterprisedb.com) wrote: > > On May 12, 2021, at 12:58 PM, Robert Haas wrote: > > - Group things by which section of postgresql.conf they're in, and > > then further restrict some of them as security-sensitive. This is > > reasonably close to what you've got,

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, May 13, 2021 at 01:33:27PM -0400, Stephen Frost wrote: > > I'm coming around to have a similar feeling. While having an > > alternative query ID might be useful, I have a hard time seeing it as > > likely to be a hugely popular featur

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 01:51:07PM -0400, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Thu, May 13, 2021 at 01:33:27PM -0400, Stephen Frost wrote: > > > I'm coming around to have a similar feeling. While having an > > > alternative query ID might be usefu

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 07:39:45PM +0200, Christoph Berg wrote: > Re: Bruce Momjian > > Well, now that we have clear warnings when it is misconfigured, > > especially when querying the pg_stat_statements view, are these > > complaints still valid? Also, how is modifying > > shared_preload_librari

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Thu, May 13, 2021 at 07:39:45PM +0200, Christoph Berg wrote: > > Re: Bruce Momjian > > > Well, now that we have clear warnings when it is misconfigured, > > > especially when querying the pg_stat_statements view, are these > > > complaints

Re: amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Tom Lane
Justin Pryzby writes: > Per sqlsmith. > postgres=# select amvalidate(123); > ERROR: cache lookup failed for operator class 123 > postgres=# \errverbose > ERROR: XX000: cache lookup failed for operator class 123 > LOCATION: amvalidate, amapi.c:125 > The usual expectation is that sql callable f

Re: amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 02:22:16PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > Per sqlsmith. > > postgres=# select amvalidate(123); > > ERROR: cache lookup failed for operator class 123 > > postgres=# \errverbose > > ERROR: XX000: cache lookup failed for operator class 123 > > LOCATION:

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
Stephen Frost writes: > There's a ridiculously simple option here which is: drop the idea that > we support an extension redefining the query id and then just make it > on/off with the default to be 'on'. I do not think that defaulting it to 'on' is acceptable unless you can show that the added o

Re: Granting control of SUSET gucs to non-superusers

2021-05-13 Thread Mark Dilger
> On May 13, 2021, at 10:41 AM, Stephen Frost wrote: > > Greetings, > > * Mark Dilger (mark.dil...@enterprisedb.com) wrote: >>> On May 12, 2021, at 12:58 PM, Robert Haas wrote: >>> - Group things by which section of postgresql.conf they're in, and >>> then further restrict some of them as se

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > There's a ridiculously simple option here which is: drop the idea that > > we support an extension redefining the query id and then just make it > > on/off with the default to be 'on'. > > I do not think that defaultin

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Alvaro Herrera
On 2021-May-13, Stephen Frost wrote: > Or just accept that this is a bit hokey with the 'auto' approach. I get > Bruce has concerns about it but I'm not convinced that it's actually all > that bad to go with that. Yeah, I think the alleged confusion there is overstated. I'm happy to act as comm

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 02:29:09PM -0400, Tom Lane wrote: > Stephen Frost writes: > > There's a ridiculously simple option here which is: drop the idea that > > we support an extension redefining the query id and then just make it > > on/off with the default to be 'on'. > > I do not think that de

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 03:04:30PM -0400, Álvaro Herrera wrote: > On 2021-May-13, Stephen Frost wrote: > > > Or just accept that this is a bit hokey with the 'auto' approach. I get > > Bruce has concerns about it but I'm not convinced that it's actually all > > that bad to go with that. > > Yeah

Re: Granting control of SUSET gucs to non-superusers

2021-05-13 Thread Jacob Champion
On Thu, 2021-05-13 at 11:42 -0700, Mark Dilger wrote: > The distinction that Theme+Security would make is that capabilities > can be categorized by the area of the system: > -- planner > -- replication > -- logging > ... > but also by the security implications of what is being done: > --

Re: Granting control of SUSET gucs to non-superusers

2021-05-13 Thread Stephen Frost
Greetings, * Jacob Champion (pchamp...@vmware.com) wrote: > On Thu, 2021-05-13 at 11:42 -0700, Mark Dilger wrote: > > The distinction that Theme+Security would make is that capabilities > > can be categorized by the area of the system: > > -- planner > > -- replication > > -- logging > > .

Re: Granting control of SUSET gucs to non-superusers

2021-05-13 Thread Mark Dilger
> On May 13, 2021, at 12:18 PM, Jacob Champion wrote: > > On Thu, 2021-05-13 at 11:42 -0700, Mark Dilger wrote: >> The distinction that Theme+Security would make is that capabilities >> can be categorized by the area of the system: >> -- planner >> -- replication >> -- logging >> ... >> bu

Re: amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Robert Haas
On Thu, May 13, 2021 at 2:22 PM Tom Lane wrote: > Meh. I'm not convinced that that position ought to apply to amvalidate. I am still of the opinion that we ought to apply it across the board, for consistency. It makes it easier for humans to know which problems are known to be reachable and whic

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Peter Geoghegan
On Thu, May 13, 2021 at 5:06 AM Justin Pryzby wrote: > Why is the allowed range from 0 to 0.05? Why not 0.10 or 1.0 ? > The old GUC of the same name had max 1e10. It also had a completely different purpose and default. > I think a reduced range and a redefinition of the GUC would need to be cal

Re: amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Tom Lane
Robert Haas writes: > On Thu, May 13, 2021 at 2:22 PM Tom Lane wrote: >> Meh. I'm not convinced that that position ought to apply to amvalidate. > I am still of the opinion that we ought to apply it across the board, > for consistency. The main reason I'm concerned about applying that rule to

Re: Race condition in recovery?

2021-05-13 Thread Robert Haas
On Mon, May 10, 2021 at 4:35 AM Kyotaro Horiguchi wrote: > It seems to me the issue here is not a race condition but > WaitForWALToBecomeAvailable initializing expectedTLEs with the history > of a improper timeline. So using recoveryTargetTLI instead of > receiveTLI for the case fixes this issue.

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-05-13 Thread Alvaro Herrera
New version, a bit more ambitious. I think it's better to describe behavior for partitioned tables ahead of inheritance. Also, in the ANALYZE reference page I split the topic in two: in one single paragraph we now describe what happens with manual analyze for partitioned tables and inheritance hi

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2021-05-13 Thread Michail Nikolaev
Hello. Added a check for standby promotion with the long transaction to the test (code and docs are unchanged). Thanks, Michail. From c5e1053805c537b50b0922151bcf127754500adb Mon Sep 17 00:00:00 2001 From: Michail Nikolaev Date: Fri, 14 May 2021 00:32:30 +0300 Subject: [PATCH v3 3/4] test ---

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2021-05-13 Thread Dmitry Astapov
On Wed, May 12, 2021 at 4:54 PM Tom Lane wrote: > Dmitry Astapov writes: > > I am trying to understand the behaviour of the query planner regarding > the > > push-down of the conditions "through" the join. > > I think your mental model is wrong. What's actually happening here is > that the plan

Re: Always bump PG_CONTROL_VERSION?

2021-05-13 Thread Jan Wieck
On 5/12/21 10:04 PM, Michael Paquier wrote: On Wed, May 12, 2021 at 01:30:27PM -0700, Andres Freund wrote: That said, I don't think it's a good practice to use the control file version as an identifier for the major version. Who knows, it might be necessary to add an optional new format in a min

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
Here's a patch that attempts to prevent the infinite loop in spgdoinsert by checking whether the proposed leaf tuple is getting smaller at each iteration. We can't be totally rigid about that, because for example if the opclass shortens a 7-byte string to 5 bytes, that will make no difference in t

Re: OOM in spgist insert

2021-05-13 Thread Alvaro Herrera
On 2021-May-13, Tom Lane wrote: > Alvaro Herrera writes: > > (Looking again, the nbtpage.c hunk might have been made obsolete by > > c34787f91058 and other commits). > > OK. Here's a revision that adopts your idea, except that I left > out the nbtpage.c change since you aren't sure of that part

Re: Always bump PG_CONTROL_VERSION?

2021-05-13 Thread Tom Lane
Jan Wieck writes: > Regardless of PG_VERSION doing the job or not, shouldn't there be a bump > in PG_CONTROL_VERSION whenever there is a structural or semantic change > in the control file data? And wouldn't the easiest way to ensure that be > to bump the version with every release? No, the wa

Re: OOM in spgist insert

2021-05-13 Thread Alvaro Herrera
On 2021-May-13, Tom Lane wrote: > What do people think about back-patching this? In existing branches, > we've defined it to be an opclass bug if it fails to shorten the leaf > datum enough. But not having any defenses against that seems like > not a great idea. OTOH, the 10-cycles-to-show-prog

Re: OOM in spgist insert

2021-05-13 Thread Pavel Borisov
I think it's good to backpatch the check as it doesn't change acceptable behavior, just replace infinite loop with the acceptable thing.

Re: Always bump PG_CONTROL_VERSION?

2021-05-13 Thread Jan Wieck
On 5/13/21 6:45 PM, Tom Lane wrote: Bumping the version in the commit that changes things is not optional, because if you don't do that then you'll probably burn some other developer also working on HEAD. So I don't want people thinking they can skip this because it was done at the

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-05-13 Thread Alvaro Herrera
With English fixes from Bruce. I think the note about autovacuum in the reference page for ANALYZE is a bit out of place, but not enough to make me move the whole paragraph elsewhere. Maybe we should try to do that sometime. -- Álvaro Herrera Valdivia, Chile Officer Krupke, what are we to

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
Alvaro Herrera writes: > Hmm, it looks OK to me, but I wonder why you kept the original > CHECK_FOR_INTERRUPTS()s since these would be done once we've broken out > of the loop anyway. I tested Dilip's original test case and while we > still die on OOM, we're able to interrupt it before dying. Hm

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Andrew Dunstan
On 5/13/21 3:04 PM, Alvaro Herrera wrote: > On 2021-May-13, Stephen Frost wrote: > >> Or just accept that this is a bit hokey with the 'auto' approach. I get >> Bruce has concerns about it but I'm not convinced that it's actually all >> that bad to go with that. > Yeah, I think the alleged confu

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2021-05-13 Thread Tom Lane
Dmitry Astapov writes: > Am I right in thinking that elimination the join condition is actually > quite important part of the process? > Could it possibly be the main reason for =ANY/(x IN (..)) not to be > optimized the same way? Yup. > Is it still hard when one thinks about =ANY or (column in

  1   2   >