Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 07:19:11PM -0400, Andrew Dunstan wrote: > > On 5/13/21 3:04 PM, Alvaro Herrera wrote: > > On 2021-May-13, Stephen Frost wrote: > > > >> Or just accept that this is a bit hokey with the 'auto' approach. I get > >> Bruce has concerns about it but I'm not convinced that it's

Re: Always bump PG_CONTROL_VERSION?

2021-05-13 Thread Tom Lane
Jan Wieck writes: > On 5/13/21 6:45 PM, Tom Lane wrote: >> Bumping the version in the commit that changes >> things is not optional, because if you don't do that then you'll >> probably burn some other developer also working on HEAD. So >> I don't want people thinking they can skip this because i

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 05:33:33PM -0400, Alvaro Herrera wrote: > +++ b/doc/src/sgml/maintenance.sgml > @@ -817,6 +817,11 @@ analyze threshold = analyze base threshold + analyze > scale factor * number of tu > > is compared to the total number of tuples inserted, updated, or deleted >

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Michael Paquier
On Thu, May 13, 2021 at 07:19:11PM -0400, Andrew Dunstan wrote: > On 5/13/21 3:04 PM, Alvaro Herrera wrote: >> I'm happy to act as committer for that if he wants to step away from it. >> I'm already used to being lapidated at every corner anyway. > > Many thanks Alvaro, among other things for teac

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Alvaro Herrera
Here's a first attempt at what was suggested. If you say "auto" it remains auto in SHOW, but it gets enabled if a module asks for it. Not final yet, but I thought I'd throw it out for early commentary ... -- Álvaro Herrera Valdivia, Chile diff --git a/contrib/pg_stat_statements/expected/p

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > Here's a first attempt at what was suggested. If you say "auto" it > remains auto in SHOW, but it gets enabled if a module asks for it. > > Not final yet, but I thought I'd throw it out for early commentary ... I certainly like th

Re: Executor code - found an instance of a WHILE that should just be an IF

2021-05-13 Thread David Rowley
On Fri, 14 May 2021 at 00:27, Julien Rouhaud wrote: > > On Thu, May 13, 2021 at 08:06:18PM +0900, Michael Paquier wrote: > > On Thu, May 13, 2021 at 08:20:36PM +1200, David Rowley wrote: > > > Since there's no bug fix here, I thought that there's not much point > > > in backpatching this. > > > >

Re: Executor code - found an instance of a WHILE that should just be an IF

2021-05-13 Thread Greg Nancarrow
On Fri, May 14, 2021 at 10:27 AM David Rowley wrote: > > Thanks for the votes. Pushed. Thanks! Regards, Greg Nancarrow Fujitsu Australia

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
Alvaro Herrera writes: > On 2021-May-13, Tom Lane wrote: >> What do people think about back-patching this? In existing branches, >> we've defined it to be an opclass bug if it fails to shorten the leaf >> datum enough. But not having any defenses against that seems like >> not a great idea. OTO

Re: Testing autovacuum wraparound (including failsafe)

2021-05-13 Thread Peter Geoghegan
On Fri, Apr 23, 2021 at 7:56 PM Peter Geoghegan wrote: > I'm convinced -- decoupling the logic from the one-pass-not-two pass > case seems likely to be simpler and more useful. For both the one pass > and two pass/has indexes case. Attached draft patch does it that way. -- Peter Geoghegan v1-

RE: Support for VACUUMing Foreign Tables

2021-05-13 Thread tsunakawa.ta...@fujitsu.com
From: Bharath Rupireddy > I think it will be useful to allow foreign tables to be VACUUMed if > the underlying FDW supports, currently VACUUM doesn't support foreign > tables, see [1]. Could you let us imagine more concretely how useful it will be? While TRUNCATE can be part of an application's

Re: What is lurking in the shadows?

2021-05-13 Thread David Rowley
On Fri, 14 May 2021 at 12:00, Peter Smith wrote: > That bug led me to wonder if similar problems might be going > undetected elsewhere in the code. There is a gcc compiler option [3] > -Wshadow which informs about the similar scenario where one variable > is "shadowing" another (e.g. redeclaring a

Re: PG 14 release notes, first draft

2021-05-13 Thread Bruce Momjian
On Mon, May 10, 2021 at 02:03:08AM -0400, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.us/pgsql_docs/release-14.html > > I need clarification on many items, and the document still

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Fri, May 14, 2021 at 3:12 AM Bruce Momjian wrote: > > > Maybe we should revert this thing pending somebody doing the work to > > make a version of queryid labeling that actually is negligibly cheap. > > It certainly seems like that could be done; one more traversal of the > > parse tree can't b

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Fri, May 14, 2021 at 8:13 AM Bruce Momjian wrote: > > On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > > Here's a first attempt at what was suggested. If you say "auto" it > > remains auto in SHOW, but it gets enabled if a module asks for it. > > > > Not final yet, but I thoug

Re: PG 14 release notes, first draft

2021-05-13 Thread Amit Langote
On Thu, May 13, 2021 at 11:59 PM Bruce Momjian wrote: > On Thu, May 13, 2021 at 02:46:58PM +0900, Amit Langote wrote: > > How about writing the 2nd line instead as: > > > > Updates/deletes on partitioned tables can now use execution-time > > partition pruning. > > > > We don't seem to use the term

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Bruce Momjian
On Fri, May 14, 2021 at 09:40:15AM +0800, Julien Rouhaud wrote: > On Fri, May 14, 2021 at 8:13 AM Bruce Momjian wrote: > > > > On Thu, May 13, 2021 at 08:04:37PM -0400, Álvaro Herrera wrote: > > > Here's a first attempt at what was suggested. If you say "auto" it > > > remains auto in SHOW, but i

Re: OOM in spgist insert

2021-05-13 Thread Tom Lane
I wrote: > Anyway, here is a patch set teased apart into committable bites, > and with your other points addressed. Oh, maybe some docs would be a good thing too ... regards, tom lane diff --git a/doc/src/sgml/spgist.sgml b/doc/src/sgml/spgist.sgml index cfb2b3c836..18f1f

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
Julien Rouhaud writes: > On Fri, May 14, 2021 at 3:12 AM Bruce Momjian wrote: >> I was surprised it was ~2%. > Just to be clear, the 2% was a worst case scenario, ie. a very fast > read-only query on small data returning a single row. As soon as you > get something more realistic / expensive th

Re: PG 14 release notes, first draft

2021-05-13 Thread Justin Pryzby
On Mon, May 10, 2021 at 09:40:45AM -0500, Justin Pryzby wrote: > Should any of these be included? New SQL-accessible functionality should be included: > 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. These should be merged: > 756ab29124 Allow pageinspect to inspect G

Re: PG 14 release notes, first draft

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > These should be merged: > > 756ab29124 Allow pageinspect to inspect GiST indexes (Andrey Borodin, > > Heikki Linnakangas) > > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions Sorry, this was my error while reconci

Re: alter subscription drop publication fixes

2021-05-13 Thread Japin Li
On Thu, 13 May 2021 at 22:13, vignesh C wrote: > On Wed, May 12, 2021 at 10:15 PM Bharath Rupireddy > wrote: >> >> On Wed, May 12, 2021 at 9:55 PM vignesh C wrote: >> > While I was reviewing one of the logical decoding features, I found a >> > few issues in alter subscription drop publication.

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Michael Paquier
On Thu, May 13, 2021 at 01:27:44PM -0700, Peter Geoghegan wrote: > Almost all of the benefit of the optimization is available with the > current BYPASS_THRESHOLD_PAGES threshold (2% of heap pages have > LP_DEAD items), which has less risk than a higher threshold. I don't > think it matters much if

RE: Forget close an open relation in ReorderBufferProcessTXN()

2021-05-13 Thread osumi.takami...@fujitsu.com
On Thursday, May 13, 2021 7:43 PM Amit Kapila wrote: > On Tue, Apr 20, 2021 at 8:36 AM Amit Langote > wrote: > > Back in: > https://www.postgresql.org/message-id/CA%2BHiwqEeU19iQgjN6HF1HTP > U0L5%2 > > BJxyS5CmxaOVGNXBAfUY06Q%40mail.gmail.com > > > > I had proposed to move the map creation from m

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Thu, May 13, 2021 at 09:47:02PM -0400, Tom Lane wrote: > Julien Rouhaud writes: > > On Fri, May 14, 2021 at 3:12 AM Bruce Momjian wrote: > >> I was surprised it was ~2%. > > > Just to be clear, the 2% was a worst case scenario, ie. a very fast > > read-only query on small data returning a sin

RE: Parallel scan with SubTransGetTopmostTransaction assert coredump

2021-05-13 Thread Pengchengliu
Hi Greg, Thanks for you replay and test. When main process gets the transaction snapshot in InitializeParallelDSM->GetTransactionSnapshot, the transaction snapshot xmin is very likely follows active snapshot xmin. Use gdb it is easy to verify it. Create env as blow: 1,

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Peter Geoghegan
On Thu, May 13, 2021 at 7:14 PM Michael Paquier wrote: > Perhaps that's an awful deal, but based on which facts can you really > say that this new behavior of needing at least 2% of relation pages > with some dead items to clean up indexes is not a worse deal in some > cases? If I thought that it

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Fujii Masao
On 2021/05/14 9:04, Alvaro Herrera wrote: Here's a first attempt at what was suggested. If you say "auto" it remains auto in SHOW, but it gets enabled if a module asks for it. Not final yet, but I thought I'd throw it out for early commentary ... Many thanks! The patch basically looks good

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Julien Rouhaud
On Fri, May 14, 2021 at 12:20:00PM +0900, Fujii Masao wrote: > > > On 2021/05/14 9:04, Alvaro Herrera wrote: > > Here's a first attempt at what was suggested. If you say "auto" it > > remains auto in SHOW, but it gets enabled if a module asks for it. > > > > Not final yet, but I thought I'd thr

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2021-05-13 Thread David Rowley
On Fri, 14 May 2021 at 11:22, Tom Lane wrote: > I recall somebody (David Rowley, maybe? Too lazy to check archives.) > working on this idea awhile ago, but he didn't get to the point of > a committable patch. Yeah. Me. The discussion is in [1]. David [1] https://www.postgresql.org/message-id/

Re: compute_query_id and pg_stat_statements

2021-05-13 Thread Tom Lane
I wrote: > Maybe we should revert this thing pending somebody doing the work to > make a version of queryid labeling that actually is negligibly cheap. > It certainly seems like that could be done; one more traversal of the > parse tree can't be that expensive in itself. I suspect that the > perfo

Re: Race condition in recovery?

2021-05-13 Thread Dilip Kumar
On Fri, May 14, 2021 at 2:37 AM Robert Haas wrote: > > So why does this use recoveryTargetTLI instead of receiveTLI only > conditionally? Why not do it all the time? > > The hard thing about this code is that the assumptions are not very > clear. If we don't know why something is a certain way, th

Re: Race condition in recovery?

2021-05-13 Thread Kyotaro Horiguchi
At Thu, 13 May 2021 17:07:31 -0400, Robert Haas wrote in > On Mon, May 10, 2021 at 4:35 AM Kyotaro Horiguchi > wrote: > > It seems to me the issue here is not a race condition but > > WaitForWALToBecomeAvailable initializing expectedTLEs with the history > > of a improper timeline. So using rec

Re: parallel vacuum - few questions on docs, comments and code

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 9:00 PM Bharath Rupireddy wrote: > > Done that way. > > PSA patch. Your changes look good. About changing the "non-negative integer" to "greater than or equal to zero", there is another thread [1], I am not sure that have we concluded anything there yet. - pg_log_error("

Re: Race condition in recovery?

2021-05-13 Thread Kyotaro Horiguchi
At Fri, 14 May 2021 14:12:31 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 13 May 2021 17:07:31 -0400, Robert Haas wrote > in > > On Mon, May 10, 2021 at 4:35 AM Kyotaro Horiguchi > > wrote: > > > It seems to me the issue here is not a race condition but > > > WaitForWALToBecomeAvailable

Re: OOM in spgist insert

2021-05-13 Thread Dilip Kumar
On Fri, May 14, 2021 at 6:31 AM Tom Lane wrote: > > Alvaro Herrera writes: > > On 2021-May-13, Tom Lane wrote: > >> What do people think about back-patching this? In existing branches, > >> we've defined it to be an opclass bug if it fails to shorten the leaf > >> datum enough. But not having a

Re: subscriptioncheck failure

2021-05-13 Thread Michael Paquier
On Thu, May 13, 2021 at 07:05:55PM +0530, vignesh C wrote: > Thanks for the comments, Please find the attached patch having the changes. Cool, thanks for the new version. I have spent some time understanding the initial report from Amit as well as what you are proposing here, and refactoring the

Re: Support for VACUUMing Foreign Tables

2021-05-13 Thread Michael Paquier
On Fri, May 14, 2021 at 01:05:02AM +, tsunakawa.ta...@fujitsu.com wrote: > Could you let us imagine more concretely how useful it will be? > While TRUNCATE can be part of an application's data processing as > alternative to DELETE, I think VACUUM is purely the data storage > maintenance that's

Re: Some doubious error messages and comments

2021-05-13 Thread Kyotaro Horiguchi
At Mon, 10 May 2021 15:52:15 +0900, Michael Paquier wrote in > On Wed, Apr 28, 2021 at 08:11:47AM -0500, Justin Pryzby wrote: > > On Wed, Apr 28, 2021 at 05:36:33PM +0900, Kyotaro Horiguchi wrote: > >> 0001: I found some typos in a error message and a comment. > >> > >> multirangetypes.c: 1420

Re: Support for VACUUMing Foreign Tables

2021-05-13 Thread Dilip Kumar
On Fri, May 14, 2021 at 6:35 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Bharath Rupireddy > > I think it will be useful to allow foreign tables to be VACUUMed if > > the underlying FDW supports, currently VACUUM doesn't support foreign > > tables, see [1]. > > Could you let us imagine more c

Re: What is lurking in the shadows?

2021-05-13 Thread Michael Paquier
On Fri, May 14, 2021 at 01:16:37PM +1200, David Rowley wrote: > I'm inclined to think that since a bug has already been found due to a > local variable shadowing a global one that it would be good to review > these and then consider if it's worth doing any renaming. I think the > process of looking

Re: Support for VACUUMing Foreign Tables

2021-05-13 Thread Bharath Rupireddy
On Fri, May 14, 2021 at 6:35 AM tsunakawa.ta...@fujitsu.com wrote: > > From: Bharath Rupireddy > > I think it will be useful to allow foreign tables to be VACUUMed if > > the underlying FDW supports, currently VACUUM doesn't support foreign > > tables, see [1]. > > Could you let us imagine more c

Added missing tab completion for alter subscription set option

2021-05-13 Thread vignesh C
Hi, While I was reviewing one of the logical decoding features, I found Streaming and binary options were missing in tab completion for the alter subscription set option, the attached patch has the changes for the same. Thoughts? Regards, Vignesh From 84a0c07760c913576ef38d74ab37ffd9ee3ad686 Mon

Re: Support for VACUUMing Foreign Tables

2021-05-13 Thread Bharath Rupireddy
On Fri, May 14, 2021 at 11:48 AM Dilip Kumar wrote: > > On Fri, May 14, 2021 at 6:35 AM tsunakawa.ta...@fujitsu.com > wrote: > > > > From: Bharath Rupireddy > > > I think it will be useful to allow foreign tables to be VACUUMed if > > > the underlying FDW supports, currently VACUUM doesn't suppo

Bracket, brace, parenthesis

2021-05-13 Thread Kyotaro Horiguchi
I found the following code in multirangetypes.c > if (*ptr == '{') > ptr++; > else > ereport(ERROR, > (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION), >errmsg("malformed multirange literal: \"%s\"",

Re: Added missing tab completion for alter subscription set option

2021-05-13 Thread Bharath Rupireddy
On Fri, May 14, 2021 at 12:00 PM vignesh C wrote: > > Hi, > > While I was reviewing one of the logical decoding features, I found > Streaming and binary options were missing in tab completion for the > alter subscription set option, the attached patch has the changes for > the same. > Thoughts? +

<    1   2