Re: Identify missing publications from publisher while create/alter subscription.

2022-03-29 Thread vignesh C
On Wed, Mar 30, 2022 at 11:22 AM Amit Kapila wrote: > > On Tue, Mar 29, 2022 at 8:11 PM vignesh C wrote: > > > > On Tue, Mar 29, 2022 at 11:02 AM Amit Kapila > > wrote: > > > > > > > Thanks for the suggestion, I have changed the patch as suggested. > > Attached v16 patch has the changes for the

Re: Add LZ4 compression in pg_dump

2022-03-29 Thread Michael Paquier
On Tue, Mar 29, 2022 at 09:14:03AM -0400, Robert Haas wrote: > I don't know what you mean by this. I think the specification stuff > could be reused in a lot of places. If you can ask for a base backup > with zstd:level=3,long=1,fancystuff=yes or whatever we end up with, > why not enable exactly th

Re: Possible fails in pg_stat_statements test

2022-03-29 Thread Anton A. Melnikov
Hello, thank you much for your attention and for your thought. On 20.03.2022 20:36, Andres Freund wrote: This patch introduces an additional counter of wal records not related to the query being executed. They're not unrelated though. Yes, i've missformulated here. Indeed there is a relatio

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-29 Thread Justin Pryzby
+ diff = (int32) (vacrel->NewRelfrozenXid - vacrel->relfrozenxid); + Assert(diff > 0); Did you see that this crashed on windows cfbot? https://api.cirrus-ci.com/v1/artifact/task/4592929254670336/log/tmp_check/postmaster.log TRAP: Failed

Re: Extend compatibility of PostgreSQL::Test::Cluster

2022-03-29 Thread Michael Paquier
On Tue, Mar 29, 2022 at 05:56:02PM -0400, Andrew Dunstan wrote: > I'm not sure why this item has been moved to the next CF without any > discussion I could see on the mailing list. It was always my intention > to commit it this time, and I propose to do so tomorrow with the comment > Michael has re

Re: Add LZ4 compression in pg_dump

2022-03-29 Thread Michael Paquier
On Tue, Mar 29, 2022 at 09:46:27AM +, gkokola...@pm.me wrote: > On Tuesday, March 29th, 2022 at 9:27 AM, Michael Paquier > wrote: >> On Sat, Mar 26, 2022 at 01:14:41AM -0500, Justin Pryzby wrote: >> Wow. This stuff is old enough to vote (c3e18804), dead since its >> introduction. There is ind

Re: Identify missing publications from publisher while create/alter subscription.

2022-03-29 Thread Amit Kapila
On Tue, Mar 29, 2022 at 8:11 PM vignesh C wrote: > > On Tue, Mar 29, 2022 at 11:02 AM Amit Kapila wrote: > > > > Thanks for the suggestion, I have changed the patch as suggested. > Attached v16 patch has the changes for the same. > Thanks, I have one more comment. postgres=# Alter subscription

Re: Granting SET and ALTER SYSTE privileges for GUCs

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 9:00 PM Mark Dilger wrote: > A grant or revoke on an unrecognized custom parameter will create a SUSET > placeholder, [...] > which cleans up the problem, with one exception: if the user executes a > "revoke set on parameter some.such from public" prior to loading the

Re: Is there any documentation on how to correctly create extensions in HA(primary-standby) setup?

2022-03-29 Thread Julien Rouhaud
Hi, On Sun, Mar 27, 2022 at 09:07:12AM +0530, Bharath Rupireddy wrote: > On Fri, Mar 25, 2022 at 10:20 AM Greg Stark wrote: > > > > This doesn't seem to be getting any further attention. It sounds like > > Julien didn't agree with the scope of the text. Bharath do you think > > Julien's comments

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Dilip Kumar
On Wed, Mar 30, 2022 at 6:47 AM Andres Freund wrote: > > > du -s /tmp/initdb/ > WAL_LOG: 35112 > FILE_COPY: 29288 > > So it seems we should specify a strategy in initdb? It kind of makes sense - > we're not going to read anything from those database. And because of the > ringbuffer of 256kB, we'll

Re: Assert in pageinspect with NULL pages

2022-03-29 Thread Julien Rouhaud
Hi, On Mon, Mar 28, 2022 at 08:29:29PM +0300, Maxim Orlov wrote: > I've suddenly found that the test in this patch is based on a fact that > heap pages don't have PageSpecial or it is of different size with btree > pages Special area: > > CREATE TABLE test1 (a int8, b int4range); > > SELECT bt_pag

Re: Jumble Query with COERCE_SQL_SYNTAX

2022-03-29 Thread Julien Rouhaud
Hi, On Tue, Mar 29, 2022 at 03:52:57PM +0300, Yura Sokolov wrote: > > v14 introduced the way to get original text for some kind of expressions > using new 'funcformat' - COERCE_SQL_SYNTAX: > - EXTRACT(part from timestamp) > - (text IS [form] NORMALIZED) > and others. > > Mentioned EXTRACT and NO

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-29 Thread Peter Geoghegan
On Tue, Mar 29, 2022 at 11:58 AM Peter Geoghegan wrote: > > I think I understand what the first paragraph of the header comment > > for heap_tuple_needs_freeze() is trying to say, but the second one is > > quite confusing. I think this is again because it veers into talking > > about what the call

Re: Column Filtering in Logical Replication

2022-03-29 Thread Amit Kapila
On Tue, Mar 29, 2022 at 6:09 PM Tomas Vondra wrote: > > On 3/29/22 13:47, Amit Kapila wrote: > > On Tue, Mar 29, 2022 at 4:33 PM Tomas Vondra > > wrote: > >> > >> On 3/29/22 12:00, Amit Kapila wrote: > > Thanks, I'll take a look later. > > >>> > >>> This is still failing [1][2]. >

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

2022-03-29 Thread Kyotaro Horiguchi
At Wed, 30 Mar 2022 10:06:17 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 29 Mar 2022 13:29:15 +0200, Erik Rijkers wrote in > > Op 29-03-2022 om 12:50 schreef Matthias van de Meent: > > > An shorter (?) reproducer might be the following, which forces any > > > value for 'a' to be toasted

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread David Rowley
On Wed, 30 Mar 2022 at 14:48, Andres Freund wrote: > > On 2022-03-30 14:30:32 +1300, David Rowley wrote: > > Maybe nodes below an Append/MergeAppend with run-time pruning could compile > > on-demand and other nodes up-front. Or maybe there's no problem with making > > everything on-demand. > > Ye

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Andres Freund
Hi, On 2022-03-30 14:30:32 +1300, David Rowley wrote: > On Wed, 30 Mar 2022 at 13:20, Andres Freund wrote: > > I wonder whether it'd make sense to combine that with awareness of a few > > plan > > types that can lead to large portions of child nodes never being executed. > > One > > the case wh

Re: logical replication empty transactions

2022-03-29 Thread Masahiko Sawada
On Tue, Mar 29, 2022 at 6:15 PM houzj.f...@fujitsu.com wrote: > > On Tuesday, March 29, 2022 5:12 PM Amit Kapila > wrote: > > > > On Tue, Mar 29, 2022 at 2:05 PM houzj.f...@fujitsu.com > > wrote: > > > > > > Attach the new version patch which addressed the above comments and > > > slightly adju

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-03-29 Thread David G. Johnston
On Tue, Mar 22, 2022 at 6:52 AM Michail Nikolaev wrote: > Hello, Andres. > > > Fails to apply at the moment: http://cfbot.cputube.org/patch_37_2947.log > > Thanks for notifying me. BTW, some kind of automatic email in case of > status change could be very helpful. > > > Marked as waiting for auth

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread David Rowley
On Wed, 30 Mar 2022 at 13:20, Andres Freund wrote: > I wonder whether it'd make sense to combine that with awareness of a few plan > types that can lead to large portions of child nodes never being executed. One > the case where the current behaviour is the worst is runtime partition pruning > in

Re: remove reset_shared()

2022-03-29 Thread Julien Rouhaud
Hi, On Tue, Mar 29, 2022 at 03:17:02PM -0700, Nathan Bossart wrote: > Hi hackers, > > Is there any reason to keep reset_shared() around anymore? It is now just > a wrapper function for CreateSharedMemoryAndSemaphores(), and AFAICT the > information in the comments is already covered by comments i

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 11:55:05 -0400, Robert Haas wrote: > I committed v6 instead. Just noticed that it makes initdb a bit slower / the cluster a bit bigger, because now there's WAL traffic from creating the databases. There's an optimization (albeit insufficient) to reduce WAL traffic in bootstrap

Re: TRAP: FailedAssertion("HaveRegisteredOrActiveSnapshot()", File: "toast_internals.c", Line: 670, PID: 19403)

2022-03-29 Thread Kyotaro Horiguchi
At Tue, 29 Mar 2022 13:29:15 +0200, Erik Rijkers wrote in > Op 29-03-2022 om 12:50 schreef Matthias van de Meent: > > An shorter (?) reproducer might be the following, which forces any > > value for 'a' to be toasted and thus triggering the check in > > init_toast_snapshot regardless of value len

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 5:56 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Tue, Mar 29, 2022 at 5:50 PM Andres Freund wrote: > >> Hi, >> >> On 2022-03-29 17:06:24 -0700, David G. Johnston wrote: >> > On Tue, Mar 29, 2022 at 4:43 PM Andres Freund >> wrote: >> > > But more importa

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 5:50 PM Andres Freund wrote: > Hi, > > On 2022-03-29 17:06:24 -0700, David G. Johnston wrote: > > On Tue, Mar 29, 2022 at 4:43 PM Andres Freund > wrote: > > > But more importantly, a > > > per-relation/function reset field wouldn't address Tomas's concern: He > > > wants

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 5:20 PM Peter Geoghegan wrote: > On Tue, Mar 29, 2022 at 4:55 AM Michail Nikolaev > wrote: > > I think that you could do a better job of explaining and promoting the > problem that you're trying to solve here. Emphasis on the problem, not > so much the solution. As a sp

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 17:06:24 -0700, David G. Johnston wrote: > On Tue, Mar 29, 2022 at 4:43 PM Andres Freund wrote: > > But more importantly, a > > per-relation/function reset field wouldn't address Tomas's concern: He > > wants a > > single thing to check to see if any stats have been reset - and

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Thomas Munro
On Wed, Mar 30, 2022 at 11:25 AM Andres Freund wrote: > Didn't immediate find a reference to a cat equivalent. Maybe just gzip the > file? That can read from stdin across platforms afaict. . o O ( gzip | gzip -d )

Re: Temporary tables versus wraparound... again

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 19:51:26 -0400, Greg Stark wrote: > On Mon, 28 Mar 2022 at 16:30, Andres Freund wrote: > > > > > Make ON COMMIT DELETE ROWS reset relfrozenxmin and other table stats > > > like normal truncate. Otherwise even typical short-lived transactions > > > using temporary tab

Re: Temporary tables versus wraparound... again

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 4:52 PM Greg Stark wrote: > On Mon, 28 Mar 2022 at 16:30, Andres Freund wrote: > > > > > Make ON COMMIT DELETE ROWS reset relfrozenxmin and other table > stats > > > like normal truncate. Otherwise even typical short-lived > transactions > > > using temporary

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-03-29 Thread Peter Geoghegan
On Tue, Mar 29, 2022 at 4:55 AM Michail Nikolaev wrote: > > Overall, I think that this patch has serious design flaws, and that > > this issue is really just a symptom of a bigger problem. > > Could you please advise me on something? The ways I see: > * give up :) I would never tell anybody to gi

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Andres Freund
Hi, On 2022-03-30 12:41:41 +1300, David Rowley wrote: > On Wed, 30 Mar 2022 at 12:16, Andres Freund wrote: > > > I did propose a patch to address this in [1]. It does need more work > > > and I do plan to come back to it for v16. > > > > FWIW, that doesn't seem quite right - won't it stop JITing

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 4:43 PM Andres Freund wrote: > But more importantly, a > per-relation/function reset field wouldn't address Tomas's concern: He > wants a > single thing to check to see if any stats have been reset - and that's imo > a > quite reasonable desire. > Per the original email:

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 23:38:29 +, Jacob Champion wrote: > On Sat, 2022-03-26 at 11:36 -0700, Andres Freund wrote: > > > I also note that exposing it as a GUC means we have zero control over > > > who/what can read it. Maybe that's not a problem, but it needs to be > > > thought about before we go

Re: Temporary tables versus wraparound... again

2022-03-29 Thread Greg Stark
On Mon, 28 Mar 2022 at 16:30, Andres Freund wrote: > > > Make ON COMMIT DELETE ROWS reset relfrozenxmin and other table stats > > like normal truncate. Otherwise even typical short-lived transactions > > using temporary tables can easily cause them to reach relfrozenxid. > > Might be w

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Tom Lane
David Rowley writes: > On Wed, 30 Mar 2022 at 12:16, Andres Freund wrote: >> FWIW, that doesn't seem quite right - won't it stop JITing e.g. on the inner >> side of a nested loop, just because it's cheap, even though that's where the >> bulk of the benefits comes from? > Yeah, I think the total

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 14:14:05 -0700, David G. Johnston wrote: > On Tue, Mar 29, 2022 at 1:37 PM Andres Freund wrote: > > > > Secondly, to do anything really meaningful you need to calculate deltas, > > > and be able to detect if some of the stats were reset for the particular > > > interval. And th

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread David Rowley
On Wed, 30 Mar 2022 at 12:16, Andres Freund wrote: > > I did propose a patch to address this in [1]. It does need more work > > and I do plan to come back to it for v16. > > FWIW, that doesn't seem quite right - won't it stop JITing e.g. on the inner > side of a nested loop, just because it's chea

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-03-29 Thread Jacob Champion
On Sat, 2022-03-26 at 11:36 -0700, Andres Freund wrote: > > I also note that exposing it as a GUC means we have zero control over > > who/what can read it. Maybe that's not a problem, but it needs to be > > thought about before we go down that path. > > Yes, I think that's a fair concern. I like

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Andres Freund
Hi, On 2022-03-30 09:05:44 +1300, David Rowley wrote: > I really believe that the main problem here is that JIT only enables > when the *total* plan cost reaches a certain threshold. Yes, that is/was a clear design mistake. It wasn't quite as bad back when it was written - partitioning blows the

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-03-29 Thread Nathan Bossart
Thanks for taking a look! On Thu, Mar 24, 2022 at 01:17:01PM +1300, Thomas Munro wrote: > /* we're only handling directories here, skip if it's not ours */ > -if (lstat(path, &statbuf) == 0 && !S_ISDIR(statbuf.st_mode)) > +if (lstat(path, &statbuf) != 0) > +ereport(ERROR, > +

Re: Frontend error logging style

2022-03-29 Thread Tom Lane
Daniel Gustafsson writes: > On 29 Mar 2022, at 16:38, Peter Eisentraut > wrote: >> Most of these should probably be addressed separately from Tom's patch. > Yeah, I think so too. Agreed. I tried to confine my patch to mechanical changes, except for changing places where the detail/hint featur

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Andrew Dunstan
On 3/29/22 17:19, Robert Haas wrote: > On Tue, Mar 29, 2022 at 4:36 PM Thomas Munro wrote: >> It failed: >> >> https://cirrus-ci.com/task/5567070686412800 >> https://api.cirrus-ci.com/v1/artifact/task/5567070686412800/log/contrib/basebackup_to_shell/tmp_check/log/001_basic_primary.log >> https:/

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 17:19:44 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 4:36 PM Thomas Munro wrote: > > It failed: > > > > https://cirrus-ci.com/task/5567070686412800 > > https://api.cirrus-ci.com/v1/artifact/task/5567070686412800/log/contrib/basebackup_to_shell/tmp_check/log/001_basic_pri

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Peter Geoghegan
On Tue, Mar 29, 2022 at 3:04 PM Tom Lane wrote: > I think David's questions are sufficiently cogent and difficult > that we should not add jit_warn_above_fraction at this time. +1 -- Peter Geoghegan

Re: Frontend error logging style

2022-03-29 Thread Daniel Gustafsson
> On 29 Mar 2022, at 16:38, Peter Eisentraut > wrote: > > On 29.03.22 12:13, Daniel Gustafsson wrote: > > Most of these should probably be addressed separately from Tom's patch. Yeah, I think so too. I'll await input from Tom on how he wants to do, but I'm happy to take these to a new thread.

remove reset_shared()

2022-03-29 Thread Nathan Bossart
Hi hackers, Is there any reason to keep reset_shared() around anymore? It is now just a wrapper function for CreateSharedMemoryAndSemaphores(), and AFAICT the information in the comments is already covered by comments in the shared memory code. I think it's arguable that the name of the function

Re: Window Function "Run Conditions"

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 15:11:52 +1300, David Rowley wrote: > One thing which I'm not sure about with the patch is how I'm handling > the evaluation of the runcondition in nodeWindowAgg.c. Instead of > having ExecQual() evaluate an OpExpr such as "row_number() over (...) > <= 10", I'm replacing the Win

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Mar 29, 2022 at 1:06 PM David Rowley wrote: >> That means that even if the total execution time of a plan was a true >> reflection of the total estimated plan cost, then the fraction of time >> spent (as is measured by jit_warn_above_fraction) doing JIT would >>

Re: Extend compatibility of PostgreSQL::Test::Cluster

2022-03-29 Thread Andrew Dunstan
On 1/21/22 09:59, Andrew Dunstan wrote: > On 1/21/22 02:47, Michael Paquier wrote: >> On Tue, Jan 18, 2022 at 06:35:39PM -0500, Andrew Dunstan wrote: >>> Here's a version that does that and removes some recent bitrot. >> I have been looking at the full set of features of Cluster.pm and the >> req

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 4:36 PM Thomas Munro wrote: > It failed: > > https://cirrus-ci.com/task/5567070686412800 > https://api.cirrus-ci.com/v1/artifact/task/5567070686412800/log/contrib/basebackup_to_shell/tmp_check/log/001_basic_primary.log > https://api.cirrus-ci.com/v1/artifact/task/5567070686

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread David G. Johnston
On Tue, Mar 29, 2022 at 1:37 PM Andres Freund wrote: > > Secondly, to do anything really meaningful you need to calculate deltas, > > and be able to detect if some of the stats were reset for the particular > > interval. And the stat_reset timestamp was designed to be a simple way > > to detect t

Re: Commitfest Update

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 4:47 PM Justin Pryzby wrote: > If it were me, I'd move these out of the way somehow; WOA/RWF or move to June: > > https://commitfest.postgresql.org/37/3291/ Add PGDLLIMPORT to all direct or > indirect GUC variables I plan to do this yet, but it seemed best to leave it unt

Re: Higher level questions around shared memory stats

2022-03-29 Thread Andres Freund
Hi, On 2022-03-29 16:24:02 -0400, Robert Haas wrote: > On Tue, Mar 29, 2022 at 3:17 PM Andres Freund wrote: > > 1) How to react to corrupted statsfiles? > > > > IMO it'd make more sense to just throw away all stats > > in that case. > > That seems reasonable to me. I think there's some downside,

Re: Commitfest Update

2022-03-29 Thread Justin Pryzby
On Thu, Mar 17, 2022 at 02:07:16PM -0400, Greg Stark wrote: > So far this commitfest these 10 patches have been marked committed. > That leaves us with 175 "Needs Review" and 28 "Ready for Comitter" so > quite a ways to go ... If it were me, I'd move these out of the way somehow; WOA/RWF or move t

Re: pg_stat_reset_single_*_counters vs pg_stat_database.stats_reset

2022-03-29 Thread Andres Freund
Hi, On 2022-03-24 13:12:24 +0100, Tomas Vondra wrote: > I agree it should have been documented, but I still stand behind the > current behavior. I'm not willing to die on this hill, but I think the > reasoning was/is sound. > > Firstly, calculating transactions/second, reads/second just from by >

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Peter Geoghegan
On Tue, Mar 29, 2022 at 1:06 PM David Rowley wrote: > That means that even if the total execution time of a plan was a true > reflection of the total estimated plan cost, then the fraction of time > spent (as is measured by jit_warn_above_fraction) doing JIT would > entirely depend on the number o

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Thomas Munro
On Wed, Mar 30, 2022 at 8:30 AM Thomas Munro wrote: > On Wed, Mar 30, 2022 at 3:08 AM Robert Haas wrote: > > Here's a new version, hopefully rectifying that deficiency. I also add > > a second patch here, documenting basebackup_to_shell.required_role, > > because Joe Conway observed elsewhere tha

Re: CREATE TABLE ( .. STORAGE ..)

2022-03-29 Thread Matthias van de Meent
On Wed, 2 Feb 2022 at 11:13, Teodor Sigaev wrote: > > Hi! > > > Are they both set to name or ColId? Although they are the same. > > > > Thank you, fixed, that was just an oversight. > > > 2 For ColumnDef new member storage_name, did you miss the function > > _copyColumnDef() _equalColumnDef()? >

Re: Higher level questions around shared memory stats

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 3:17 PM Andres Freund wrote: > 1) How to react to corrupted statsfiles? > > IMO it'd make more sense to just throw away all stats > in that case. That seems reasonable to me. I think there's some downside, in that stats are important, and having some of them might be bette

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread David Rowley
On Wed, 30 Mar 2022 at 02:38, Robert Haas wrote: > I think WARNING is fine. After all, the parameter is called > "jit_warn_above_fraction". I had a think about this patch. I guess it's a little similar to checkpoint_warning. The good thing about the checkpoint_warning is that in the LOG message

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Thomas Munro
On Wed, Mar 30, 2022 at 3:08 AM Robert Haas wrote: > On Fri, Mar 25, 2022 at 5:52 PM Thomas Munro wrote: > > On Sat, Mar 26, 2022 at 9:55 AM Thomas Munro wrote: > > This line doesn't look too healthy: > > > > pg_basebackup: error: backup failed: ERROR: shell command "type con > > > "C:cirruscon

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 2:37 PM Tom Lane wrote: > Robert Haas writes: > > On Tue, Mar 29, 2022 at 2:17 PM Tom Lane wrote: > >> it's some random algorithm that they probably feel at liberty > >> to change. > > > I guess that characterization surprises me. The man page for > > getopt_long() says t

Higher level questions around shared memory stats

2022-03-29 Thread Andres Freund
Hi, Separate from the minutia in [1] I'd like to discuss a few questions of more general interest. I'll post another question or two later. 1) How to react to corrupted statsfiles? In HEAD we stop reading stats at the point we detect the stats file to be corrupted. The contents of the currently

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-29 Thread Peter Geoghegan
On Tue, Mar 29, 2022 at 10:03 AM Robert Haas wrote: > I can understand this version of the commit message. Woohoo! I like > understanding things. That's good news. > I think the header comments for FreezeMultiXactId() focus way too much > on what the caller is supposed to do and not nearly enoug

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 29, 2022 at 2:17 PM Tom Lane wrote: >> it's some random algorithm that they probably feel at liberty >> to change. > I guess that characterization surprises me. The man page for > getopt_long() says this, and has for a long time at least on systems > I've used:

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 2:17 PM Tom Lane wrote: > Robert Haas writes: > > On Tue, Mar 29, 2022 at 1:53 PM Tom Lane wrote: > >> That test script is expecting glibc-like laxness of switch > >> parsing. Put the switches before the non-switch arguments. > > > I just did that. :-) > > Yup, you pushe

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 29, 2022 at 1:53 PM Tom Lane wrote: >> That test script is expecting glibc-like laxness of switch >> parsing. Put the switches before the non-switch arguments. > I just did that. :-) Yup, you pushed while I was typing. FWIW, I don't think it's "Windows" enfor

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 1:53 PM Tom Lane wrote: > Andres Freund writes: > > Looks like there's some problem with commandline parsing? > > That test script is expecting glibc-like laxness of switch > parsing. Put the switches before the non-switch arguments. I just did that. :-) -- Robert Haas

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Tom Lane
Andres Freund writes: > Looks like there's some problem with commandline parsing? That test script is expecting glibc-like laxness of switch parsing. Put the switches before the non-switch arguments. regards, tom lane

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 1:35 PM Andres Freund wrote: > # Running: createdb -T foobar2 foobar6 -S wal_log > createdb: error: too many command-line arguments (first is "wal_log") > Try "createdb --help" for more information. > not ok 31 - createdb -T foobar2 foobar6 -S wal_log exit code 0 > > Looks

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Andres Freund
On 2022-03-29 11:55:05 -0400, Robert Haas wrote: > That commit having been reverted, I committed v6 instead. Let's see > what breaks... It fails in CI (for the mirror of the postgres repo on github): https://cirrus-ci.com/task/6279465603956736?logs=test_bin#L121 tap test log: https://api.cirrus-c

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-03-29 Thread Peter Eisentraut
On 29.03.22 15:09, Aleksander Alekseev wrote: Hi hackers, I took another look at the patchset. Personally I don't think it will get much better than it is now. I'm inclined to change the status of the CF entry to "Ready for Committer" unless anyone disagrees. About v25-0002-Use-64-bit-format

Re: GSoC: Improve PostgreSQL Regression Test Coverage

2022-03-29 Thread Stephen Frost
Greetings, * Christos Maris (christos.c.ma...@gmail.com) wrote: > This is Christos Maris , and > I'd like to declare my interest in the GSoC project mentioned above. > I'm an experienced Fullstack Developer, but still an open-source beginner. > That, com

Re: Corruption during WAL replay

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 12:34 PM Stephen Frost wrote: > Anyhow, this whole thread has struck me as a good reason to polish those > patches off and add on top of them an extended checksum ability, first, > independent of TDE, and remove the dependency of those patches from the > TDE effort and inst

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-29 Thread Robert Haas
On Sun, Mar 27, 2022 at 11:24 PM Peter Geoghegan wrote: > Attached is v12. My current goal is to commit all 3 patches before > feature freeze. Note that this does not include the more complicated > patch including with previous revisions of the patch series (the > page-level freezing work that app

Re: [PATCH] Full support for index LP_DEAD hint bits on standby

2022-03-29 Thread Michail Nikolaev
UPD: > I was thinking it is safe to have additional hint bits > on primary, but it seems like no. Oh, sorry for the mistake, it is about standby of course. > BTW I am wondering if it is possible > to achieve the same situation by pg_rewind and standby promotion… Looks like it is impossible, bec

Re: Corruption during WAL replay

2022-03-29 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Mar 25, 2022 at 10:34 AM Tom Lane wrote: > > I dunno. Compatibility and speed concerns aside, that seems like an awful > > lot of bits to be expending on every page compared to the value. > > I dunno either, but over on the TDE t

Re: make MaxBackends available in _PG_init

2022-03-29 Thread Robert Haas
On Sat, Mar 26, 2022 at 1:23 PM Andres Freund wrote: > > And indeed, any third party code that previously needed to access what > > MaxBackends is supposed to store should already be using that formula, and > > the new GetMaxBackends() doesn't do anything about it. > > It couldn't rely on MaxBacke

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-03-29 Thread Robert Haas
On Mon, Mar 28, 2022 at 3:08 PM Robert Haas wrote: > smgrcreate() as we would for most WAL records or whether it should be > adopting the new system introduced by > 49d9cfc68bf4e0d32a948fe72d5a0ef7f464944e. I wrote about this concern > over here: > > http://postgr.es/m/CA+TgmoYcUPL+WOJL2ZzhH=zmrhj

Re: Correct docs re: rewriting indexes when table rewrite is skipped

2022-03-29 Thread Matthias van de Meent
On Tue, 29 Mar 2022 at 16:04, James Coleman wrote: > > Back in 367bc42 (for 9.2!) we "avoid[ed] index rebuild[ing] for > no-rewrite ALTER TABLE > .. ALTER TYPE." However the docs still claim that "a table rewrite is > not needed; but any indexes on the affected columns must still be > rebuilt." A

Re: [PATCH] Enable SSL library detection via PQsslAttribute

2022-03-29 Thread Jacob Champion
On Tue, 2022-03-29 at 14:08 +0200, Daniel Gustafsson wrote: > Pushed with a few small tweaks to make it match project style, thanks! Thank you! --Jacob

Re: pg14 psql broke \d datname.nspname.relname

2022-03-29 Thread Robert Haas
On Fri, Mar 25, 2022 at 3:42 PM Mark Dilger wrote: > I think your change is fine, so I've rolled it into this next patch. OK, cool. Here are some more comments. In describe.c, why are the various describeWhatever functions returning true when validateSQLNamePattern returns false? It seems to me

Re: Add header support to text format and matching feature

2022-03-29 Thread Peter Eisentraut
On 25.03.22 05:21, Greg Stark wrote: Great to see the first of the two patches committed. It looks like the second patch got some feedback from Peter in February and has been marked "Waiting on author" ever since. Remi, will you have a chance to look at this this month? Peter, are these commen

Re: Identify missing publications from publisher while create/alter subscription.

2022-03-29 Thread vignesh C
On Tue, Mar 29, 2022 at 4:12 PM Amit Kapila wrote: > > On Tue, Mar 29, 2022 at 11:01 AM Amit Kapila wrote: > > > > On Sat, Mar 26, 2022 at 7:53 PM vignesh C wrote: > > > > > > The patch was not applying on HEAD, attached patch which is rebased on > > > top of HEAD. > > > > > > > IIUC, this patch

Re: Identify missing publications from publisher while create/alter subscription.

2022-03-29 Thread vignesh C
On Tue, Mar 29, 2022 at 11:02 AM Amit Kapila wrote: > > On Sat, Mar 26, 2022 at 7:53 PM vignesh C wrote: > > > > The patch was not applying on HEAD, attached patch which is rebased on > > top of HEAD. > > > > IIUC, this patch provides an option that allows us to give an error if > while creating/

Re: Frontend error logging style

2022-03-29 Thread Peter Eisentraut
On 29.03.22 12:13, Daniel Gustafsson wrote: Most of these should probably be addressed separately from Tom's patch. @@ -508,24 +502,15 @@ writefile(char *path, char **lines) if (fclose(out_file)) - { - pg_log_error("could not write file \"%s\": %m", path); -

Re: Frontend error logging style

2022-03-29 Thread Peter Eisentraut
On 27.03.22 22:19, Tom Lane wrote: Here's a rebase up to today's HEAD. I've fixed the merge problems, but there may be some stray new error calls that could be converted to use pg_fatal() and haven't been. I don't want to do a full fresh scan of the code until we're about ready to commit this.

Restructure ALTER TABLE notes to clarify table rewrites and verification scans

2022-03-29 Thread James Coleman
Over in the "Document atthasmissing default optimization avoids verification table scan" thread David Johnston (who I've cc'd) suggested that my goals might be better implemented with a simple restructuring of the Notes section of the ALTER TABLE docs. I think this is also along the lines of Tom La

Re: pgsql: Add 'basebackup_to_shell' contrib module.

2022-03-29 Thread Robert Haas
On Fri, Mar 25, 2022 at 5:52 PM Thomas Munro wrote: > On Sat, Mar 26, 2022 at 9:55 AM Thomas Munro wrote: > > https://api.cirrus-ci.com/v1/artifact/task/5637156969381888/log/contrib/basebackup_to_shell/tmp_check/log/regress_log_001_basic > > This line doesn't look too healthy: > > pg_basebackup:

Correct docs re: rewriting indexes when table rewrite is skipped

2022-03-29 Thread James Coleman
Back in 367bc42 (for 9.2!) we "avoid[ed] index rebuild[ing] for no-rewrite ALTER TABLE .. ALTER TYPE." However the docs still claim that "a table rewrite is not needed; but any indexes on the affected columns must still be rebuilt." I've attached a simple patch to update the docs to match the curr

Re: Document atthasmissing default optimization avoids verification table scan

2022-03-29 Thread James Coleman
On Sun, Mar 27, 2022 at 11:12 PM David G. Johnston wrote: > > On Sun, Mar 27, 2022 at 11:17 AM James Coleman wrote: >> >> Hmm, I didn't realize that was project policy, > > > Guideline/Rule of Thumb is probably a better concept. Ah, OK, thanks. >> >> but I'm a bit >> surprised given that the s

Re: Add parameter jit_warn_above_fraction

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 6:09 AM Julien Rouhaud wrote: > The last remaining thing is whether logging at WARNING level is the correct > choice. I'm personally fine with it, because the people who are going to use > it will probably use the same approach as for log_min_duration_statements: > enable

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 9:28 AM Alvaro Herrera wrote: > OK, this is a bug that's been open for years. A fix can be committed > after the feature freeze anyway. +1 -- Robert Haas EDB: http://www.enterprisedb.com

Re: Accommodate startup process in a separate ProcState array slot instead of in MaxBackends slots.

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 3:21 AM Bharath Rupireddy wrote: > Thanks. LGTM. Committed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2022-03-29 Thread Alvaro Herrera
On 2022-Mar-29, Robert Haas wrote: > I'm fine with that approach, but I'd like to ask that we proceed > expeditiously, because I have another patch that I want to commit that > touches this area. I can commit to helping with whatever we decide to > do here, but I don't want to keep that patch on i

Re: Add LZ4 compression in pg_dump

2022-03-29 Thread Robert Haas
On Tue, Mar 29, 2022 at 1:03 AM Michael Paquier wrote: > > Then, if people are willing to adopt the syntax that the > > backup_compression.c/h stuff supports as a project standard (+1 from > > me) we can go the other way and rename that stuff to be more generic, > > taking backup out of the name.

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2022-03-29 Thread Aleksander Alekseev
Hi hackers, > I took another look at the patchset. Personally I don't think it will get much > better than it is now. I'm inclined to change the status of the CF entry to > "Ready for Committer" unless anyone disagrees. > > About v25-0002-Use-64-bit-format-to-output-XIDs.patch: > > I don't see th

Jumble Query with COERCE_SQL_SYNTAX

2022-03-29 Thread Yura Sokolov
Good day. v14 introduced the way to get original text for some kind of expressions using new 'funcformat' - COERCE_SQL_SYNTAX: - EXTRACT(part from timestamp) - (text IS [form] NORMALIZED) and others. Mentioned EXTRACT and NORMALIZED statements has parts, that are not usual arguments but some kind

Re: refactoring basebackup.c (zstd workers)

2022-03-29 Thread Robert Haas
On Mon, Mar 28, 2022 at 8:11 PM Tom Lane wrote: > I suspect Robert wrote it that way intentionally --- but if so, > I agree it could do with more than zero commentary. Well, the point is, we stop advancing kwend when we get to the end of the keyword, and *vend when we get to the end of the value.

  1   2   >