Re: [UNVERIFIED SENDER] Re: pg_upgrade can result in early wraparound on databases with high transaction load

2022-07-05 Thread Tom Lane
Andrew Dunstan writes: > On 2022-07-05 Tu 15:17, Tom Lane wrote: >> Hmm ... could you first look into why 09878cdd4 broke it? I'd supposed >> that that was just detecting situations we must already have dealt with >> in order for the pg_upgrade test to work, but crake's not happy. > It's complai

Re: Patch proposal: New hooks in the connection path

2022-07-05 Thread Joe Conway
On 7/5/22 03:37, Bharath Rupireddy wrote: On Mon, Jul 4, 2022 at 6:23 PM Drouvot, Bertrand wrote: On 7/2/22 1:00 AM, Nathan Bossart wrote: > Could we model this after fmgr_hook? The first argument in that hook > indicates where it is being called from. This doesn't alleviate the need > for se

Re: pg_checkpointer is not a verb or verb phrase

2022-07-05 Thread Tom Lane
Robert Haas writes: > On Tue, Jul 5, 2022 at 3:42 PM Nathan Bossart > wrote: >> On Tue, Jul 05, 2022 at 01:38:43PM -0400, Robert Haas wrote: >>> I added the catversion bump, but left out the .po >>> file changes, figuring it was better to let those files get updated >>> via the normal process.

Re: should check interrupts in BuildRelationExtStatistics ?

2022-07-05 Thread Tom Lane
Justin Pryzby writes: > On Fri, Jul 01, 2022 at 07:19:11PM -0400, Tom Lane wrote: >> That, um, piqued my interest. After a bit of digging, >> I modestly propose the attached. I'm not sure if it's >> okay to back-patch this, because maybe someone out there >> is relying on qsort() to be incapable

Re: should check interrupts in BuildRelationExtStatistics ?

2022-07-05 Thread Andres Freund
Hi, On 2022-07-01 19:19:11 -0400, Tom Lane wrote: > That, um, piqued my interest. After a bit of digging, > I modestly propose the attached. I'm not sure if it's > okay to back-patch this, because maybe someone out there > is relying on qsort() to be incapable of throwing an error > --- but it'd

Re: should check interrupts in BuildRelationExtStatistics ?

2022-07-05 Thread Tom Lane
I wrote: >>> I modestly propose the attached. I'm not sure if it's >>> okay to back-patch this, because maybe someone out there >>> is relying on qsort() to be incapable of throwing an error I thought I'd better try to check that, and I soon found several places that *are* relying on qsort() to n

Re: should check interrupts in BuildRelationExtStatistics ?

2022-07-05 Thread Tom Lane
Andres Freund writes: > I'm worried about this. Interrupting random qsorts all over seems like it > could end up corrupting state. We do things like qsort in building snapshots > etc. Are we confident that we handle interrupts reliably in all those places? Nope ... see my followup just now.

Re: First draft of the PG 15 release notes

2022-07-05 Thread Bruce Momjian
On Tue, Jul 5, 2022 at 02:57:52PM -0700, Noah Misch wrote: > Since having too-permissive ACLs is usually symptom-free, I share your > forecast about the more-common question. Expect questions on mailing lists, > stackoverflow, etc. The right way to answer those questions is roughly this: > >

Re: should check interrupts in BuildRelationExtStatistics ?

2022-07-05 Thread Michael Paquier
On Tue, Jul 05, 2022 at 07:37:03PM -0400, Tom Lane wrote: > Nonetheless, it'd be a good idea to use an interruptible sort in > as many places as we can. If we don't mind YA copy of the qsort > code, I'd be inclined to propose inventing qsort_interruptible > which is like qsort_arg but also does CH

Re: avoid multiple hard links to same WAL file after a crash

2022-07-05 Thread Michael Paquier
On Tue, Jul 05, 2022 at 09:58:38AM -0700, Nathan Bossart wrote: > Thanks! I wonder if we should add a comment in writeTimeLineHistoryFile() > about possible concurrent use by a WAL receiver and the startup process and > why that is okay. Agreed. Adding an extra note at the top of the routine wou

Re: Doc about how to set max_wal_senders when setting minimal wal_level

2022-07-05 Thread Tom Lane
Japin Li writes: > [ v4-wal-level-documentation.patch ] Hm, I don't care for the wording here: +A precondition for using minimal WAL is to disable WAL archiving and +streaming replication by setting archive_mode to +off, and to +0. "Precondition" is an overly fa

Re: doc: Clarify Routines and Extension Membership

2022-07-05 Thread Tom Lane
"David G. Johnston" writes: + A function that's marked as dependent on an extension is dropped when the + extension is dropped, even if cascade is not specified. + dependency checking in restrict mode . + A function can depend upon multiple extensions, and will be dropped when

Re: EINTR in ftruncate()

2022-07-05 Thread Andres Freund
Hi, On 2022-07-04 13:07:50 +0200, Alvaro Herrera wrote: > On 2022-Jul-01, Andres Freund wrote: > > > On 2022-07-01 19:55:16 +0200, Alvaro Herrera wrote: > > > On 2022-Jul-01, Andres Freund wrote: > > > > > What is the reason for the || ProcDiePending || QueryCancelPending bit? > > > > What > >

Re: doc: Fix description of how the default user name is chosen

2022-07-05 Thread Tom Lane
"David G. Johnston" writes: > In passing, the authentication error examples use the phrase > "database user name" in a couple of locations. The word > database in both cases is both unusual and unnecessary for > understanding. The reference to user name means the one in/for the >

Re: PG 15 (and to a smaller degree 14) regression due to ExprEvalStep size

2022-07-05 Thread Andres Freund
Hi, On 2022-06-29 11:40:45 +1200, David Rowley wrote: > On Sat, 18 Jun 2022 at 08:06, Andres Freund wrote: > > I also attached my heavily-WIP patches for the ExprEvalStep issues, I > > accidentally had only included a small part of the contents of the json fix. > > I've now looked at the 0003 pa

Re: doc: Fix description of how the default user name is chosen

2022-07-05 Thread David G. Johnston
On Tue, Jul 5, 2022 at 5:20 PM Tom Lane wrote: > "David G. Johnston" writes: > > In passing, the authentication error examples use the phrase > > "database user name" in a couple of locations. The word > > database in both cases is both unusual and unnecessary for > > understand

Re: should check interrupts in BuildRelationExtStatistics ?

2022-07-05 Thread Thomas Munro
On Wed, Jul 6, 2022 at 11:37 AM Tom Lane wrote: > qsort_interruptible +1 FWIW compute_scalar_stats() was already noted as hot and perhaps a candidate for specialisation (with more study needed), but qsort_interruptible() seems like a sane use of ~4KB of text to me.

Re: automatically generating node support functions

2022-07-05 Thread Tom Lane
... BTW, I thought of a consideration that we probably need some answer for. As far as I can see, the patch assigns NodeTag values sequentially in the order it sees the struct declarations in the input files; an order that doesn't have a lot to do with our past practice. The problem with that is

Re: Logging query parmeters in auto_explain

2022-07-05 Thread Michael Paquier
On Fri, Jul 01, 2022 at 09:58:52AM +0900, Michael Paquier wrote: > I have kept things as I originally intended, and applied 0001 for the > refactoring pieces. And done as well with 0002. So we are good for this thread. -- Michael signature.asc Description: PGP signature

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-07-05 Thread Kyotaro Horiguchi
At Mon, 4 Jul 2022 10:49:33 +0200, Alvaro Herrera wrote in > > Mmm. My patch moves the point of failure of the scenario a bit but > > still a little short. However, as my understanding, it seems like the > > task of the PQpipelineSync()-PQgetResult() pair to consume the > > CloseComplete. If Ii

Re: Issue with pg_stat_subscription_stats

2022-07-05 Thread Masahiko Sawada
On Wed, Jul 6, 2022 at 6:52 AM Andres Freund wrote: > > Hi, > > On 2022-07-04 11:01:01 +0900, Masahiko Sawada wrote: > > I've attached the patch, fix_drop_subscriptions_stats.patch, to fix it. > > LGTM. Unless somebody sees a reason not to, I'm planning to commit that to 15 > and HEAD. > > > > I'v

Re: tuplesort Generation memory contexts don't play nicely with index builds

2022-07-05 Thread David Rowley
On Wed, 29 Jun 2022 at 12:59, David Rowley wrote: > I noticed while doing some memory context related work that since we > now use generation.c memory contexts for tuplesorts (40af10b57) that > tuplesort_putindextuplevalues() causes memory "leaks" in the > generation context due to index_form_tupl

Re: Issue with pg_stat_subscription_stats

2022-07-05 Thread Andres Freund
On 2022-07-06 10:25:02 +0900, Masahiko Sawada wrote: > > I think most of this could just be pgstat_reset_entry(). > > I think pgstat_reset_entry() doesn't work for this case as it skips > resetting the entry if it doesn't exist. True - but a pgstat_get_entry_ref(create = true); pgstat_reset_entry

Re: pg_upgrade (12->14) fails on aggregate

2022-07-05 Thread Michael Paquier
On Tue, Jul 05, 2022 at 01:07:48PM -0400, Tom Lane wrote: > It looks about ready to me. Pushed with some minor cosmetic > adjustments. crake and drongo look unhappy after that, as of the upgrade from 9.6: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2022-07-05%2020%3A48%3A21 h

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-05 Thread Masahiko Sawada
On Mon, Jul 4, 2022 at 9:42 PM Amit Kapila wrote: > > On Mon, May 30, 2022 at 11:13 AM Masahiko Sawada > wrote: > > > > I've attached three POC patches: > > > > I think it will be a good idea if you can add a short commit message > at least to say which patch is proposed for HEAD and which one i

Re: AIX support - alignment issues

2022-07-05 Thread Thomas Munro
On Tue, Jul 5, 2022 at 4:53 PM Tom Lane wrote: > Thomas Munro writes: > > On Mon, Jul 4, 2022 at 12:08 PM Tom Lane wrote: > >> I would not stand in the way of dropping HP-UX and IA64 support as of > >> v16. (I do still feel that HPPA is of interest, to keep us honest > >> about spinlock support

Re: pg_rewind: warn when checkpoint hasn't happened after promotion

2022-07-05 Thread Kyotaro Horiguchi
At Tue, 5 Jul 2022 14:46:13 -0400, James Coleman wrote in > On Tue, Jul 5, 2022 at 2:39 PM Robert Haas wrote: > > > > On Sat, Jun 4, 2022 at 8:59 AM James Coleman wrote: > > > A quick background refresher: after promoting a standby rewinding the > > > former primary requires that a checkpoint h

Re: Issue with pg_stat_subscription_stats

2022-07-05 Thread Masahiko Sawada
On Wed, Jul 6, 2022 at 10:48 AM Andres Freund wrote: > > On 2022-07-06 10:25:02 +0900, Masahiko Sawada wrote: > > > I think most of this could just be pgstat_reset_entry(). > > > > I think pgstat_reset_entry() doesn't work for this case as it skips > > resetting the entry if it doesn't exist. > >

Re: First draft of the PG 15 release notes

2022-07-05 Thread Noah Misch
On Tue, Jul 05, 2022 at 07:47:52PM -0400, Bruce Momjian wrote: > On Tue, Jul 5, 2022 at 02:57:52PM -0700, Noah Misch wrote: > > Since having too-permissive ACLs is usually symptom-free, I share your > > forecast about the more-common question. Expect questions on mailing lists, > > stackoverflow,

RE: Re-order "disable_on_error" in tab-complete COMPLETE_WITH

2022-07-05 Thread osumi.takami...@fujitsu.com
On Tuesday, July 5, 2022 1:05 PM Amit Kapila wrote: > On Tue, Jul 5, 2022 at 4:03 AM Peter Smith wrote: > > > > On Mon, Jul 4, 2022 at 10:29 PM Euler Taveira wrote: > > > > > > On Mon, Jul 4, 2022, at 5:37 AM, Amit Kapila wrote: > > > > > > Yeah, it seems we have overlooked this point. I think w

Re: Eliminating SPI from RI triggers - take 2

2022-07-05 Thread Amit Langote
On Wed, Jul 6, 2022 at 3:24 AM Jacob Champion wrote: > On Thu, Jun 30, 2022 at 11:23 PM Amit Langote wrote: > > > > I will continue investigating what to do about points (1) and (2) > > mentioned above and see if we can do away with using SQL in the > > remaining cases. > > Hi Amit, looks like is

Re: AIX support - alignment issues

2022-07-05 Thread Andres Freund
Hi, On 2022-07-06 14:21:50 +1200, Thomas Munro wrote: > --- a/src/backend/port/hpux/tas.c.template > +++ /dev/null > @@ -1,40 +0,0 @@ > -/* > - * tas() for HPPA. > - * > - * To generate tas.s using this template: > - * 1. cc +O2 -S -c tas.c > - * 2. edit tas.s: > - * - replace the LD

Re: pg_upgrade (12->14) fails on aggregate

2022-07-05 Thread Tom Lane
Michael Paquier writes: > crake and drongo look unhappy after that, as of the upgrade from 9.6: Yeah. I think that 08385ed26 fixes this, but we've had no new reports yet :-( regards, tom lane

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2022-07-05 Thread Amit Kapila
On Tue, Jul 5, 2022 at 7:20 PM Melih Mutlu wrote: > > I created a patch to reuse tablesync workers and their replication slots for > more tables that are not synced yet. So that overhead of creating and > dropping workers/replication slots can be reduced. > > Current version of logical replicati

Re: Re-order "disable_on_error" in tab-complete COMPLETE_WITH

2022-07-05 Thread Michael Paquier
On Wed, Jul 06, 2022 at 02:48:07AM +, osumi.takami...@fujitsu.com wrote: > On Tuesday, July 5, 2022 1:05 PM Amit Kapila wrote: >> It can help if there is any bug-fix in the same code path or if some other >> code >> adjustment in the same area is required in the back branch. >> I feel the cha

Re: AIX support - alignment issues

2022-07-05 Thread Tom Lane
Thomas Munro writes: > OK, here's a new attempt, this time leaving the hppa bits in. The > main tricksy bit is where s_lock.h is simplified a bit by moving the > fully inline GCC-only hppa support up a bit (it was handled a bit > weirdly with some #undef jiggery-pokery before to share stuff betwe

Re: Re-order "disable_on_error" in tab-complete COMPLETE_WITH

2022-07-05 Thread Peter Smith
FYI, I confirmed the same patch applies and works OK for tags/REL_15_BETA2. -- [postgres@CentOS7-x64 ~]$ psql --version psql (PostgreSQL) 15beta2 [postgres@CentOS7-x64 ~]$ psql psql (15beta2) Type "help" for help. postgres=# create subscription mysub connection 'blah' publication mypub with

Re: AIX support - alignment issues

2022-07-05 Thread Thomas Munro
On Wed, Jul 6, 2022 at 3:26 PM Andres Freund wrote: > On 2022-07-06 14:21:50 +1200, Thomas Munro wrote: > > - * Notice that this means that we actually clear the word to set > > - * the lock and set the word to clear the lock. This is the > > - * opposite behavior from the SPARC LDSTU

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-07-05 Thread John Naylor
On Sat, Jun 25, 2022 at 8:05 AM Andres Freund wrote: > > I tried your patch with: > > > > DROP TABLE IF EXISTS json_as_text; > > CREATE TABLE json_as_text AS SELECT (SELECT json_agg(row_to_json(pd)) as t > > FROM pg_description pd) FROM generate_series(1, 100); > > VACUUM FREEZE json_as_text; >

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-07-05 Thread Andres Freund
Hi, On 2022-07-06 12:10:20 +0700, John Naylor wrote: > diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c > index eeedc0645a..ad4858c623 100644 > --- a/src/common/jsonapi.c > +++ b/src/common/jsonapi.c > @@ -851,10 +851,26 @@ json_lex_string(JsonLexContext *lex) > } >

Re: AIX support - alignment issues

2022-07-05 Thread Andres Freund
Hi, 0001 looks good to me. There's a leftover itanium reference in a comment in src/include/port/atomics/generic-msvc.h There's also a bunch of #ifdef __ia64__ in src/backend/utils/misc/guc-file.c, contrib/seg/segscan.c and contrib/cube/cubescan.c Otherwise lgtm as well. Greetings, Andres Fre

Re: AIX support - alignment issues

2022-07-05 Thread Tom Lane
Andres Freund writes: > There's also a bunch of #ifdef __ia64__ in src/backend/utils/misc/guc-file.c, > contrib/seg/segscan.c and contrib/cube/cubescan.c And all our other flex output files --- AFAICS that's part of flex's recipe and not under our control. regards, tom la

Re: AIX support - alignment issues

2022-07-05 Thread Andres Freund
On 2022-07-06 01:33:58 -0400, Tom Lane wrote: > Andres Freund writes: > > There's also a bunch of #ifdef __ia64__ in > > src/backend/utils/misc/guc-file.c, > > contrib/seg/segscan.c and contrib/cube/cubescan.c > > And all our other flex output files --- AFAICS that's part of flex's > recipe and

Re: [PATCH] Fix pg_upgrade test from v10

2022-07-05 Thread Michael Paquier
On Tue, Jul 05, 2022 at 02:08:24PM -0500, Justin Pryzby wrote: > ..since it tries to apply all the *.patch files to the master branch, one > after > another. For branches other than master, I suggest to name the patches *.txt > or similar. Or, just focus for now on allowing upgrades *to* master.

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-05 Thread Amit Kapila
On Wed, Jul 6, 2022 at 7:38 AM Masahiko Sawada wrote: > > I'll post a new version patch in the next email with replying to other > comments. > Okay, thanks for working on this. Few comments/suggestions on poc_remember_last_running_xacts_v2 patch: 1. +ReorderBufferSetLastRunningXactsCatalogChang

Re: Unify DLSUFFIX on Darwin

2022-07-05 Thread Peter Eisentraut
On 24.06.22 16:13, Tom Lane wrote: [ thinks for a bit... ] Might be worth double-checking that pg_upgrade doesn't get confused in a cross-version upgrade. A quick grep doesn't find that it refers to DLSUFFIX anywhere, but it definitely does pay attention to extensions' shared library names. p

Re: Improve TAP tests of pg_upgrade for cross-version tests

2022-07-05 Thread Michael Paquier
On Thu, Jun 09, 2022 at 04:49:01PM +0900, Michael Paquier wrote: > Rebased to cope with the recent changes in this area. Please find attached an updated version of this patch, where I have extended the support of the upgrade script down to 9.5 as origin version, as ~9.4 now fail because of cluster

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-07-05 Thread Masahiko Sawada
On Tue, Jul 5, 2022 at 8:00 PM Amit Kapila wrote: > > On Mon, Jul 4, 2022 at 6:12 PM Amit Kapila wrote: > > > > On Mon, May 30, 2022 at 11:13 AM Masahiko Sawada > > wrote: > > > > > > I've attached three POC patches: > > > > > > > I think it will be a good idea if you can add a short commit mes

Re: refactor some protocol message sending in walsender and basebackup

2022-07-05 Thread Peter Eisentraut
On 01.07.22 23:36, Nathan Bossart wrote: On Thu, Jun 23, 2022 at 04:36:36PM +0200, Peter Eisentraut wrote: Some places in walsender.c and basebackup_copy.c open-code the sending of RowDescription and DataRow protocol messages. But there are already more compact and robust solutions available fo

<    1   2