Fix logging for invalid recovery timeline

2024-12-20 Thread David Steele
Hackers, I noticed while debugging a user issue that the checkpoint logged in "Latest checkpoint is at %X/%X on timeline %u, but in the history of the requested timeline, the server forked off from that timeline at %X/%X." is being pulled from the value stored in the control file rather than

Re: FileFallocate misbehaving on XFS

2024-12-20 Thread Andres Freund
Hi, On 2024-12-19 17:47:13 +1100, Michael Harris wrote: > I finally managed to get the patched version installed in a production > database where the error is occurring very regularly. Thanks! > Here is a sample of the output: > > 2024-12-19 01:08:50 CET [2533222]: LOG: mdzeroextend FileFall

Document How Commit Handles Aborted Transactions

2024-12-20 Thread David G. Johnston
Hi, The commit reference page lacks an "Outputs" section even though it is capable of outputting both "COMMIT" and "ROLLBACK". The attached adds this section, describes when each applies, and then incorporates the same into the main description for commit as well as the transaction section of the

Re: Sort functions with specialized comparators

2024-12-20 Thread Andrey M. Borodin
> On 16 Dec 2024, at 14:02, John Naylor wrote: > > Sorry, I forgot this part earlier. Yes, let's have the private function. PFA v6. I was poking around intarray and trying not to bash everything. It seems to me that overall code readability should be seriously reworked. Even if no one is go

Re: FileFallocate misbehaving on XFS

2024-12-20 Thread Jakub Wartak
On Thu, Dec 19, 2024 at 7:49 AM Michael Harris wrote: > Hello, > > I finally managed to get the patched version installed in a production > database where the error is occurring very regularly. > > Here is a sample of the output: > > 2024-12-19 01:08:50 CET [2533222]: LOG: mdzeroextend FileFall

Re: Proposal for Updating CRC32C with AVX-512 Algorithm.

2024-12-20 Thread Sterrett, Matthew
On 12/7/2024 12:42 AM, Devulapalli, Raghuveer wrote: [0] https://cirrus-ci.com/task/6023394207989760 [1] https://cirrus-ci.com/task/5460444254568448 [2] https://cirrus-ci.com/task/6586344161411072 I was able to fix [0] and [1], but I can't think of why [2] fails. When I tried to reproduce th

Re: Can rs_cindex be < 0 for bitmap heap scans?

2024-12-20 Thread Ranier Vilela
Em qui., 19 de dez. de 2024 às 19:50, Melanie Plageman < melanieplage...@gmail.com> escreveu: > On Wed, Dec 18, 2024 at 9:50 PM Richard Guo > wrote: > > > > On Thu, Dec 19, 2024 at 8:18 AM Melanie Plageman > > wrote: > > > I pushed the straightforward option for now so that it's fixed. > > > > I

Re: A few patches to clarify snapshot management

2024-12-20 Thread Heikki Linnakangas
On 16/12/2024 23:56, Nathan Bossart wrote: On Mon, Dec 16, 2024 at 12:06:33PM +0200, Heikki Linnakangas wrote: While working on the CSN snapshot patch, I got sidetracked looking closer into the snapshot tracking in snapmgr.c. Attached are a few patches to clarify some things. I haven't yet loo

Re: AIO v2.0

2024-12-20 Thread Jelte Fennema-Nio
On Fri, 20 Dec 2024 at 01:54, Andres Freund wrote: > Arguably the configuration *did* tell us, by having a higher hard limit... > > But opting into a higher rlimit, while obviously adhering to the hard limit > and perhaps some other config knob, seems fine? Yes, totally fine. That's exactly the

Re: AIO v2.0

2024-12-20 Thread Andres Freund
Hi, On 2024-12-20 18:27:13 +0100, Jelte Fennema-Nio wrote: > On Fri, 20 Dec 2024 at 01:54, Andres Freund wrote: > > Arguably the configuration *did* tell us, by having a higher hard limit... > > > > But opting into a higher rlimit, while obviously adhering to the hard limit > > and perhaps some

Re: per backend I/O statistics

2024-12-20 Thread Bertrand Drouvot
Hi, On Fri, Dec 20, 2024 at 08:00:00AM +0200, Alexander Lakhin wrote: > Hello Michael, > > 19.12.2024 06:21, Michael Paquier wrote: > > Fixed that, bumped the two version counters, and done. > > Could you, please, look at recent failures produced by grassquit (which > has fsync = on in it's conf

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-20 Thread Amit Kapila
On Mon, Dec 16, 2024 at 4:10 PM Nisha Moond wrote: > > Here is the v56 patch set with the above comments incorporated. > Review comments: === 1. + { + {"idle_replication_slot_timeout", PGC_SIGHUP, REPLICATION_SENDING, + gettext_noop("Sets the duration a replication slot can remain idl

Re: per backend I/O statistics

2024-12-20 Thread Bertrand Drouvot
Hi, On Thu, Dec 19, 2024 at 06:12:04AM +, Bertrand Drouvot wrote: > On Thu, Dec 19, 2024 at 01:21:54PM +0900, Michael Paquier wrote: > > bumped the two version counters, and done. > >The existing structure could be expanded in the > >future to add more information about other statisti

Re: Parametrization minimum password lenght

2024-12-20 Thread Bertrand Drouvot
On Thu, Dec 19, 2024 at 09:57:31AM -0600, Nathan Bossart wrote: > On Thu, Dec 19, 2024 at 09:36:17AM -0600, Nathan Bossart wrote: > > On Thu, Dec 19, 2024 at 07:25:30AM +, Bertrand Drouvot wrote: > >> - errmsg("password is too short"))); > >> +

Re: per backend I/O statistics

2024-12-20 Thread Michael Paquier
On Fri, Dec 20, 2024 at 09:09:00AM +, Bertrand Drouvot wrote: > Thanks for the report! I was not able able to reproduce (even with > asan-enabled) > but I think the test is wrong. Indeed the backend could fsync too during the > test > (see register_dirty_segment() and the case where the reque

Re: Test to dump and restore objects left behind by regression

2024-12-20 Thread Ashutosh Bapat
On Wed, Dec 18, 2024 at 7:39 PM Daniel Gustafsson wrote: > > > On 18 Dec 2024, at 12:28, Ashutosh Bapat > > wrote: > > In general I think it's fine to have such an expensive test gated behind a > PG_TEST_EXTRA flag, and since it's only run on demand we might as well run it > for all formats whil

Re: Make tuple deformation faster

2024-12-20 Thread David Rowley
On Thu, 5 Dec 2024 at 13:09, Andres Freund wrote: > On 2024-12-05 11:52:01 +1300, David Rowley wrote: > > On Thu, 5 Dec 2024 at 03:51, Andres Freund wrote: > > > Possibly stupid idea: Could we instead store the attributes *before* the > > > main > > > TupleDescData, with increasing "distance" fo

Re: Bug: mdunlinkfiletag unlinks mainfork seg.0 instead of indicated fork+segment

2024-12-20 Thread Matthias van de Meent
On Sat, 21 Dec 2024 at 01:05, Thomas Munro wrote: > > On Sat, Dec 21, 2024 at 11:41 AM Matthias van de Meent > wrote: > > The unlinking of forks in the FileTag infrastructure has been broken > > since b0a55e43 in PG16, > > while a segment number other than 0 has never > > been unlinked (at least

Re: a litter question about mdunlinkfiletag function

2024-12-20 Thread Matthias van de Meent
On Tue, 15 Oct 2024 at 04:50, px shi wrote: >> >> You will find other places where relpathperm() is called without having >> a FileTag structure available, e.g. ReorderBufferProcessTXN(). > > > I apologize for the confusion. What I meant to say is that in the > mdunlinkfiletag() function, the for

Re: Possible integer overflow in bringetbitmap()

2024-12-20 Thread Michael Paquier
On Tue, Dec 10, 2024 at 12:33:08PM +0900, Michael Paquier wrote: > Sure, you could do (a) and (b) together. It also seems to me that it > is just simpler to make totalpages a int64 to map automatically with > the result expected by the caller of bringetbitmap(), and we know that > based on MaxBloc

Re: Bug: mdunlinkfiletag unlinks mainfork seg.0 instead of indicated fork+segment

2024-12-20 Thread Thomas Munro
On Sat, Dec 21, 2024 at 11:41 AM Matthias van de Meent wrote: > The unlinking of forks in the FileTag infrastructure has been broken > since b0a55e43 in PG16, Well spotted. -p = relpathperm(ftag->rlocator, MAIN_FORKNUM); +p = _mdfd_segpath_rflb(rlfb, ftag->forknum, ftag->segno); As you

Add XMLNamespaces to XMLElement

2024-12-20 Thread Jim Jones
Hi, I'd like to propose the implementation of the XMLNamespaces option for XMLElement. XMLNAMESPACES(nsuri AS nsprefix) XMLNAMESPACES(DEFAULT default-nsuri) XMLNAMESPACES(NO DEFAULT) * nsprefix:  Namespace's prefix. * nsuri: Namespace's URI. * DEFAULT default-nsuri: S

Re: Statistics Import and Export

2024-12-20 Thread Jeff Davis
On Thu, 2024-12-19 at 21:23 -0800, Jeff Davis wrote: > > 0001-0005 - changes to pg_dump/pg_upgrade > > Attached is a version 36j... The testing can use some work here. I noticed that if I take out the stats entirely, the tests still pass, because pg_upgrade still gets the same before/after result

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Tom Lane
Trey Boudreau writes: > so I'd like to propose a 'LISTEN *' equivalent to 'UNLISTEN *'. Seems reasonable in the abstract, and given the UNLISTEN * precedent it's hard to quibble with that syntax choice. I think what actually needs discussing are the semantics, specifically how this'd interact wi

Re: [PoC] Federated Authn/z with OAUTHBEARER

2024-12-20 Thread Daniel Gustafsson
> On 20 Dec 2024, at 02:00, Jacob Champion > wrote: Thanks for the new version, I was doing a v39 review but I'll roll that over into a v40 review now. As I was reading I was trying to identify parts can be broken out and committed ahead of time. This not only to trim down size, but mostly to

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread David G. Johnston
On Friday, December 20, 2024, Tom Lane wrote: > Trey Boudreau writes: > > * "Listen to all but X" seems like a reasonable desire. > This I concur with, and would add: let me name my channels accounting.payables, accounting.receivables, sales.leads; and let me listen or ignore all accounting/sal

Bug: mdunlinkfiletag unlinks mainfork seg.0 instead of indicated fork+segment

2024-12-20 Thread Matthias van de Meent
Hi, I noticed that the MD smgr internals misbehave when unlink requests for specific forks or specific segments are sent through SyncOps, as it currently always unlinks segment 0 of the main fork, even if only a different fork and/or segment was requested. While probably not extremely critical, i

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Tom Lane
"David G. Johnston" writes: > On Friday, December 20, 2024, Tom Lane wrote: >> * "Listen to all but X" seems like a reasonable desire. > This I concur with, and would add: let me name my channels > accounting.payables, accounting.receivables, sales.leads; and let me listen > or ignore all accoun

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Daniel Gustafsson
> On 20 Dec 2024, at 23:07, Tom Lane wrote: > ..it makes "LISTEN *" act the same as though you had somehow explicitly listed > every possible channel. When thinking about it while reading this thread, this is what I came up with as well. Since the current workings of LISTEN is so well establish

Re: pure parsers and reentrant scanners

2024-12-20 Thread Peter Eisentraut
On 20.12.24 02:07, Tom Lane wrote: I noticed that lapwing is bleating about ccache gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=st

Re: downgrade some aclchk.c errors to internal

2024-12-20 Thread Peter Eisentraut
On 20.12.24 12:47, Peter Eisentraut wrote: In aclchk.c, there are a few error messages that use ereport() but it seems like they should be internal error messages.  Moreover, they are using get_object_class_descr(), which is only meant for internal errors.  (For example, it does not have trans

Re: Can rs_cindex be < 0 for bitmap heap scans?

2024-12-20 Thread Melanie Plageman
On Thu, Dec 19, 2024 at 9:36 PM Richard Guo wrote: > > On Fri, Dec 20, 2024 at 7:50 AM Melanie Plageman > wrote: > Looks correct to me. > > BTW, I kind of doubt that the overflow risk fixed in 28328ec87 is a > real issue in real-world scenarios. Is it realistically possible to > have more than I

Additional comments around need_escapes in pg_parse_json()

2024-12-20 Thread Corey Huinker
I recently had occasion to use the pg_parse_json() function for creating input functions for pg_ndistinct and pg_dependencies. While it is obvious now that I should have been parsing with need_escapes=true, it wasn't obvious from the outset, and so I'm proposing adding a few comments to help the n

Some ExecSeqScan optimizations

2024-12-20 Thread Amit Langote
Hi, I’ve been looking into possible optimizations for ExecSeqScan() and chatting with colleagues about cases where it shows up prominently in analytics-style query plans. For example, in queries like SELECT agg(somecol) FROM big_table WHERE , ExecScan() often dominates the profile. Digging into i

downgrade some aclchk.c errors to internal

2024-12-20 Thread Peter Eisentraut
In aclchk.c, there are a few error messages that use ereport() but it seems like they should be internal error messages. Moreover, they are using get_object_class_descr(), which is only meant for internal errors. (For example, it does not have translation support.) I dug through this and it

Re: Fix crash when non-creator being an iteration on shared radix tree

2024-12-20 Thread John Naylor
On Fri, Dec 20, 2024 at 4:12 AM Masahiko Sawada wrote: > > On Wed, Dec 18, 2024 at 10:32 PM John Naylor wrote: > > 2. The iter_context is separate because the creator's new context > > could be a bump context which doesn't support pfree. But above we > > assume we can pfree in the caller's conte

Re: Make tuple deformation faster

2024-12-20 Thread David Rowley
On Fri, 20 Dec 2024 at 23:04, David Rowley wrote: > I've now pushed the 0001 patch and the 0002 patch as one commit. I'm > going to give the buildfarm a bit of time then push the commit to > remove pg_attribute.attcacheoff. I'm planning on letting that settle > overnight then if all is well push

Re: pure parsers and reentrant scanners

2024-12-20 Thread Tom Lane
Peter Eisentraut writes: > On 20.12.24 02:07, Tom Lane wrote: >> I noticed that lapwing is bleating about >> cubescan.c:1689:5: warning: no previous prototype for 'cube_yyget_column' >> [-Wmissing-prototypes] >> cubescan.c:1765:6: warning: no previous prototype for 'cube_yyset_column' >> [-Wmiss

Re: pure parsers and reentrant scanners

2024-12-20 Thread Julien Rouhaud
On Fri, Dec 20, 2024 at 11:23 PM Tom Lane wrote: > > Could we get that animal updated to > some newer OS version? There is already adder animal that is running debian sid on i386. The only remaining interest in lapwing is to have older versions of everything, so if that's useless I can just tras

Re: pure parsers and reentrant scanners

2024-12-20 Thread Tom Lane
Julien Rouhaud writes: > On Fri, Dec 20, 2024 at 11:23 PM Tom Lane wrote: >> Could we get that animal updated to >> some newer OS version? > There is already adder animal that is running debian sid on i386. The > only remaining interest in lapwing is to have older versions of > everything, so i

Re: Speed up ICU case conversion by using ucasemap_utf8To*()

2024-12-20 Thread Jeff Davis
On Fri, 2024-12-20 at 06:20 +0100, Andreas Karlsson wrote: > SELECT count(upper) FROM (SELECT upper(('Kålhuvud ' || i) COLLATE > "sv-SE-x-icu") FROM generate_series(1, 100) i); > > master:  ~540 ms > Patched: ~460 ms > glibc:   ~410 ms It looks like you are opening and closing the UCaseMap o

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-20 Thread Masahiko Sawada
On Thu, Dec 19, 2024 at 6:31 PM Michael Paquier wrote: > > On Thu, Dec 19, 2024 at 09:27:04AM -0800, Masahiko Sawada wrote: > > On Wed, Dec 18, 2024 at 6:26 PM Michael Paquier wrote: > >> v2 does not have these weaknesses by design. > > > > I agree that v2 is better than v3 in terms of that. > >

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Tom Lane
Trey Boudreau writes: > My first pass at the documentation looks like this: > > The special wildcard * cancels all listener > registrations for the current session and replaces them with a > virtual registration that matches all channels. Further > LISTEN and UNLISTEN class="parameter">cha

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Trey Boudreau
> On Dec 20, 2024, at 2:58 PM, Tom Lane wrote: > Seems reasonable in the abstract, and given the UNLISTEN * precedent > it's hard to quibble with that syntax choice. I think what actually > needs discussing are the semantics, specifically how this'd interact > with other LISTEN/UNLISTEN actions

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread David G. Johnston
On Fri, Dec 20, 2024 at 1:58 PM Tom Lane wrote: > Trey Boudreau writes: > > so I'd like to propose a 'LISTEN *' equivalent to 'UNLISTEN *'. > > Seems reasonable in the abstract, and given the UNLISTEN * precedent > it's hard to quibble with that syntax choice. I think what actually > needs disc

Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Trey Boudreau
Howdy all, NOTE: Grey-beard coder, pgsql newbie. All info/tips/suggestions welcome! I have a use-case where I’d like to LISTEN for all NOTIFY channels. Right now I simply issue a LISTEN for every channel name of interest, but in production the channels will number in the low thousands. The cur

Re: Fix crash when non-creator being an iteration on shared radix tree

2024-12-20 Thread Masahiko Sawada
On Fri, Dec 20, 2024 at 2:27 AM John Naylor wrote: > > On Fri, Dec 20, 2024 at 4:12 AM Masahiko Sawada wrote: > > > > On Wed, Dec 18, 2024 at 10:32 PM John Naylor > > wrote: > > > > 2. The iter_context is separate because the creator's new context > > > could be a bump context which doesn't sup

Re: allow changing autovacuum_max_workers without restarting

2024-12-20 Thread Nathan Bossart
I think I've been saying I would commit this since August, but now I am planning to do so first thing in the new year. In v11 of the patch, I moved the initial startup WARNING to autovac_init() to avoid repeatedly logging when the launcher restarts (e.g., for emergency vacuums when the autovacuum

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread David G. Johnston
On Fri, Dec 20, 2024 at 2:42 PM Trey Boudreau wrote: > > > On Dec 20, 2024, at 2:58 PM, Tom Lane wrote: > > Seems reasonable in the abstract, and given the UNLISTEN * precedent > > it's hard to quibble with that syntax choice. I think what actually > > needs discussing are the semantics, specif

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread David G. Johnston
On Fri, Dec 20, 2024 at 2:42 PM Trey Boudreau wrote: > We could have a different set of keywords, like LISTEN_ALL/UNLISTEN_ALL > that doesn’t interfere with the existing behavior. > > I think we will need something along these lines. We've given * a meaning in UNLISTEN * that doesn't match what

wrong comments in ClassifyUtilityCommandAsReadOnly

2024-12-20 Thread jian he
hi. /* * Determine the degree to which a utility command is read only. * * Note the definitions of the relevant flags in src/include/utility/tcop.h. */ static int ClassifyUtilityCommandAsReadOnly(Node *parsetree) Is the comment wrong? it should be " * Note the definitions of the relevant fla

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Vik Fearing
On 20/12/2024 23:45, Tom Lane wrote: Don't think that quite flies. We might have to regurgitate the state explicitly: LISTEN * UNLISTEN foo.* LISTEN foo.bar.* showing that we're listening to channels foo.bar.*, but not other channels beginning "foo", and also to all c

Re: Fix logging for invalid recovery timeline

2024-12-20 Thread Andrey M. Borodin
> On 20 Dec 2024, at 20:37, David Steele wrote: > > "Latest checkpoint is at %X/%X on timeline %u, but in the history of the > requested timeline, the server forked off from that timeline at %X/%X." I think errdetai here is very hard to follow. I seem to understand what is going on after re

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Vik Fearing
On 21/12/2024 05:23, Tom Lane wrote: Vik Fearing writes: Could I perhaps propose a sort of wildmat[1] syntax? The above sequence could be expressed simply as:     LISTEN *,!foo.*,foo.bar.* That doesn't absolve you from having to say what happens if the user then issues another "LISTEN zed"

Re: Discussion on a LISTEN-ALL syntax

2024-12-20 Thread Tom Lane
Vik Fearing writes: > Could I perhaps propose a sort of wildmat[1] syntax? > The above sequence could be expressed simply as: >     LISTEN *,!foo.*,foo.bar.* That doesn't absolve you from having to say what happens if the user then issues another "LISTEN zed" or "UNLISTEN foo.bar.baz" command.

Re: Fix crash when non-creator being an iteration on shared radix tree

2024-12-20 Thread John Naylor
On Sat, Dec 21, 2024 at 2:17 AM Masahiko Sawada wrote: > > On Fri, Dec 20, 2024 at 2:27 AM John Naylor wrote: > > v3-0001 allocates the iter data in the caller's context. It's a small > > patch, but still a core behavior change so proposed for master-only. I > > believe your v1 is still the leas

Re: Add XMLNamespaces to XMLElement

2024-12-20 Thread Pavel Stehule
Hi so 21. 12. 2024 v 0:51 odesílatel Jim Jones napsal: > Hi, > > I'd like to propose the implementation of the XMLNamespaces option for > XMLElement. > > XMLNAMESPACES(nsuri AS nsprefix) > XMLNAMESPACES(DEFAULT default-nsuri) > XMLNAMESPACES(NO DEFAULT) > > * nsprefix: Namespace's p