Re: Various small doc improvements; plpgsql, schemas, permissions, oidvector

2024-03-30 Thread Andrey M. Borodin
> On 30 Mar 2024, at 11:39, Karl O. Pinc wrote: > > Well, Friday has come and gone and I've not gotten to this. > I'll see if I can spend time tomorrow. No worries, Karl! I just wanted to know if anyone is interested in this thread, and, now is obvious that you are. Thanks for your work! B

Re: Statistics Import and Export

2024-03-30 Thread Magnus Hagander
On Sat, Mar 30, 2024 at 1:26 AM Corey Huinker wrote: > > > On Fri, Mar 29, 2024 at 7:34 PM Jeff Davis wrote: > >> On Fri, 2024-03-29 at 18:02 -0400, Stephen Frost wrote: >> > I’d certainly think “with stats” would be the preferred default of >> > our users. >> >> I'm concerned there could still

Fix out-of-bounds in the function PQescapeinternal (src/interfaces/libpq/fe-exec.c)

2024-03-30 Thread Ranier Vilela
Hi, While I working in [1], Coverity reported some errors: src/bin/pg_basebackup/pg_createsubscriber.c CID 1542690: (#1 of 2): Out-of-bounds access (OVERRUN) alloc_strlen: Allocating insufficient memory for the terminating null of the string. [Note: The source code implementation of the function

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-30 Thread Robert Haas
On Sat, Mar 30, 2024 at 1:57 AM Melanie Plageman wrote: > I think that we are actually successfully removing more RECENTLY_DEAD > HOT tuples than in master with heap_page_prune()'s new approach, and I > think it is correct; but let me know if I am missing something. /me blinks. Isn't zero the on

Re: pgbnech: allow to cancel queries during benchmark

2024-03-30 Thread Alvaro Herrera
Due to commit 61461a300c1c, this patch needs to be reworked. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "La fuerza no está en los medios físicos sino que reside en una voluntad indomable" (Gandhi)

Re: Building with musl in CI and the build farm

2024-03-30 Thread walther
Here's an update on the progress to run musl (Alpine Linux) in the buildfarm. Wolfgang Walther: The animal runs in a docker container via GitHub Actions in [2]. Right now it's still running with --test, until I get the credentials to activate it. The animals have been activated and are repor

Re: Add new error_action COPY ON_ERROR "log"

2024-03-30 Thread Bharath Rupireddy
On Thu, Mar 28, 2024 at 6:31 PM Masahiko Sawada wrote: > > That is, > since the LOG_VERBOSITY option is an enum parameter, it might make > more sense to require the value, instead of making the value optional. > For example, the following command could not be obvious for users: > > COPY test FROM

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-30 Thread Alexander Korotkov
On Fri, Mar 29, 2024 at 6:50 PM Euler Taveira wrote: > On Fri, Mar 29, 2024, at 9:44 AM, Alexander Korotkov wrote: > > This generally makes sense, but I'm not sure about this. The > milliseconds timeout was used initially but received critics in [1]. > > > Alexander, I see why you changed the pat

Re: Combine Prune and Freeze records emitted by vacuum

2024-03-30 Thread Melanie Plageman
On Sat, Mar 30, 2024 at 8:00 AM Robert Haas wrote: > > On Sat, Mar 30, 2024 at 1:57 AM Melanie Plageman > wrote: > > I think that we are actually successfully removing more RECENTLY_DEAD > > HOT tuples than in master with heap_page_prune()'s new approach, and I > > think it is correct; but let me

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-30 Thread Kartyshov Ivan
Thank you Alexander for working on patch, may be we should change some names: 1) test 043_wait_lsn.pl -> to 043_waitlsn.pl like waitlsn.c and waitlsn.h In waitlsn.c and waitlsn.h variables: 2) targret_lsn -> trgLSN like curLSN 3) lsn -> trgLSN like curLSN -- Ivan Kartyshov Postgres Professio

RE: Psql meta-command conninfo+

2024-03-30 Thread Maiquel Grassi
> For the current patch, I have a few more comments. > 1/ The output should be reorganized to show the fields that are part of > “conninfo” first. > With regards to the documentation. I think it's a good idea that every field > has an > description. However, I have some comments: > 1/ Fo

Re: broken JIT support on Fedora 40

2024-03-30 Thread Dmitry Dolgov
> On Sat, Mar 30, 2024 at 04:38:11PM +1300, Thomas Munro wrote: > On Fri, Mar 22, 2024 at 7:15 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > For verification, I've modified the deform.outblock to call LLVMBuildRet > > > instead of LLVMBuildRetVoid and this seems to help -- inline and deform

Re: Statistics Import and Export

2024-03-30 Thread Jeff Davis
On Sat, 2024-03-30 at 01:34 -0400, Corey Huinker wrote: > > - 002pg_upgrade.pl now dumps before/after databases with --no- > statistics. I tried to find out why some tables were getting their > relstats either not set, or set and reset, never affecting the > attribute stats. I even tried turning a

Re: Psql meta-command conninfo+

2024-03-30 Thread Alvaro Herrera
Hello, Note that, in the patch as posted, the column names are not translatable. In order to be translatable, the code needs to do something like appendPQExpBuffer(&buf, " NULL AS \"%s\",\n" " NULL AS \"%s\",\n" " NULL AS \"%s\",\n"

Re: Statistics Import and Export

2024-03-30 Thread Tom Lane
Jeff Davis writes: > This re-raises the question of whether stats are part of a schema-only > dump or not. Did we settle conclusively that they are? Surely they are data, not schema. It would make zero sense to restore them if you aren't restoring the data they describe. Hence, it'll be a bit m

Re: Statistics Import and Export

2024-03-30 Thread Jeff Davis
On Sat, 2024-03-30 at 13:18 -0400, Tom Lane wrote: > Surely they are data, not schema.  It would make zero sense to > restore > them if you aren't restoring the data they describe. The complexity is that pg_upgrade does create the data, but relies on a schema-only dump. So we'd need to at least ac

Re: Statistics Import and Export

2024-03-30 Thread Tom Lane
Jeff Davis writes: > On Sat, 2024-03-30 at 13:18 -0400, Tom Lane wrote: >> Surely they are data, not schema. It would make zero sense to >> restore them if you aren't restoring the data they describe. > The complexity is that pg_upgrade does create the data, but relies on a > schema-only dump. S

Re: Add pg_basetype() function to obtain a DOMAIN base type

2024-03-30 Thread Tom Lane
I wrote: > A fairly common solution to that is to return NULL for bad input, > but in this case we could just have it return the OID unchanged. After sleeping on it, I concluded that was a bad idea and we'd be best off returning NULL for invalid type OIDs. So this is just about back to Steve's or

Re: Statistics Import and Export

2024-03-30 Thread Jeff Davis
On Sat, 2024-03-30 at 13:39 -0400, Tom Lane wrote: > (You could also imagine an explicit positive --stats switch that > would > override --schema-only, but I don't see that it's worth the trouble.) That would have its own utility for reproducing planner problems outside of production systems. (Th

Re: Popcount optimization using AVX512

2024-03-30 Thread Nathan Bossart
I used John Naylor's test_popcount module [0] to put together the attached graphs (note that the "small arrays" one is semi-logarithmic). For both graphs, the X-axis is the number of 64-bit words in the array, and Y-axis is the amount of time in milliseconds to run pg_popcount() on it 100,000 time

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-03-30 Thread Thomas Munro
On Thu, Sep 7, 2023 at 11:44 PM Daniel Gustafsson wrote: > > On 7 Sep 2023, at 13:30, Thomas Munro wrote: > > I don't like the idea that our *next* release's library version > > horizon is controlled by Red Hat's "ELS" phase. > > Agreed. If we instead fence it by "only non-EOL version" then 1.1.

Re: Security lessons from liblzma

2024-03-30 Thread Robert Haas
On Fri, Mar 29, 2024 at 7:00 PM Andres Freund wrote: > I am doubtful that every committer would find something sneaky hidden in > e.g. one of the test changes in a large commit. It's not too hard to hide > something sneaky. I comparison to that hiding something in configure.ac seems > less likely

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-03-30 Thread Tom Lane
Thomas Munro writes: > I was reminded of this thread by ambient security paranoia. As it > stands, we require 1.0.2 (but we very much hope that package > maintainers and others in control of builds don't decide to use it). > Should we skip 1.1.1 and move to requiring 3 for v17? I'd be kind of sa

Re: Security lessons from liblzma

2024-03-30 Thread Andres Freund
Hi, On 2024-03-30 16:50:26 -0400, Robert Haas wrote: > We might also want to move toward signing commits and tags. One of the > meson maintainers was recommending that on-list not long ago. I don't know how valuable the commit signing really is, but I strongly agree that we should sign both tags

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-03-30 Thread Thomas Munro
On Sun, Mar 31, 2024 at 9:59 AM Tom Lane wrote: > Thomas Munro writes: > > I was reminded of this thread by ambient security paranoia. As it > > stands, we require 1.0.2 (but we very much hope that package > > maintainers and others in control of builds don't decide to use it). > > Should we ski

Re: Table AM Interface Enhancements

2024-03-30 Thread Alexander Korotkov
Hi, Pavel! I've pushed 0001, 0002 and 0006. On Fri, Mar 29, 2024 at 5:23 PM Pavel Borisov wrote: >> >> I think for better code look this could be removed: >> >vlock: >> > CHECK_FOR_INTERRUPTS(); >> together with CHECK_FOR_INTERRUPTS(); in heapam_tuple_insert_with_arbiter

Re: Statistics Import and Export

2024-03-30 Thread Corey Huinker
> > I'm getting late into this discussion and I apologize if I've missed this > being discussed before. But. > > Please don't. > > That will make it *really* hard for any form of automation or drivers of > this. The information needs to go somewhere where such tools can easily > consume it, and an

Re: Statistics Import and Export

2024-03-30 Thread Corey Huinker
> > I didn't have any specific proposal in mind, was just trying to think > outside the box. > What if we added a separate resection SECTION_STATISTICS which is run following post-data?

Re: Security lessons from liblzma

2024-03-30 Thread Bruce Momjian
On Sat, Mar 30, 2024 at 04:50:26PM -0400, Robert Haas wrote: > On Fri, Mar 29, 2024 at 7:00 PM Andres Freund wrote: > > I am doubtful that every committer would find something sneaky hidden in > > e.g. one of the test changes in a large commit. It's not too hard to hide > > something sneaky. I com

Re: broken JIT support on Fedora 40

2024-03-30 Thread Thomas Munro
On Sun, Mar 31, 2024 at 5:59 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Yeah, sorry, I'm a bit baffled about this situation myself. Yesterday > I've opened a one-line PR fix that should address the issue, maybe this > would help. In the meantime I've attached what did work for me as a > work

Re: Security lessons from liblzma

2024-03-30 Thread Joe Conway
On 3/30/24 17:12, Andres Freund wrote: Hi, On 2024-03-30 16:50:26 -0400, Robert Haas wrote: We might also want to move toward signing commits and tags. One of the meson maintainers was recommending that on-list not long ago. I don't know how valuable the commit signing really is, but I strong

Re: Security lessons from liblzma

2024-03-30 Thread Joe Conway
On 3/30/24 19:54, Joe Conway wrote: On 2024-03-30 16:50:26 -0400, Robert Haas wrote: or what Tom does when he builds the release tarballs. Tom follows this, at least last time I checked: https://wiki.postgresql.org/wiki/Release_process Reading through that, I wonder if this part is true any

Re: Statistics Import and Export

2024-03-30 Thread Tom Lane
Corey Huinker writes: >> I didn't have any specific proposal in mind, was just trying to think >> outside the box. > What if we added a separate resection SECTION_STATISTICS which is run > following post-data? Maybe, but that would have a lot of side-effects on pg_dump's API and probably on some

Re: Security lessons from liblzma

2024-03-30 Thread Tom Lane
Joe Conway writes: > On 3/30/24 19:54, Joe Conway wrote: >> Tom follows this, at least last time I checked: >> https://wiki.postgresql.org/wiki/Release_process > Reading through that, I wonder if this part is true anymore: >In principle this could be done anywhere, but again there's a concer

Re: pg_combinebackup --copy-file-range

2024-03-30 Thread Thomas Munro
On Sun, Mar 31, 2024 at 1:37 PM Tomas Vondra wrote: > So I decided to take a stab at Thomas' idea, i.e. reading the data to > ... > I'll see how this works on EXT4/ZFS next ... Wow, very cool! A couple of very quick thoughts/notes: ZFS: the open source version only gained per-file block cloning

Re: Security lessons from liblzma

2024-03-30 Thread Bruce Momjian
On Sat, Mar 30, 2024 at 07:54:00PM -0400, Joe Conway wrote: > Virtually every RPM source, including ours, contains out of tree patches > that get applied on top of the release tarball. At least for the PGDG > packages, it would be nice to integrate them into our git repo as build > options or whate

Re: pg_combinebackup --copy-file-range

2024-03-30 Thread Thomas Munro
+wb = copy_file_range(s->fd, &offsetmap[i], wfd, NULL, BLCKSZ, 0); Can you collect adjacent blocks in one multi-block call? And then I think the contract is that you need to loop if it returns short.

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-30 Thread Alexander Korotkov
Hi! On Sat, Mar 30, 2024 at 6:14 PM Kartyshov Ivan wrote: > > Thank you Alexander for working on patch, may be we should change some > names: > 1) test 043_wait_lsn.pl -> to 043_waitlsn.pl like waitlsn.c and > waitlsn.h I renamed that to 043_wal_replay_wait.pl to match the name of SQL procedure.

Re: a wrong index choose when statistics is out of date

2024-03-30 Thread Andy Fan
Hello everyone, > After some more thoughts about the diference of the two ideas, then I > find we are resolving two different issues, just that in the wrong index > choose cases, both of them should work generally. Here is the formal version for the attribute reloptions direction. commit 0d842

Re: Change GUC hashtable to use simplehash?

2024-03-30 Thread John Naylor
On Thu, Mar 28, 2024 at 12:37 PM Jeff Davis wrote: > > > v21-0003 adds a new file hashfn_unstable.c for convenience functions > > and converts all the duplicate frontend uses of hash_string_pointer. > > Why not make hash_string() inline, too? I'm fine with it either way, > I'm just curious why you

Re: pg_combinebackup --copy-file-range

2024-03-30 Thread Tomas Vondra
On 3/31/24 03:03, Thomas Munro wrote: > On Sun, Mar 31, 2024 at 1:37 PM Tomas Vondra > wrote: >> So I decided to take a stab at Thomas' idea, i.e. reading the data to >> ... >> I'll see how this works on EXT4/ZFS next ... > > Wow, very cool! A couple of very quick thoughts/notes: > > ZFS: the o

Re: pg_combinebackup --copy-file-range

2024-03-30 Thread Thomas Munro
On Sun, Mar 31, 2024 at 5:33 PM Tomas Vondra wrote: > I'm on 2.2.2 (on Linux). But there's something wrong, because the > pg_combinebackup that took ~150s on xfs/btrfs, takes ~900s on ZFS. > > I'm not sure it's a ZFS config issue, though, because it's not CPU or > I/O bound, and I see this on both

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-03-30 Thread Bharath Rupireddy
On Thu, Mar 28, 2024 at 3:13 PM Bertrand Drouvot wrote: > > Regarding 0002: Thanks for reviewing it. > Some testing: > > T1 === > > When the slot is invalidated on the primary, then the reason is propagated to > the sync slot (if any). That's fine but we are loosing the inactive_since on > the

Re: Query execution in Perl TAP tests needs work

2024-03-30 Thread Noah Misch
On Wed, Aug 30, 2023 at 09:48:42AM +1200, Thomas Munro wrote: > On Wed, Aug 30, 2023 at 1:49 AM Noah Misch wrote: > > On Tue, Aug 29, 2023 at 04:25:24PM +1200, Thomas Munro wrote: > > > On Tue, Aug 29, 2023 at 1:48 PM Noah Misch wrote: > > > > https://github.com/cpan-authors/IPC-Run/issues/166#is

Re: generic plans and "initial" pruning

2024-03-30 Thread Andrey M. Borodin
> On 6 Dec 2023, at 23:52, Robert Haas wrote: > > I hope that it's at least somewhat useful. > > On 5 Jan 2024, at 15:46, vignesh C wrote: > > There is a leak reported Hi Amit, this is a kind reminder that some feedback on your patch[0] is waiting for your reply. Thank you for your w

Re: [PATCH] pgbench log file headers

2024-03-30 Thread Andrey M. Borodin
> On 6 Jan 2024, at 20:20, vignesh C wrote: > > One of the test has failed in CFBOT at[1] with: Hi Adam, This is a kind reminder that CF entry [0] is waiting for an update. Thanks! Best regards, Andrey Borodin. [0] https://commitfest.postgresql.org/47/4660/

Re: pg_combinebackup --copy-file-range

2024-03-30 Thread Tomas Vondra
On 3/31/24 06:46, Thomas Munro wrote: > On Sun, Mar 31, 2024 at 5:33 PM Tomas Vondra > wrote: >> I'm on 2.2.2 (on Linux). But there's something wrong, because the >> pg_combinebackup that took ~150s on xfs/btrfs, takes ~900s on ZFS. >> >> I'm not sure it's a ZFS config issue, though, because it

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-30 Thread Bharath Rupireddy
On Sun, Mar 31, 2024 at 7:41 AM Alexander Korotkov wrote: > > Hi! Thanks for the patch. I have a few comments on the v16 patch. 1. Is there any specific reason for pg_wal_replay_wait() being a procedure rather than a function? I haven't read the full thread, but I vaguely noticed the discussion