Re: Improve heapgetpage() performance, overhead from serializable

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 12:07:22 +0700, John Naylor wrote: > Just in time ;-) The commit message should also have "reviewed by > Zhang Mingli" and "tested by Quan Zongliang", who shared results in a > thread for a withrawn CF entry with a similar idea but covering fewer > cases: Good call. Added and pu

Re: [PATCH] Exponential backoff for auth_delay

2024-04-07 Thread Michael Banck
Hi, On Wed, Mar 20, 2024 at 11:22:12PM +0100, Daniel Gustafsson wrote: > > On 20 Mar 2024, at 22:21, Jacob Champion > > wrote: > > > > On Wed, Mar 20, 2024 at 2:15 PM Jacob Champion > > wrote: > >> I think solutions for case 1 and case 2 are necessarily at odds under > >> the current design, i

Re: Change prefetch and read strategies to use range in pg_prewarm ... and raise a question about posix_fadvise WILLNEED

2024-04-07 Thread Nazir Bilal Yavuz
Hi Andrey, On Sun, 7 Apr 2024 at 08:29, Andrey M. Borodin wrote: > > > > > On 15 Mar 2024, at 17:12, Nazir Bilal Yavuz wrote: > > > > I did not have the time to check other things you mentioned but I > > tested the read performance. The table size is 5.5GB, I did 20 runs in > > total. > > Hi Naz

Re: Table AM Interface Enhancements

2024-04-07 Thread Pavel Borisov
Hi, Alexander! On Sun, 7 Apr 2024 at 07:33, Alexander Korotkov wrote: > Hi, Pavel! > > On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov > wrote: > > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: > >> > >> On Tue, 2024-04-02 at 11:49 +0300, Alexander Korotkov wrote: > >> > I don't like the idea th

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 08:21, Andres Freund wrote: > I added WITH BINARY, SET STORAGE EXTERNAL and tested both unix socket and > localhost. I also reduced row counts and iteration counts, because I am > impatient, and I don't think it matters much here. Attached the modified > version. Thanks for

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 02:49, Andres Freund wrote: > On 2024-04-07 00:19:35 +0200, Daniel Gustafsson wrote: >>> On 6 Apr 2024, at 23:44, Andres Freund wrote: >> The non-context aware fix would be to just print the last 1024 (or something) >> bytes from the logfile: > > That'd be better, yes. I'd m

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread John Naylor
On Sun, Apr 7, 2024 at 9:08 AM John Naylor wrote: > I've attached a mostly-polished update on runtime embeddable values, > storing up to 3 offsets in the child pointer (1 on 32-bit platforms). And...since there's a new bump context patch, I wanted to anticipate squeezing an update on top of that,

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread Jelte Fennema-Nio
On Sun, 7 Apr 2024 at 03:39, Andres Freund wrote: > Changing the global vars to size_t seems mildly bogus to me. All it's > achieving is to use slightly more memory. It also just seems unrelated to the > change. I took a closer look at this. I agree that changing PqSendBufferSize to size_t is unn

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread John Naylor
On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: > I'm planning on pushing these, pending a final look at 0002 and 0003 > on Sunday morning NZ time (UTC+12), likely in about 10 hours time. +1 I haven't looked at v6, but I've tried using it in situ, and it seems to work as well as hoped: http

LDAP & Kerberos test woes

2024-04-07 Thread Heikki Linnakangas
While refactoring the Kerberos test module in preparation for adding libpq encryption negotiation tests [1], I noticed that if the test script die()s during setup, the whole test is marked as SKIPped rather than failed. The cleanup END section is missing this trick: --- a/src/test/kerberos/t/0

Re: Can't find not null constraint, but \d+ shows that

2024-04-07 Thread Tender Wang
It has been several days since the last email. Do you have any suggestions, please? What I'm concerned about is that adding a new AT_PASS is good fix? Is it a big try? More concerned is that it can cover all ALTER TABLE cases? Any thoughts. -- Tender Wang OpenPie: https://en.openpie.com/

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Tomas Vondra
On 4/7/24 06:17, Melanie Plageman wrote: > On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: >> On 4/6/24 23:34, Melanie Plageman wrote: >>> ... I realized it makes more sense to add a FIXME (I used XXX. I'm not when to use what) with a link to the message where Andres

Re: LogwrtResult contended spinlock

2024-04-07 Thread Alvaro Herrera
I pushed the "copy" pointer now, except that I renamed it to "insert", which is what we call the operation being tracked. I also added some comments. One perhaps significant change from what Bharath submitted is that we now use the return value from monotonic advance to return an updated value of

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 22:05, John Naylor wrote: > > On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: > > > I'm planning on pushing these, pending a final look at 0002 and 0003 > > on Sunday morning NZ time (UTC+12), likely in about 10 hours time. > > +1 I've now pushed all 3 patches. Thank

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread David Rowley
On Sun, 7 Apr 2024 at 22:05, Jelte Fennema-Nio wrote: > > On Sun, 7 Apr 2024 at 03:39, Andres Freund wrote: > > Changing the global vars to size_t seems mildly bogus to me. All it's > > achieving is to use slightly more memory. It also just seems unrelated to > > the > > change. > > I took a clo

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread Ranier Vilela
Em sáb., 6 de abr. de 2024 às 22:39, Andres Freund escreveu: > Hi, > > On 2024-04-07 00:45:31 +0200, Jelte Fennema-Nio wrote: > > On Sat, 6 Apr 2024 at 22:21, Andres Freund wrote: > > > The small regression for small results is still kinda visible, I > haven't yet > > > tested the patch downthre

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andrew Dunstan
On 2024-04-06 Sa 20:49, Andres Freund wrote: That's probably unnecessary optimization, but it seems a tad silly to read an entire, potentially sizable, file to just use the last 1k. Not sure if the way slurp_file() uses seek supports negative ofsets, the docs read to me like that may only be su

Re: MultiXact\SLRU buffers configuration

2024-04-07 Thread Andrey M. Borodin
> On 6 Apr 2024, at 14:24, Andrey M. Borodin wrote: > > What do you think? OK, I'll follow this plan. As long as most parts of this thread were committed, I'll mark CF item as "committed". Thanks to everyone involved! See you in a followup thread about sleeping on CV. Best regards, Andrey

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra wrote: > > On 4/7/24 06:17, Melanie Plageman wrote: > >> What bothers me on 0006-0008 is that the justification in the commit > >> messages is "future commit will do something". I think it's fine to have > >> a separate "prepareatory" patches (I really l

Re: remaining sql/json patches

2024-04-07 Thread jian he
On Sun, Apr 7, 2024 at 12:30 PM jian he wrote: > > other than that, it looks good to me. while looking at it again. + | NESTED path_opt Sconst + COLUMNS '(' json_table_column_definition_list ')' + { + JsonTableColumn *n = makeNode(JsonTableColumn); + + n->coltype = JTC_NESTED; + n->pathspec = (Js

Re: remaining sql/json patches

2024-04-07 Thread Amit Langote
On Sun, Apr 7, 2024 at 10:21 PM jian he wrote: > On Sun, Apr 7, 2024 at 12:30 PM jian he wrote: > > > > other than that, it looks good to me. > while looking at it again. > > + | NESTED path_opt Sconst > + COLUMNS '(' json_table_column_definition_list ')' > + { > + JsonTableColumn *n = makeNode(J

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Tomas Vondra
On 4/7/24 15:11, Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra > wrote: >> >> On 4/7/24 06:17, Melanie Plageman wrote: What bothers me on 0006-0008 is that the justification in the commit messages is "future commit will do something". I think it's fine to have >>>

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra wrote: > > > > On 4/7/24 06:17, Melanie Plageman wrote: > > On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: > >> On 4/6/24 23:34, Melanie Plageman wrote: > >>> ... > > I realized it makes more sense to add a FIXME (I used XXX. I'm

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 10:10 AM Tomas Vondra wrote: > > On 4/7/24 15:11, Melanie Plageman wrote: > > > Also, the iterators in the TableScanDescData might be something I > > could live with in the source code for a couple months before we make > > the rest of the changes in July+. But, adding them

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Tomas Vondra
On 4/7/24 16:24, Melanie Plageman wrote: > On Sun, Apr 7, 2024 at 7:38 AM Tomas Vondra > wrote: >> >> >> >> On 4/7/24 06:17, Melanie Plageman wrote: >>> On Sun, Apr 07, 2024 at 02:27:43AM +0200, Tomas Vondra wrote: On 4/6/24 23:34, Melanie Plageman wrote: > ... >> >> I realized

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > On 2024-04-06 Sa 20:49, Andres Freund wrote: >> That's probably unnecessary optimization, but it seems a tad silly to read an >> entire, potentially sizable, file to just use the last 1k. Not sure if the >> way >> slurp_file() uses seek supports

Re: BitmapHeapScan streaming read user and prelim refactoring

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 10:42 AM Tomas Vondra wrote: > > > > On 4/7/24 16:24, Melanie Plageman wrote: > >>> Thanks! I thought about it a bit more, and I got worried about the > >>> > >>> Assert(scan->rs_empty_tuples_pending == 0); > >>> > >>> in heap_rescan() and heap_endscan(). > >>> > >>> I

Re: CASE control block broken by a single line comment

2024-04-07 Thread Erik Wienhold
On 2024-04-07 06:33 +0200, Tom Lane wrote: > Erik Wienhold writes: > > I'm surprised that the comment is not skipped by the scanner at this > > point. Maybe because the parser just reads the raw expression between > > WHEN and THEN with plpgsql_append_source_text via read_sql_construct. > > > Ho

Re: CASE control block broken by a single line comment

2024-04-07 Thread Erik Wienhold
I wrote: > Attached v2 tries to do that. Hit send too soon. Attached now. -- Erik >From 23a20a410dc92946141b6c6fa5100473eac482cf Mon Sep 17 00:00:00 2001 From: Erik Wienhold Date: Sat, 6 Apr 2024 22:36:54 +0200 Subject: [PATCH v2] plpgsql: accept trailing line comment in CASE WHEN Expression

❓ JSON Path Dot Precedence

2024-04-07 Thread David E. Wheeler
Hello Hackers, A question about the behavior of the JSON Path parser. The docs[1] have this to say about numbers: > Numeric literals in SQL/JSON path expressions follow JavaScript rules, which > are different from both SQL and JSON in some minor details. For example, > SQL/JSON path allows .1

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > > On 2024-04-06 Sa 20:49, Andres Freund wrote: > > >> That's probably unnecessary optimization, but it seems a tad silly to read > >> an > >> entire, potentially sizable, file to just use

Re: remaining sql/json patches

2024-04-07 Thread jian he
On Sun, Apr 7, 2024 at 9:36 PM Amit Langote wrote: > > > 0002 needs an expanded commit message but I've run out of energy today. > some cosmetic issues in v51, 0002. in struct JsonTablePathScan, /* ERROR/EMPTY ON ERROR behavior */ bool errorOnError; the comments seem not right. I think "errorOn

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 18:28, Andres Freund wrote: > > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: >>> On 2024-04-06 Sa 20:49, Andres Freund wrote: >> That's probably unnecessary optimization, but it seems a tad silly to read

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Alexander Lakhin
Hello Peter, 03.04.2024 22:53, Peter Geoghegan wrote: On Mon, Apr 1, 2024 at 6:33 PM Peter Geoghegan wrote: Note: v18 doesn't have any adjustments to the costing, as originally planned. I'll probably need to post a revised patch with improved (or at least polished) costing in the next few days

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Andres Freund
On 2024-04-07 18:51:40 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2024, at 18:28, Andres Freund wrote: > > > > On 2024-04-07 16:52:05 +0200, Daniel Gustafsson wrote: > >>> On 7 Apr 2024, at 14:51, Andrew Dunstan wrote: > >>> On 2024-04-06 Sa 20:49, Andres Freund wrote: > >> > That's prob

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Peter Geoghegan
On Sun, Apr 7, 2024 at 1:00 PM Alexander Lakhin wrote: > Please look at an assertion failure (reproduced starting from 5bf748b86), > triggered by the following query: > CREATE TABLE t (a int, b int); > CREATE INDEX t_idx ON t (a, b); > INSERT INTO t (a, b) SELECT g, g FROM generate_series(0, 999)

Re: LDAP & Kerberos test woes

2024-04-07 Thread Heikki Linnakangas
On 07/04/2024 13:19, Heikki Linnakangas wrote: 1st patch fixes the LDAP setup tests, and 2nd patch fixes the error handling in the END blocks. Committed and backpatched these test fixes. -- Heikki Linnakangas Neon (https://neon.tech)

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > I had been planning to commit v14 this morning but got cold feet with > the BMR-based interface. Heikki didn't like it much, and in the end, > neither did I. I have now removed it, and it seems much better. No > other significant changes

Re: pg_combinebackup --copy-file-range

2024-04-07 Thread Tomas Vondra
On 4/7/24 19:46, Tomas Vondra wrote: > On 4/5/24 21:43, Tomas Vondra wrote: >> Hi, >> >> ... >> >> 2) The prefetching is not a huge improvement, at least not for these >> three filesystems (btrfs, ext4, xfs). From the color scale it might seem >> like it helps, but those values are relative to the

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 1:33 PM Nazir Bilal Yavuz wrote: > > Hi, > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > I had been planning to commit v14 this morning but got cold feet with > > the BMR-based interface. Heikki didn't like it much, and in the end, > > neither did I. I have n

Re: MultiXact\SLRU buffers configuration

2024-04-07 Thread Alvaro Herrera
On 2024-Feb-03, Andrey M. Borodin wrote: > Here's the test draft. This test reliably reproduces sleep on CV when waiting > next multixact to be filled into "members" SLRU. > Cost of having this test: > 1. We need a new injection point type "wait" (in addition to "error" and > "notice"). It canno

Re: Trigger violates foreign key constraint

2024-04-07 Thread Tom Lane
Laurenz Albe writes: > Patch v3 is attached. I agree with documenting this hazard, but I think it'd be better to do so in the "Triggers" chapter. There is no hazard unless you are writing user-defined triggers, which is surely far fewer people than use foreign keys. So I suggest something like

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-07 Thread Alexander Lakhin
Hi Alexander and Dmitry, 07.04.2024 01:22, Alexander Korotkov wrote: I've pushed 0001 and 0002. I didn't push 0003 for the following reasons. Please try the following (erroneous) query: CREATE TABLE t1(i int, t text) PARTITION BY LIST (t); CREATE TABLE t1pa PARTITION OF t1 FOR VALUES IN ('A')

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread Andres Freund
Hi, On 2024-04-01 11:53:28 +0900, Masahiko Sawada wrote: > On Fri, Mar 29, 2024 at 4:21 PM John Naylor wrote: > > I've marked it Ready for Committer. > > Thank you! I've attached the patch that I'm going to push tomorrow. Locally I ran a 32bit build with ubsan enabled (by accident actually), whi

Re: MultiXact\SLRU buffers configuration

2024-04-07 Thread Andrey M. Borodin
> On 7 Apr 2024, at 21:41, Alvaro Herrera wrote: > > Well, it would be nice to have *some* test, but as you say it is way > more complex than the thing being tested, and it zooms in on the > functioning of the multixact creation in insane quantum-physics ways ... > to the point that you can no

Re: Table AM Interface Enhancements

2024-04-07 Thread Pavel Borisov
Hi, Alexander! On Sun, 7 Apr 2024 at 12:34, Pavel Borisov wrote: > Hi, Alexander! > > On Sun, 7 Apr 2024 at 07:33, Alexander Korotkov > wrote: > >> Hi, Pavel! >> >> On Fri, Apr 5, 2024 at 6:58 PM Pavel Borisov >> wrote: >> > On Tue, 2 Apr 2024 at 19:17, Jeff Davis wrote: >> >> >> >> On Tue, 2

Re: EXCLUDE COLLATE in CREATE/ALTER TABLE document

2024-04-07 Thread Tom Lane
"Daniel Verite" writes: > Also the use of "and/or" in the previous version conveys the fact > that operator class and ordering options are not mutually > exclusive. But when using "any of the following" in the new text, > doesn't it loose that meaning? Yeah; and/or is perfectly fine here and does

Re: LogwrtResult contended spinlock

2024-04-07 Thread Jeff Davis
On Sun, 2024-04-07 at 14:19 +0200, Alvaro Herrera wrote: > I pushed the "copy" pointer now, except that I renamed it to > "insert", > which is what we call the operation being tracked.  I also added some > comments. The "copy" pointer, as I called it, is not the same as the "insert" pointer (as re

Re: ❓ JSON Path Dot Precedence

2024-04-07 Thread Erik Wienhold
On 2024-04-07 18:13 +0200, David E. Wheeler wrote: > A question about the behavior of the JSON Path parser. The docs[1] > have this to say about numbers: > > > Numeric literals in SQL/JSON path expressions follow JavaScript > > rules, which are different from both SQL and JSON in some minor > >

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Thu, Apr 04, 2024 at 02:03:30PM +0300, Nazir Bilal Yavuz wrote: > > On Wed, 3 Apr 2024 at 23:44, Melanie Plageman > wrote: > > > > I don't see the point of BlockSampler_HasMore() anymore. I removed it in > > the attached and made BlockSampler_Next() return InvalidBlockNumber > > under the sam

Re: Document efficient self-joins / UPDATE LIMIT techniques.

2024-04-07 Thread Tom Lane
Corey Huinker writes: >> I've also used the technique quite a lot, but only using the PK, >> didn't know about the ctid trick, so many thanks for documenting it. > tid-scans only became a thing a few versions ago (12?). Prior to that, PK > was the only way to go. I think we had TID scans for awh

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tomas Vondra
On 4/7/24 14:37, David Rowley wrote: > On Sun, 7 Apr 2024 at 22:05, John Naylor wrote: >> >> On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: >>> >> I'm planning on pushing these, pending a final look at 0002 and 0003 >>> on Sunday morning NZ time (UTC+12), likely in about 10 hours time. >> >>

Re: Security lessons from liblzma

2024-04-07 Thread Ranier Vilela
Hi, Sorry, without any connection with the technical part of the thread. But I couldn't help but record this, and congratulate Andres Freund, for the excellent work. It's not every day that a big press, in Brazil or around the world, publishes something related to technology people. Today I came

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tomas Vondra
On 4/7/24 22:35, Tomas Vondra wrote: > On 4/7/24 14:37, David Rowley wrote: >> On Sun, 7 Apr 2024 at 22:05, John Naylor wrote: >>> >>> On Sat, Apr 6, 2024 at 7:37 PM David Rowley wrote: >>> I'm planning on pushing these, pending a final look at 0002 and 0003 on Sunday morning NZ ti

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Sun, Apr 7, 2024 at 3:57 PM Melanie Plageman wrote: > > On Thu, Apr 04, 2024 at 02:03:30PM +0300, Nazir Bilal Yavuz wrote: > > > > On Wed, 3 Apr 2024 at 23:44, Melanie Plageman > > wrote: > > > > > > I don't see the point of BlockSampler_HasMore() anymore. I removed it in > > > the attached a

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > Hi, > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > I had been planning to commit v14 this morning but got cold feet with > > the BMR-based interface. Heikki didn't like it much, and in the end, > > neither did I. I hav

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 22:35:47 +0200, Tomas Vondra wrote: > I haven't investigated, but I'd considering it works on 64-bit, I guess > it's not considering alignment somewhere. I can dig more if needed. I think I may the problem: #define KeeperBlock(set) ((BumpBlock *) ((char *) (set) + sizeof(BumpC

Re: pub/sub - specifying optional parameters without values.

2024-04-07 Thread Tom Lane
Peter Smith writes: > Here is a similar update for another page: "55.4 Streaming Replication > Protocol" [0]. This patch was prompted by a review comment reply at > [1] (#2). > I've used text almost the same as the boilerplate text added by the > previous commit [2] Pushed, except I put it at th

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tomas Vondra
On 4/7/24 23:09, Andres Freund wrote: > Hi, > > On 2024-04-07 22:35:47 +0200, Tomas Vondra wrote: >> I haven't investigated, but I'd considering it works on 64-bit, I guess >> it's not considering alignment somewhere. I can dig more if needed. > > I think I may the problem: > > > #define Kee

Re: Streaming I/O, vectored I/O (WIP)

2024-04-07 Thread Nazir Bilal Yavuz
Hi, On Mon, 8 Apr 2024 at 00:01, Nazir Bilal Yavuz wrote: > > Hi, > > On Sun, 7 Apr 2024 at 20:33, Nazir Bilal Yavuz wrote: > > > > Hi, > > > > On Tue, 2 Apr 2024 at 11:40, Thomas Munro wrote: > > > > > > I had been planning to commit v14 this morning but got cold feet with > > > the BMR-based

Re: Streaming read-ready sequential scan code

2024-04-07 Thread Thomas Munro
On Sun, Apr 7, 2024 at 1:34 PM Melanie Plageman wrote: > Attached v13 0001 is your fix and 0002 is a new version of the > sequential scan streaming read user. Off-list Andres mentioned that I > really ought to separate the parallel and serial sequential scan users > into two different callbacks. I

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > I've pushed 0001, 0002 and 0006. I briefly looked at 27bc1772fc81 and I don't think the state post this commit makes sense. Before this commit another block based AM could implement analyze without much code duplication. Now a large po

Re: Flushing large data immediately in pqcomm

2024-04-07 Thread Melih Mutlu
David Rowley , 6 Nis 2024 Cmt, 04:34 tarihinde şunu yazdı: > Does anyone else want to try the attached script on the v5 patch to > see if their numbers are better? > I'm seeing the below results with your script on my machine (). I ran it several times, results were almost similar each time. mas

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 16:59:26 -0400, Melanie Plageman wrote: > From 1dc2343661f3edb3b1bc4307afb0e956397eb76c Mon Sep 17 00:00:00 2001 > From: Melanie Plageman > Date: Sun, 7 Apr 2024 14:55:22 -0400 > Subject: [PATCH v10 1/3] Make heapam_scan_analyze_next_[tuple|block] static. > > 27bc1772fc81 remov

Re: ❓ JSON Path Dot Precedence

2024-04-07 Thread David E. Wheeler
On Apr 7, 2024, at 15:46, Erik Wienhold wrote: > I guess jsonpath assumes that hex, octal, and binary literals are > integers. So there's no ambiguity about any fractional part that might > follow. Yeah, that’s what the comment in the flex file says: https://github.com/postgres/postgres/blob/b

Re: Streaming read-ready sequential scan code

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 09:36:59 +1200, Thomas Munro wrote: > I've been on the fence about that flag for sequential scan... Some > days I want to consider changing to READ_STREAM_DEFAULT and relying on > our "anti-heuristics" to suppress advice, which would work out the > same in most cases but might o

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-07 Thread Alexander Korotkov
Hi, Alexander! On Sun, Apr 7, 2024 at 10:00 PM Alexander Lakhin wrote: > 07.04.2024 01:22, Alexander Korotkov wrote: > > I've pushed 0001 and 0002. I didn't push 0003 for the following reasons. > > Please try the following (erroneous) query: > CREATE TABLE t1(i int, t text) PARTITION BY LIST (t)

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Melanie Plageman
On Sun, Apr 07, 2024 at 03:00:00PM -0700, Andres Freund wrote: > Hi, > > On 2024-04-07 16:59:26 -0400, Melanie Plageman wrote: > > From 1dc2343661f3edb3b1bc4307afb0e956397eb76c Mon Sep 17 00:00:00 2001 > > From: Melanie Plageman > > Date: Sun, 7 Apr 2024 14:55:22 -0400 > > Subject: [PATCH v10 1/3

Re: Cluster::restart dumping logs when stop fails

2024-04-07 Thread Daniel Gustafsson
> On 7 Apr 2024, at 18:51, Daniel Gustafsson wrote: >> On 7 Apr 2024, at 18:28, Andres Freund wrote: >> I'm ok with printing path + some content or just the path. > > I think printing the last 512 bytes or so would be a good approach, I'll take > care of it later tonight. That would be a backpa

Re: POC, WIP: OR-clause support for indexes

2024-04-07 Thread Alexander Korotkov
Hi! On Mon, Apr 1, 2024 at 9:38 AM Andrei Lepikhov wrote: > On 28/3/2024 16:54, Alexander Korotkov wrote: > > The current patch has a boolean guc enable_or_transformation. > > However, when we have just a few ORs to be transformated, then we > > should get less performance gain from the transform

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Tom Lane
Tomas Vondra writes: > Yup, changing it to this: > #define KeeperBlock(set) ((BumpBlock *) ((char *) (set) + > MAXALIGN(sizeof(BumpContext > fixes the issue for me. Mamba is happy with that change, too. regards, tom lane

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-07 Thread Michael Paquier
On Sat, Apr 06, 2024 at 07:47:43PM +0200, Daniel Gustafsson wrote: > My apologies, I thought I did but clearly failed. My point was that this is a > special/corner case where we try to find one of two different libraries (with > different ideas about backwards compatability etc) for supporting a s

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-07 Thread Michael Paquier
On Thu, Apr 04, 2024 at 03:10:27PM +1300, David Rowley wrote: > Someone asked me about this, so thought it might be useful to post here. I've received the same question. > To express this as UTC, It's: > > postgres=# select '2024-04-08 00:00-12:00' at time zone 'UTC'; > timezone >

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Daniel Gustafsson
> On 8 Apr 2024, at 00:41, Tom Lane wrote: > > Tomas Vondra writes: >> Yup, changing it to this: > >> #define KeeperBlock(set) ((BumpBlock *) ((char *) (set) + >> MAXALIGN(sizeof(BumpContext > >> fixes the issue for me. > > Mamba is happy with that change, too. Unrelated to that one, see

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-07 Thread Daniel Gustafsson
> On 8 Apr 2024, at 00:46, Michael Paquier wrote: > > On Sat, Apr 06, 2024 at 07:47:43PM +0200, Daniel Gustafsson wrote: >> My apologies, I thought I did but clearly failed. My point was that this is >> a >> special/corner case where we try to find one of two different libraries (with >> differ

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 00:55:42 +0200, Daniel Gustafsson wrote: > > On 8 Apr 2024, at 00:41, Tom Lane wrote: > > > > Tomas Vondra writes: > >> Yup, changing it to this: > > > >> #define KeeperBlock(set) ((BumpBlock *) ((char *) (set) + > >> MAXALIGN(sizeof(BumpContext > > > >> fixes the issue

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread David Rowley
On Mon, 8 Apr 2024 at 09:09, Andres Freund wrote: > I suspect that KeeperBlock() isn't returning true, because IsKeeperBlock > misses > the MAXALIGN(). I think that about fits with: Thanks for investigating that. I've just pushed a fix for the macro and also adjusted a location which was *corre

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-04-07 Thread John Naylor
On Mon, Apr 8, 2024 at 2:07 AM Andres Freund wrote: > > Looking at the code, the failure isn't suprising anymore: > chardata[MaxBlocktableEntrySize]; > BlocktableEntry *page = (BlocktableEntry *) data; > > 'char' doesn't enforce any alignment, but you're storing a Block

Re: Add bump memory context type and use it for tuplesorts

2024-04-07 Thread Daniel Gustafsson
> On 8 Apr 2024, at 01:04, Andres Freund wrote: > What makes you think that's unrelated? To me that looks like the same issue? Nvm, I misread the assert, ETOOLITTLESLEEP. Sorry for the noise. -- Daniel Gustafsson

Re: Table AM Interface Enhancements

2024-04-07 Thread Alexander Korotkov
Hi, On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > I've pushed 0001, 0002 and 0006. > > I briefly looked at 27bc1772fc81 and I don't think the state post this commit > makes sense. Before this commit another block based AM could

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-04-07 Thread Andres Freund
Hi, On 2024-03-15 18:36:55 +0900, Sutou Kouhei wrote: > +warning_level = get_option('warning_level') > +# See https://mesonbuild.com/Builtin-options.html#details-for-warning_level > for > +# warning_level values. > +if warning_level == '1' > + common_builtin_flags += ['-Wall', '/W2'] > +elif war

Re: Table AM Interface Enhancements

2024-04-07 Thread Alexander Korotkov
On Mon, Apr 8, 2024 at 2:25 AM Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I don't think the state post this > > co

Re: Table AM Interface Enhancements

2024-04-07 Thread Andres Freund
Hi, On 2024-04-08 02:25:17 +0300, Alexander Korotkov wrote: > On Mon, Apr 8, 2024 at 12:40 AM Andres Freund wrote: > > On 2024-03-30 23:33:04 +0200, Alexander Korotkov wrote: > > > I've pushed 0001, 0002 and 0006. > > > > I briefly looked at 27bc1772fc81 and I don't think the state post this > >

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Melanie Plageman
On Sat, Apr 6, 2024 at 7:08 PM Thomas Munro wrote: > > On second thoughts, I think the original "invalidate" terminology was > fine, no need to invent a new term. > > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I >

Re: Extension Enhancement: Buffer Invalidation in pg_buffercache

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 11:07:58 +1200, Thomas Munro wrote: > I thought of a better name for the bufmgr.c function though: > InvalidateUnpinnedBuffer(). That name seemed better to me after I > festooned it with warnings about why exactly it's inherently racy and > only for testing use. I still dislike

Re: POC, WIP: OR-clause support for indexes

2024-04-07 Thread Justin Pryzby
On Mon, Apr 08, 2024 at 01:34:37AM +0300, Alexander Korotkov wrote: > Hi! > > On Mon, Apr 1, 2024 at 9:38 AM Andrei Lepikhov > wrote: > > On 28/3/2024 16:54, Alexander Korotkov wrote: > > > The current patch has a boolean guc enable_or_transformation. > > > However, when we have just a few ORs to

Re: Popcount optimization using AVX512

2024-04-07 Thread Tom Lane
Nathan Bossart writes: > Here is what I have staged for commit, which I intend to do shortly. Today's Coverity run produced this warning, which seemingly was triggered by one of these commits, but I can't make much sense of it: *** CID 1596255: Uninitialized variables (UNINIT) /usr/lib/gcc/x86

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Tom Lane
Coverity pointed out something that looks like a potentially live problem in 5bf748b86: /srv/coverity/git/pgsql-git/postgresql/src/backend/access/nbtree/nbtutils.c: 2950 in _bt_preprocess_keys() 2944 * need to make sure that we don't throw away an array 2945

Re: WIP Incremental JSON Parser

2024-04-07 Thread Tom Lane
Coverity complained that this patch leaks memory: /srv/coverity/git/pgsql-git/postgresql/src/bin/pg_combinebackup/load_manifest.c: 212 in load_backup_manifest() 206 bytes_left -= rc; 207 json_parse_manifest_incremental_chunk( 208

Re: meson: Specify -Wformat as a common warning flag for extensions

2024-04-07 Thread Sutou Kouhei
Hi Andres, Thanks for reviewing this! In <20240407232635.fq4kc5556laha...@awork3.anarazel.de> "Re: meson: Specify -Wformat as a common warning flag for extensions" on Sun, 7 Apr 2024 16:26:35 -0700, Andres Freund wrote: > This seems like a fair amount of extra configure tests. Particularly

Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Tom Lane
Today's Coverity run produced this: /srv/coverity/git/pgsql-git/postgresql/src/include/lib/simplehash.h: 1138 in bh_nodeidx_stat() 1132 avg_collisions = 0; 1133 } 1134 1135 sh_log("size: " UINT64_FORMAT ", members: %u, filled: %f, total chain: %u, max chain: %u,

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Peter Geoghegan
On Sun, Apr 7, 2024 at 8:48 PM Tom Lane wrote: > > Coverity pointed out something that looks like a potentially live > problem in 5bf748b86: > > /srv/coverity/git/pgsql-git/postgresql/src/backend/access/nbtree/nbtutils.c: > 2950 in _bt_preprocess_keys() > 2944 * need

Re: Use streaming read API in ANALYZE

2024-04-07 Thread Thomas Munro
On Mon, Apr 8, 2024 at 10:26 AM Melanie Plageman wrote: > On Sun, Apr 07, 2024 at 03:00:00PM -0700, Andres Freund wrote: > > > src/backend/commands/analyze.c | 89 ++ > > > 1 file changed, 26 insertions(+), 63 deletions(-) > > > > That's a very nice demonstration o

Re: Weird test mixup

2024-04-07 Thread Michael Paquier
On Sat, Apr 06, 2024 at 10:34:46AM +0500, Andrey M. Borodin wrote: > I find name of the function "injection_points_local()" strange, > because there is no verb in the name. How about > "injection_points_set_local"? That makes sense. > I'm not sure if we should refactor anything here, but > Injec

Re: Popcount optimization using AVX512

2024-04-07 Thread Nathan Bossart
On Sun, Apr 07, 2024 at 08:42:12PM -0400, Tom Lane wrote: > Today's Coverity run produced this warning, which seemingly was > triggered by one of these commits, but I can't make much sense > of it: > > *** CID 1596255: Uninitialized variables (UNINIT) > /usr/lib/gcc/x86_64-linux-gnu/10/include/a

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2024-04-07 Thread Tom Lane
Peter Geoghegan writes: > On Sun, Apr 7, 2024 at 8:48 PM Tom Lane wrote: >> Coverity pointed out something that looks like a potentially live >> problem in 5bf748b86: >> ... which certainly makes it look like array->scan_key could be >> a null-pointer dereference. > But the "Assert(xform[j].ikey

Re: Experiments with Postgres and SSL

2024-04-07 Thread Heikki Linnakangas
Committed this. Thank you to everyone involved! On 04/04/2024 14:08, Matthias van de Meent wrote: Patch 0003: The read size in secure_raw_read is capped to port->raw_buf_remaining if the raw buf has any data. While the user will probably call into this function again, I think that's a waste of

Re: Experiments with Postgres and SSL

2024-04-07 Thread Tom Lane
Heikki Linnakangas writes: > Committed this. Thank you to everyone involved! Looks like perlcritic isn't too happy with the test code: koel and crake say ./src/test/libpq_encryption/t/001_negotiate_encryption.pl: Return value of flagged function ignored - chmod at line 138, column 2. See pages

Re: Popcount optimization using AVX512

2024-04-07 Thread Nathan Bossart
On Sun, Apr 07, 2024 at 08:23:32PM -0500, Nathan Bossart wrote: > The Intel documentation for _mm256_undefined_si256() [0] > indicates that it is intended to return "undefined elements," so it seems > like the use of an uninitialized variable might be intentional. See also https://gcc.gnu.org/git/

Re: Coverity complains about simplehash.h's SH_STAT()

2024-04-07 Thread Andres Freund
Hi, On 2024-04-07 21:03:53 -0400, Tom Lane wrote: > Today's Coverity run produced this: > > /srv/coverity/git/pgsql-git/postgresql/src/include/lib/simplehash.h: 1138 in > bh_nodeidx_stat() > 1132 avg_collisions = 0; > 1133 } > 1134 > 1135 sh_log("size: " UINT64_F

  1   2   >