Re: Conflict Detection and Resolution

2024-06-17 Thread Dilip Kumar
On Mon, Jun 17, 2024 at 5:38 AM Tomas Vondra wrote: > > > The issue with using commit timestamps is that, when multiple nodes > > are involved, the commit timestamp won't accurately represent the > > actual order of operations. There's no reliable way to determine the > > perfect order of each op

Re: RFC: adding pytest as a supported test framework

2024-06-17 Thread Matthias van de Meent
Hi Greg, Jelte, On Sat, 15 Jun 2024 at 23:53, Greg Sabino Mullane wrote: > > On Sat, Jun 15, 2024 at 12:48 PM Jelte Fennema-Nio wrote: >> >> Afaict, there's a significant part of our current community who feel the same way (and I'm pretty sure every sub-30 year old person who >> newly joins the

Re: Pgoutput not capturing the generated columns

2024-06-17 Thread Peter Smith
Hi, here are my review comments for patch v7-0001. == 1. GENERAL - \dRs+ Shouldn't the new SUBSCRIPTION parameter be exposed via "describe" (e.g. \dRs+ mysub) the same as the other boolean parameters? == Commit message 2. When 'include_generated_columns' is false then the PUBLICATION co

Re: Using LibPq in TAP tests via FFI

2024-06-17 Thread Alvaro Herrera
On 2024-Jun-16, Andrew Dunstan wrote: > +sub query_oneval > +{ > + my $self = shift; > + my $sql = shift; > + my $missing_ok = shift; # default is not ok > + my $conn = $self->{conn}; > + my $result = PQexec($conn, $sql); > + my $ok = $result && (PQresultStatus($result) ==

Re: SQL/JSON query functions context_item doc entry and type requirement

2024-06-17 Thread jian he
On Mon, Jun 17, 2024 at 2:43 PM Amit Langote wrote: > > Hi, > > On Tue, Jun 4, 2024 at 12:11 AM jian he wrote: > > > > hi > > based on gram.y and function transformJsonValueExpr. > > > > gram.y: > > | JSON_QUERY '(' > > json_value_expr ',' a_expr json_passing_clause_opt > > json_returning_clause_

Re: Conflict Detection and Resolution

2024-06-17 Thread Amit Kapila
On Wed, Jun 12, 2024 at 10:03 AM Dilip Kumar wrote: > > On Tue, Jun 11, 2024 at 7:44 PM Tomas Vondra > wrote: > > > > Yes, that's correct. However, many cases could benefit from the > > > update_deleted conflict type if it can be implemented reliably. That's > > > why we wanted to give it a try.

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Daniel Gustafsson
> On 17 Jun 2024, at 01:46, Andres Freund wrote: > When connecting with a libpq based client, the TLS establishment ends up like > this in many configurations; > > C->S: TLSv1 393 Client Hello > S->C: TLSv1.3 167 Hello Retry Request, Change Cipher Spec > C->S: TLSv1.3 432 Change Cipher Spec, Cli

Re: Vacuum statistics

2024-06-17 Thread Alena Rybakina
I have written the documentary and attached the patch. On 08.06.2024 09:30, Alena Rybakina wrote: Iam currentlyworkingondividingthispatchintothreepartstosimplifythe reviewprocess:oneofthemwillcontaincodeforcollectingvacuumstatisticsontables,the secondonindexesandthe lastondatabases.I alsowrit

Re: POC, WIP: OR-clause support for indexes

2024-06-17 Thread Alena Rybakina
Hi, thank you for your work with this subject! On 14.06.2024 15:00, Alexander Korotkov wrote: On Mon, Apr 8, 2024 at 1:34 AM Alexander Korotkov wrote: I've revised the patch. Did some beautification, improvements for documentation, commit messages etc. I've pushed the 0001 patch without 0002

Re: Avoid orphaned objects dependencies, take 3

2024-06-17 Thread Bertrand Drouvot
Hi, On Thu, Jun 13, 2024 at 04:52:09PM +, Bertrand Drouvot wrote: > Hi, > > On Thu, Jun 13, 2024 at 10:49:34AM -0400, Robert Haas wrote: > > On Fri, Jun 7, 2024 at 4:41 AM Bertrand Drouvot > > wrote: > > > Do you still find the code hard to maintain with v9? > > > > I don't think it substan

Re: speed up a logical replica setup

2024-06-17 Thread Peter Eisentraut
On 07.06.24 05:49, Euler Taveira wrote: Here it is a patch series to fix the issues reported in recent discussions. The patches 0001 and 0003 aim to fix the buildfarm issues. The patch 0002 removes synchronized failover slots on subscriber since it has no use. I also included an optional patch

Re: speed up a logical replica setup

2024-06-17 Thread Peter Eisentraut
On 07.06.24 11:17, Hayato Kuroda (Fujitsu) wrote: Other minor comments are included by the attached diff file. It contains changes to follow conventions and pgindent/pgperltidy. I have included some of your changes in the patches from Euler that I committed today. The 0004 patch might get rer

Re: POC, WIP: OR-clause support for indexes

2024-06-17 Thread Alexander Korotkov
On Mon, Jun 17, 2024 at 1:33 PM Alena Rybakina wrote: > I noticed that 7 libraries have been added to > src/backend/optimizer/plan/initsplan.c, and as far as I remember, Tom Lane > has already expressed doubts about the approach that requires adding a large > number of libraries [0], but I'm af

Re: POC, WIP: OR-clause support for indexes

2024-06-17 Thread Alexander Korotkov
On Mon, Jun 17, 2024 at 7:02 AM Andrei Lepikhov wrote: > On 6/14/24 19:00, Alexander Korotkov wrote: > > This patch could use some polishing, but I'd like to first hear some > > feedback on general design. > Thanks for your time and efforts. I have skimmed through the code—there > is a minor fix i

Re: RFC: adding pytest as a supported test framework

2024-06-17 Thread Aleksander Alekseev
Hi Jacob, > For the v18 cycle, I would like to try to get pytest [1] in as a > supported test driver, in addition to the current offerings. > > (I'm tempted to end the email there.) Huge +1 from me and many thanks for working on this. Two cents from me. I spent a fair part of my career writing

Is creating logical replication slots in template databases useful at all?

2024-06-17 Thread Bharath Rupireddy
Hi, While looking at the commit https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=29d0a77fa6606f9c01ba17311fc452dabd3f793d, I noticed that get_old_cluster_logical_slot_infos gets called for even template1 and template0 databases. Which means, pg_upgrade executes queries against th

Re: Using LibPq in TAP tests via FFI

2024-06-17 Thread Andrew Dunstan
On 2024-06-17 Mo 5:07 AM, Alvaro Herrera wrote: On 2024-Jun-16, Andrew Dunstan wrote: +sub query_oneval +{ + my $self = shift; + my $sql = shift; + my $missing_ok = shift; # default is not ok + my $conn = $self->{conn}; + my $result = PQexec($conn, $sql); +

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-06-17 Thread Bharath Rupireddy
Hi, On Sat, Apr 13, 2024 at 9:36 AM Bharath Rupireddy wrote: > > There was a point raised by Amit > https://www.postgresql.org/message-id/CAA4eK1K8wqLsMw6j0hE_SFoWAeo3Kw8UNnMfhsWaYDF1GWYQ%2Bg%40mail.gmail.com > on when to do the XID age based invalidation - whether in checkpointer > or when vacuu

Re: ON ERROR in json_query and the like

2024-06-17 Thread Chapman Flack
Hi, On 06/17/24 02:20, Amit Langote wrote: >>>Apparently, the functions expect JSONB so that a cast is implied >>>when providing TEXT. However, the errors during that cast are >>>not subject to the ON ERROR clause. >>> >>>17beta1=# SELECT JSON_QUERY('invalid', '$' NULL ON ERROR); >

Re: SQL/JSON query functions context_item doc entry and type requirement

2024-06-17 Thread Chapman Flack
Hi, On 06/17/24 02:43, Amit Langote wrote: > context_item expression can be a value of > any type that can be cast to jsonb. This includes types > such as char, text, bpchar, > character varying, and bytea (with > ENCODING UTF8), as well as any domains over these types. Reading this message in c

Re: ON ERROR in json_query and the like

2024-06-17 Thread Markus Winand
> On 17.06.2024, at 08:20, Amit Langote wrote: > > Hi, > > (apologies for not replying to this thread sooner) > > On Tue, May 28, 2024 at 6:57 PM Pavel Stehule wrote: >> út 28. 5. 2024 v 11:29 odesílatel Markus Winand >> napsal: >>> >>> Hi! >>> >>> I’ve noticed two “surprising” (to me) b

Re: DROP OWNED BY fails to clean out pg_init_privs grants

2024-06-17 Thread Daniel Gustafsson
> On 15 Jun 2024, at 01:46, Tom Lane wrote: > > Robert Haas writes: >> I think the only thing we absolutely have to fix here is the dangling >> ACL references. > > Here's a draft patch that takes care of Hannu's example, and I think > it fixes other potential dangling-reference scenarios too. >

Re: RFC: adding pytest as a supported test framework

2024-06-17 Thread Jelte Fennema-Nio
On Sun, 16 Jun 2024 at 23:04, Robert Haas wrote: > > On Sat, Jun 15, 2024 at 6:00 PM Melanie Plageman > wrote: > > Writing a new test framework in a popular language that makes it more > > likely that more people will write more tests and test infrastructure > > is such a completely different thi

Re: Using LibPq in TAP tests via FFI

2024-06-17 Thread Alvaro Herrera
On 2024-Jun-17, Andrew Dunstan wrote: > On 2024-06-17 Mo 5:07 AM, Alvaro Herrera wrote: > > You made this function join the tuples from multiple queries together, > > but the output format doesn't show anything for queries that return > > empty. I think this strategy doesn't cater for the case o

Re: ON ERROR in json_query and the like

2024-06-17 Thread Pavel Stehule
po 17. 6. 2024 v 15:07 odesílatel Markus Winand napsal: > > > On 17.06.2024, at 08:20, Amit Langote wrote: > > > > Hi, > > > > (apologies for not replying to this thread sooner) > > > > On Tue, May 28, 2024 at 6:57 PM Pavel Stehule > wrote: > >> út 28. 5. 2024 v 11:29 odesílatel Markus Winand <

Re: Using LibPq in TAP tests via FFI

2024-06-17 Thread Andrew Dunstan
On 2024-06-16 Su 6:38 PM, Andres Freund wrote: Hi, On 2024-06-16 17:43:05 -0400, Andrew Dunstan wrote: On Fri, Jun 14, 2024 at 12:25 PM Andres Freund wrote: I guess it's a question of how widely available FFI::Platypus is. I know it's available pretty much out of the box on Strawberry Perl an

Re: Direct SSL connection and ALPN loose ends

2024-06-17 Thread Jacob Champion
On Mon, Apr 29, 2024 at 8:24 AM Heikki Linnakangas wrote: > I was mostly worried about the refactoring of the > retry logic in libpq (and about the pre-existing logic too to be honest, > it was complicated before these changes already). Some changes in the v17 negotiation fallback order caught my

Inconsistency between try_mergejoin_path and create_mergejoin_plan

2024-06-17 Thread Alexander Pyhalov
Hi. There's the following inconsistency between try_mergejoin_path() and create_mergejoin_plan(). When clause operator has no commutator, we can end up with mergejoin path. Later create_mergejoin_plan() will call get_switched_clauses(). This function can error out with ERROR: could not find

Re: Show WAL write and fsync stats in pg_stat_io

2024-06-17 Thread Melanie Plageman
On Thu, Jun 13, 2024 at 5:24 AM Nazir Bilal Yavuz wrote: > > On Sun, 9 Jun 2024 at 18:05, Nitin Jadhav > wrote: > > > > > If possible, let's have all the I/O stats (even for WAL) in > > > pg_stat_io. Can't we show the WAL data we get from buffers in the hits > > > column and then have read_bytes

Re: DROP OWNED BY fails to clean out pg_init_privs grants

2024-06-17 Thread Tom Lane
Daniel Gustafsson writes: > On 15 Jun 2024, at 01:46, Tom Lane wrote: >> The core change is to install SHARED_DEPENDENCY_INITACL entries in >> pg_shdepend for all references to non-pinned roles in pg_init_privs, >> whether they are the object's owner or not. Doing that ensures that >> we can't d

Re: Separate HEAP WAL replay logic into its own file

2024-06-17 Thread Melanie Plageman
On Mon, Jun 17, 2024 at 2:20 AM Li, Yong wrote: > > Hi PostgreSQL hackers, > > For most access methods in PostgreSQL, the implementation of the access > method itself and the implementation of its WAL replay logic are organized in > separate source files. However, the HEAP access method is an e

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-06-17 Thread Nathan Bossart
On Mon, Jun 17, 2024 at 05:55:04PM +0530, Bharath Rupireddy wrote: > Here are my thoughts on when to do the XID age invalidation. In all > the patches sent so far, the XID age invalidation happens in two > places - one during the slot acquisition, and another during the > checkpoint. As the suggest

Re: Allow logical failover slots to wait on synchronous replication

2024-06-17 Thread Bertrand Drouvot
Hi, On Tue, Jun 11, 2024 at 10:00:46AM +, Bertrand Drouvot wrote: > Hi, > > On Mon, Jun 10, 2024 at 09:25:10PM -0500, Nathan Bossart wrote: > > On Mon, Jun 10, 2024 at 03:51:05PM -0700, John H wrote: > > > The existing 'standby_slot_names' isn't great for users who are running > > > clusters

Re: POC, WIP: OR-clause support for indexes

2024-06-17 Thread Alena Rybakina
On 17.06.2024 15:11, Alexander Korotkov wrote: On Mon, Jun 17, 2024 at 1:33 PM Alena Rybakina wrote: I noticed that 7 libraries have been added to src/backend/optimizer/plan/initsplan.c, and as far as I remember, Tom Lane has already expressed doubts about the approach that requires adding a

Re: Conflict Detection and Resolution

2024-06-17 Thread Robert Haas
On Mon, Jun 17, 2024 at 1:42 AM Amit Kapila wrote: > The difference w.r.t the existing mechanisms for holding deleted data > is that we don't know whether we need to hold off the vacuum from > cleaning up the rows because we can't say with any certainty whether > other nodes will perform any confl

Re: Direct SSL connection and ALPN loose ends

2024-06-17 Thread Heikki Linnakangas
On 17/06/2024 17:11, Jacob Champion wrote: On Mon, Apr 29, 2024 at 8:24 AM Heikki Linnakangas wrote: I was mostly worried about the refactoring of the retry logic in libpq (and about the pre-existing logic too to be honest, it was complicated before these changes already). Some changes in the

FYI: LLVM Runtime Crash

2024-06-17 Thread David E. Wheeler
Hackers, FYI, I wanted to try using PostgreSQL with LLVM on my Mac, but the backend repeatedly crashes during `make check`. I found the same issue in master and REL_16_STABLE. The crash message is: FATAL: fatal llvm error: Unsupported stack probing method server closed the connection unexpecte

[PATCH] Improve error message when trying to lock virtual tuple.

2024-06-17 Thread Sven Klemm
Hello, When currently trying to lock a virtual tuple the returned error will be a misleading `could not read block 0`. This patch adds a check for the tuple table slot being virtual to produce a clearer error. This can be triggered by extensions returning virtual tuples. While this is of course a

Re: [PATCH] Improve error message when trying to lock virtual tuple.

2024-06-17 Thread Aleksander Alekseev
Hi, > When currently trying to lock a virtual tuple the returned error > will be a misleading `could not read block 0`. This patch adds a > check for the tuple table slot being virtual to produce a clearer > error. > > This can be triggered by extensions returning virtual tuples. > While this is o

Re: RFC: adding pytest as a supported test framework

2024-06-17 Thread Andrew Dunstan
On 2024-06-17 Mo 4:27 AM, Matthias van de Meent wrote: Hi Greg, Jelte, On Sat, 15 Jun 2024 at 23:53, Greg Sabino Mullane wrote: > Those young-uns are also the same group who hold their nose when coding in C, and are always clamoring for rewriting Postgres in Rust. Could you point me to

Re: Direct SSL connection and ALPN loose ends

2024-06-17 Thread Jacob Champion
On Mon, Jun 17, 2024 at 8:24 AM Heikki Linnakangas wrote: > By "negotiation", which part of the protocol are we talking about > exactly? In the middle of the TLS handshake? After sending the startup > packet? By "negotiation" I mean the server's response to the startup packet. I.e. "supported"/"n

Re: Avoid orphaned objects dependencies, take 3

2024-06-17 Thread Robert Haas
On Fri, Jun 14, 2024 at 3:54 AM Bertrand Drouvot wrote: > > Ah, right. So, I was assuming that, with either this version of your > > patch or the earlier version, we'd end up locking the constraint > > itself. Was I wrong about that? > > The child contraint itself is not locked when going through

Re: DROP OWNED BY fails to clean out pg_init_privs grants

2024-06-17 Thread Daniel Gustafsson
> On 17 Jun 2024, at 16:56, Tom Lane wrote: > Daniel Gustafsson writes: >> I wonder if this will break any tools/scripts in prod which relies on the >> previous (faulty) behaviour. It will be interesting to see if anything shows >> up on -bugs. Off the cuff it seems like a good idea judging by

Re: Issues with ON CONFLICT UPDATE and REINDEX CONCURRENTLY

2024-06-17 Thread Michail Nikolaev
Hello, everyone. > I think It is even possible to see !alive index in the same situation (it is worse case), but I was unable to reproduce it so far. Fortunately, it is not possible. So, seems like I have found the source of the problem: 1) infer_arbiter_indexes calls RelationGetIndexList to get

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 12:00:30 +0200, Daniel Gustafsson wrote: > > On 17 Jun 2024, at 01:46, Andres Freund wrote: > > > When connecting with a libpq based client, the TLS establishment ends up > > like > > this in many configurations; > > > > C->S: TLSv1 393 Client Hello > > S->C: TLSv1.3 167 Hello

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Jacob Champion
On Mon, Jun 17, 2024 at 10:01 AM Andres Freund wrote: > On 2024-06-17 12:00:30 +0200, Daniel Gustafsson wrote: > > To set the specified curve in ssl_ecdh_curve we have to don't we? > > Sure, but it's not obvious to me why we actually want to override openssl's > defaults here. There's not even a p

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Daniel Gustafsson
> On 17 Jun 2024, at 19:01, Andres Freund wrote: > On 2024-06-17 12:00:30 +0200, Daniel Gustafsson wrote: >>> On 17 Jun 2024, at 01:46, Andres Freund wrote: >>> I don't know if it's good that we're calling SSL_CTX_set_tmp_ecdh at all, >> >> To set the specified curve in ssl_ecdh_curve we have t

tls 1.3: sending multiple tickets

2024-06-17 Thread Andres Freund
Hi, To investigate an unrelated issue, I set up key logging in the backend (we probably should build that in) and looked at the decrypted data. And noticed that just after TLS setup finished the server sends three packets in a row: C->S: TLSv1.3: finished C->S: TLSv1.3: application data (startup

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 19:29:47 +0200, Daniel Gustafsson wrote: > >> I wonder if that made OpenSSL override the min protocol version and switch > >> to a TLS1.3 ClientHello since it otherwise couldn't announce the curve. > > > > The client seems to announce the curve in the initial ClientHello even wit

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Daniel Gustafsson
> On 17 Jun 2024, at 19:44, Andres Freund wrote: >> Let's bring that to Erica's patch for allowing a list of curves. > > I'm kinda wondering if we ought to do something about this in the > backbranches. Forcing unnecessary roundtrips onto everyone for the next five > years due to an oversight on

Re: ecdh support causes unnecessary roundtrips

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 19:51:45 +0200, Daniel Gustafsson wrote: > > On 17 Jun 2024, at 19:44, Andres Freund wrote: > > >> Let's bring that to Erica's patch for allowing a list of curves. > > > > I'm kinda wondering if we ought to do something about this in the > > backbranches. Forcing unnecessary r

Re: IPC::Run accepts bug reports

2024-06-17 Thread Robert Haas
On Sat, Jun 15, 2024 at 7:48 PM Noah Misch wrote: > Separating this from the pytest thread: > > On Sat, Jun 15, 2024 at 01:26:57PM -0400, Robert Haas wrote: > > The one > > thing I know about that *I* think is a pretty big problem about Perl > > is that IPC::Run is not really maintained. > > I don

Re: Avoid orphaned objects dependencies, take 3

2024-06-17 Thread Bertrand Drouvot
Hi, On Mon, Jun 17, 2024 at 12:24:46PM -0400, Robert Haas wrote: > On Fri, Jun 14, 2024 at 3:54 AM Bertrand Drouvot > wrote: > > > Ah, right. So, I was assuming that, with either this version of your > > > patch or the earlier version, we'd end up locking the constraint > > > itself. Was I wrong

Re: improve predefined roles documentation

2024-06-17 Thread Robert Haas
On Thu, Jun 13, 2024 at 3:48 PM Nathan Bossart wrote: > I think we could improve matters by abandoning the table and instead > documenting these roles more like we document GUCs, i.e., each one has a > section below it where we can document it in as much detail as we want. > Some of these roles sh

Re: IPC::Run accepts bug reports

2024-06-17 Thread Andres Freund
Hi, On 2024-06-15 16:48:24 -0700, Noah Misch wrote: > Separating this from the pytest thread: > > On Sat, Jun 15, 2024 at 01:26:57PM -0400, Robert Haas wrote: > > The one > > thing I know about that *I* think is a pretty big problem about Perl > > is that IPC::Run is not really maintained. > > I d

Re: Document NULL

2024-06-17 Thread David G. Johnston
On Sat, May 11, 2024 at 11:00 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > Though I haven’t settled on a phrasing I really like. But I’m trying to > avoid a parenthetical. > > Settled on this: The cardinal rule, a null value is neither equal nor unequal to any value, includi

Re: Direct SSL connection and ALPN loose ends

2024-06-17 Thread Andres Freund
Hi, On 2024-04-29 18:24:04 +0300, Heikki Linnakangas wrote: > All reported bugs have now been fixed. We now enforce ALPN in all the right > places. Please let me know if I missed something. Very minor and not really your responsibility: If provided with the necessary key information, wireshark c

Re: Refactoring backend fork+exec code

2024-06-17 Thread Nathan Bossart
While looking into [0], I noticed that main() still only checks for the --fork prefix, but IIUC commit aafc05d removed all --fork* options except for --forkchild. I've attached a patch to strengthen the check in main(). This is definitely just a nitpick. [0] https://postgr.es/m/CAKAnmmJkZtZAiSry

Re: Add support to TLS 1.3 cipher suites and curves lists

2024-06-17 Thread Andres Freund
Hi, This thread was referenced by https://www.postgresql.org/message-id/48F0A1F8-E0B4-41F8-990F-41E6BA2A6185%40yesql.se On 2024-06-13 14:34:27 +0800, Erica Zhang wrote: > diff --git a/src/backend/libpq/be-secure-openssl.c > b/src/backend/libpq/be-secure-openssl.c > index 39b1a66236..d097e81407

Re: Virtual generated columns

2024-06-17 Thread Tomasz Rybak
On Wed, 2024-05-22 at 19:22 +0200, Peter Eisentraut wrote: > On 29.04.24 10:23, Peter Eisentraut wrote: > > Here is a patch set to implement virtual generated columns. > > > The main feature patch (0005 here) generally works but has a number > > of > > open corner cases that need to be thought ab

cost delay brainstorming

2024-06-17 Thread Robert Haas
Hi, As I mentioned in my talk at 2024.pgconf.dev, I think that the biggest problem with autovacuum as it exists today is that the cost delay is sometimes too low to keep up with the amount of vacuuming that needs to be done. I sketched a solution during the talk, but it was very complicated, so I

Re: FYI: LLVM Runtime Crash

2024-06-17 Thread David E. Wheeler
On Jun 17, 2024, at 11:52, David E. Wheeler wrote: > I don’t *think* it’s something that can be fixed in Postgres core. This is > mostly in FYI in case anyone else runs into this issue or knows something > more about it. Okay, a response to the issue[1] says the bug is in Postgres: > The erro

Re: [PATCH] Improve error message when trying to lock virtual tuple.

2024-06-17 Thread Matthias van de Meent
(now send a copy to -hackers, too) On Mon, 17 Jun 2024 at 17:55, Sven Klemm wrote: > > Hello, > > When currently trying to lock a virtual tuple the returned error > will be a misleading `could not read block 0`. This patch adds a > check for the tuple table slot being virtual to produce a clearer

Xact end leaves CurrentMemoryContext = TopMemoryContext

2024-06-17 Thread Tom Lane
AtCommit_Memory and friends have done $SUBJECT for at least a couple of decades, but in the wake of analyzing bug #18512 [1], I'm feeling like that's a really bad idea. There is too much code running around the system that assumes that it's fine to leak stuff in CurrentMemoryContext. If we execut

Re: FYI: LLVM Runtime Crash

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 16:07:49 -0400, David E. Wheeler wrote: > On Jun 17, 2024, at 11:52, David E. Wheeler wrote: > > > I don’t *think* it’s something that can be fixed in Postgres core. This is > > mostly in FYI in case anyone else runs into this issue or knows something > > more about it. > >

Re: FYI: LLVM Runtime Crash

2024-06-17 Thread David E. Wheeler
On Jun 17, 2024, at 16:37, Andres Freund wrote: > I suspect the issue might be that the version of clang and LLVM are diverging > too far. Does it work if you pass CLANG=/opt/homebrew/opt/llvm/bin/clang to > configure? It does! It didn’t occur to me this would be the issue, but I presumes /usr/

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-17 Thread John H
Hi Ashutosh, Thinking about this more, could you clarify the problem/issue at hand? I think it's still not clear to me. Yes, CREATE EXTENSION can create functions that lead to unexpected privilege escalation, regardless if they are SECURITY DEFINER or SECURITY INVOKER (if the function is inadvert

Re: cost delay brainstorming

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 15:39:27 -0400, Robert Haas wrote: > As I mentioned in my talk at 2024.pgconf.dev, I think that the biggest > problem with autovacuum as it exists today is that the cost delay is > sometimes too low to keep up with the amount of vacuuming that needs > to be done. I agree it's a

Re: Reducing the log spam

2024-06-17 Thread Justin Pryzby
On Thu, May 02, 2024 at 12:47:45PM +0200, Laurenz Albe wrote: > On Mon, 2024-03-11 at 09:33 +0100, Jelte Fennema-Nio wrote: > > -   the subscriber's server log. > > +   the subscriber's server log if you remove 23505 from > > +   . > > > > This seems like a pretty big regression. Being able to kno

Re: Xact end leaves CurrentMemoryContext = TopMemoryContext

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 16:37:05 -0400, Tom Lane wrote: > As to what to do about it: I'm imagining that instead of resetting > CurrentMemoryContext to TopMemoryContext, we set it to some special > context that we expect we can reset every so often, like at the start > of the next transaction. The exist

Re: cost delay brainstorming

2024-06-17 Thread Greg Sabino Mullane
On Mon, Jun 17, 2024 at 3:39 PM Robert Haas wrote: > So, a very simple algorithm would be: If the maximum number of workers > have been running continuously for more than, say, > 10 minutes, assume we're falling behind Hmm, I don't know about the validity of this. I've seen plenty of cases wher

Re: may be a buffer overflow problem

2024-06-17 Thread Daniel Gustafsson
> On 14 Jun 2024, at 17:18, Tom Lane wrote: > > I wrote: >> Seeing that this code is exercised thousands of times a day in the >> regression tests and has had a failure rate of exactly zero (and >> yes, the tests do check the output), there must be some reason >> why it's okay. > > After looking

Re: jsonpath: Missing regex_like && starts with Errors?

2024-06-17 Thread David E. Wheeler
On Jun 16, 2024, at 11:52, David E. Wheeler wrote: > I think that’s how it should be; I prefer that it raises errors by default > but you can silence them: > > david=# select jsonb_path_query(target => '{"x": "hi"}', path => > '$.integer()', silent => false); > ERROR: jsonpath item method .in

Re: Proposal: Document ABI Compatibility

2024-06-17 Thread David E. Wheeler
On Jun 12, 2024, at 11:20, Andres Freund wrote: >> The PostgreSQL core team maintains two application binary interface (ABI) >> guarantees: one for major releases and one for minor releases. > > I.e. for major versions it's "there is none"? Is it? ISTM that there is the intention not to break

Re: Proposal: Document ABI Compatibility

2024-06-17 Thread David E. Wheeler
On Jun 12, 2024, at 11:30, Peter Geoghegan wrote: > I'm a little surprised that we don't seem to have all that many > problems with ABI breakage, though. Although we theoretically have a > huge number of APIs that extension authors might choose to use, that > isn't really true in practical terms.

Re: Proposal: Document ABI Compatibility

2024-06-17 Thread David E. Wheeler
On Jun 12, 2024, at 11:57, Peter Geoghegan wrote: > That having been said, it would be useful if there was a community web > resource for this -- something akin to coverage.postgresql.org, but > with differential ABI breakage reports. You can see an example report > here: > > https://postgr.es/m

RE: Proposal for Updating CRC32C with AVX-512 Algorithm.

2024-06-17 Thread Amonson, Paul D
> This is extremely workload dependent, it's not hard to find workloads with > lots of very small record and very few big ones... What you observed might > have "just" been the warmup behaviour where more full page writes have to > be written. Can you tell me how to avoid capturing this "warm-up"

Re: cost delay brainstorming

2024-06-17 Thread David Rowley
On Tue, 18 Jun 2024 at 07:39, Robert Haas wrote: > I think we might able to get fairly far by observing that if the > number of running autovacuum workers is equal to the maximum allowable > number of running autovacuum workers, that may be a sign of trouble, > and the longer that situation persis

Re: jsonpath: Missing regex_like && starts with Errors?

2024-06-17 Thread Chapman Flack
On 06/17/24 18:14, David E. Wheeler wrote: > So I think that’s the key: There’s not a difference between the behavior of > `like_regex` and `starts with` vs other predicate expressions. The current implementation seems to have made each of our s responsible for swallowing its own errors, which is

Re: jsonpath: Missing regex_like && starts with Errors?

2024-06-17 Thread David E. Wheeler
On Jun 17, 2024, at 6:44 PM, Chapman Flack wrote: > The current implementation seems to have made each of our > s responsible for swallowing its own errors, which > is one perfectly cromulent way to satisfy the SQL standard behavior saying > all errors within a should be swallowed. Naw, execute

Re: Inval reliability, especially for inplace updates

2024-06-17 Thread Noah Misch
On Sat, Jun 15, 2024 at 03:37:18PM -0700, Noah Misch wrote: > On Wed, May 22, 2024 at 05:05:48PM -0700, Noah Misch wrote: > > https://postgr.es/m/20240512232923.aa.nmi...@google.com wrote: > > > Separable, nontrivial things not fixed in the attached patch stack: > > > > > > - Inplace update uses t

Re: Separate HEAP WAL replay logic into its own file

2024-06-17 Thread Li, Yong
> On Jun 17, 2024, at 23:01, Melanie Plageman wrote: > > External Email > > On Mon, Jun 17, 2024 at 2:20 AM Li, Yong wrote: >> >> Hi PostgreSQL hackers, >> >> For most access methods in PostgreSQL, the implementation of the access >> method itself and the implementation of its WAL replay l

Re: Check lateral references within PHVs for memoize cache keys

2024-06-17 Thread Richard Guo
On Mon, Mar 18, 2024 at 4:36 PM Richard Guo wrote: > Here is another rebase over master so it applies again. I also added a > commit message to help review. Nothing else has changed. AFAIU currently we do not add Memoize nodes on top of join relation paths. This is because the ParamPathInfos f

Re: Inval reliability, especially for inplace updates

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 16:58:54 -0700, Noah Misch wrote: > On Sat, Jun 15, 2024 at 03:37:18PM -0700, Noah Misch wrote: > > On Wed, May 22, 2024 at 05:05:48PM -0700, Noah Misch wrote: > > > https://postgr.es/m/20240512232923.aa.nmi...@google.com wrote: > > > > Separable, nontrivial things not fixed in t

Re: Logical Replication of sequences

2024-06-17 Thread Masahiko Sawada
On Fri, Jun 14, 2024 at 4:04 PM Amit Kapila wrote: > > On Thu, Jun 13, 2024 at 6:14 PM Masahiko Sawada wrote: > > > > On Thu, Jun 13, 2024 at 7:06 PM Amit Kapila wrote: > > > > > > On Thu, Jun 13, 2024 at 1:09 PM Masahiko Sawada > > > wrote: > > > > > > > > On Wed, Jun 12, 2024 at 6:59 PM Amit

RE: Conflict Detection and Resolution

2024-06-17 Thread Zhijie Hou (Fujitsu)
On Thursday, June 13, 2024 2:11 PM Masahiko Sawada wrote: Hi, > On Wed, Jun 5, 2024 at 3:32 PM Zhijie Hou (Fujitsu) > wrote: > > > > This time at PGconf.dev[1], we had some discussions regarding this > > project. The proposed approach is to split the work into two main > > components. The firs

Re: jsonpath: Missing regex_like && starts with Errors?

2024-06-17 Thread Chapman Flack
On 06/17/24 19:17, David E. Wheeler wrote: > [1]: > https://github.com/postgres/postgres/blob/82ed67a/src/backend/utils/adt/jsonpath_exec.c#L2058-L2059 Huh, I just saw something peculiar, skimming through the code: https://github.com/postgres/postgres/blob/82ed67a/src/backend/utils/adt/jsonpath_

Re: assertion failure at cost_memoize_rescan()

2024-06-17 Thread Kohei KaiGai
2024年6月17日(月) 8:27 David Rowley : > > On Mon, 17 Jun 2024 at 10:23, Tomas Vondra > wrote: > > Interesting. Seems like a bug due to the two places clamping the values > > inconsistently. It probably does not matter in other contexts because we > > don't subtract the values like this, but here it tr

Missing docs for new enable_group_by_reordering GUC

2024-06-17 Thread Bruce Momjian
This commit added enable_group_by_reordering: commit 0452b461bc4 Author: Alexander Korotkov Date: Sun Jan 21 22:21:36 2024 +0200 Explore alternative orderings of group-by pathkeys during optimization. When evaluating a query wit

Re: may be a buffer overflow problem

2024-06-17 Thread Andres Freund
Hi, On 2024-06-17 23:52:54 +0200, Daniel Gustafsson wrote: > Since sqlca is, according to our docs, present in other database systems we > should probably keep it a 5-char array for portability reasons. Adding a > padding character should be fine though. How about, additionally, adding __attribu

Re: may be a buffer overflow problem

2024-06-17 Thread Tom Lane
Andres Freund writes: > On 2024-06-17 23:52:54 +0200, Daniel Gustafsson wrote: >> Since sqlca is, according to our docs, present in other database systems we >> should probably keep it a 5-char array for portability reasons. Adding a >> padding character should be fine though. > How about, addit

Re: Better error message when --single is not the first arg to postgres executable

2024-06-17 Thread Nathan Bossart
On Wed, Jun 05, 2024 at 11:38:48PM -0400, Greg Sabino Mullane wrote: > On Wed, Jun 5, 2024 at 3:18 PM Nathan Bossart > wrote: >> Could we remove the requirement that --single must be first? I'm not >> thrilled about adding a list of "must be first" options that needs to stay >> updated, but given

Re: assertion failure at cost_memoize_rescan()

2024-06-17 Thread David Rowley
On Tue, 18 Jun 2024 at 14:23, Kohei KaiGai wrote: > > 2024年6月17日(月) 8:27 David Rowley : > > It would be good to know what type of Path outer_path is. Normally > > we'll clamp_row_est() on that field. I suspect we must have some Path > > type that isn't doing that. > > > > KaiGai-san, what type o

Re: New GUC autovacuum_max_threshold ?

2024-06-17 Thread Nathan Bossart
I didn't see a commitfest entry for this, so I created one to make sure we don't lose track of this: https://commitfest.postgresql.org/48/5046/ -- nathan

Re: assertion failure at cost_memoize_rescan()

2024-06-17 Thread Richard Guo
On Tue, Jun 18, 2024 at 10:53 AM David Rowley wrote: > Out of the places I saw, it seems we do tend to code things so that we > don't assume the value has been clamped. E.g. > adjust_limit_rows_costs() does if (*rows < 1) *rows = 1; Agreed. In costsize.c I saw a few instances where we have

Re: State of pg_createsubscriber

2024-06-17 Thread Amit Kapila
On Mon, May 20, 2024 at 12:12 PM Amit Kapila wrote: > > On Sun, May 19, 2024 at 11:20 PM Euler Taveira wrote: > > > > On Sun, May 19, 2024, at 2:30 PM, Tom Lane wrote: > > > > I'm fairly disturbed about the readiness of pg_createsubscriber. > > The 040_pg_createsubscriber.pl TAP test is moderatel

Re: Addressing SECURITY DEFINER Function Vulnerabilities in PostgreSQL Extensions

2024-06-17 Thread Ashutosh Sharma
Hi John, On Tue, Jun 18, 2024 at 2:35 AM John H wrote: > > Hi Ashutosh, > > Thinking about this more, could you clarify the problem/issue at hand? > I think it's still not clear to me. > Yes, CREATE EXTENSION can create functions that lead to unexpected > privilege escalation, regardless > if th

Re: Xact end leaves CurrentMemoryContext = TopMemoryContext

2024-06-17 Thread David Rowley
On Tue, 18 Jun 2024 at 08:37, Tom Lane wrote: > As to what to do about it: I'm imagining that instead of resetting > CurrentMemoryContext to TopMemoryContext, we set it to some special > context that we expect we can reset every so often, like at the start > of the next transaction. The existing

Re: Conflict Detection and Resolution

2024-06-17 Thread Dilip Kumar
On Mon, Jun 17, 2024 at 3:23 PM Amit Kapila wrote: > > On Wed, Jun 12, 2024 at 10:03 AM Dilip Kumar wrote: > > > > On Tue, Jun 11, 2024 at 7:44 PM Tomas Vondra > > wrote: > > > > > > Yes, that's correct. However, many cases could benefit from the > > > > update_deleted conflict type if it can be

Re: Xact end leaves CurrentMemoryContext = TopMemoryContext

2024-06-17 Thread Tom Lane
David Rowley writes: > Instead, could we just not delete TopTransactionContext in > AtCommit_Memory() and instead do MemoryContextReset() on it? Likewise > in AtCleanup_Memory(). Hmm, that's a nice idea. Maybe reset again in AtStart_Memory, although that seems optional. My first reaction was "w

  1   2   >