Add pg_accept_connections_start_time() for better uptime calculation

2025-02-15 Thread Robins Tharakan
Hi, This patch introduces a new function pg_accept_connections_start_time(). Currently, pg_postmaster_start_time() is used to determine when the database started. However, this is not accurate since the postmaster process can sometimes be up whereas the database is not accepting connections (for

Re: generic plans and "initial" pruning

2025-02-15 Thread Junwang Zhao
Hi Amit, On Sat, Feb 15, 2025 at 3:51 PM Amit Langote wrote: > > Hi Alexander, > > On Sat, Feb 15, 2025 at 6:00 AM Alexander Lakhin wrote: > > > > Hello Amit, > > > > 06.02.2025 04:35, Amit Langote wrote: > > > > I plan to push 0001 tomorrow, barring any objections. > > > > > > Please try the fo

Re: Parallel CREATE INDEX for GIN indexes

2025-02-15 Thread Tomas Vondra
On 2/12/25 15:59, Matthias van de Meent wrote: > On Tue, 7 Jan 2025 at 12:59, Tomas Vondra wrote: >> >> ... >> >> I haven't done anything about this, but I'm not sure adding the number >> of GIN tuples to pg_stat_progress_create_index would be very useful. We >> don't know the total number of entr

Re: Pg_stat_activity

2025-02-15 Thread David G. Johnston
On Saturday, February 15, 2025, Rajesh Kumar wrote: > If I see a sudden connection spike (say once in 2 months). What are the > steps I need to follow as a dba? > > Check idle and kill? Check of stat activity for active queries? How do I > ensure how much time is long running query? Blocking quer

Pg_stat_activity

2025-02-15 Thread Rajesh Kumar
If I see a sudden connection spike (say once in 2 months). What are the steps I need to follow as a dba? Check idle and kill? Check of stat activity for active queries? How do I ensure how much time is long running query? Blocking queries? How much of blocking time is considered to be dangerous? L

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Jeff Davis
On Sat, 2025-02-15 at 16:21 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2025-02-15 15:53:20 -0500, Tom Lane wrote: > > > I can take care of it, if you like. > > > That'd be appreciated! > > Done. Thank you all. Regards, Jeff Davis

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Tom Lane
Andres Freund writes: > On 2025-02-15 15:53:20 -0500, Tom Lane wrote: >> I can take care of it, if you like. > That'd be appreciated! Done. regards, tom lane

Re: BackgroundPsql swallowing errors on windows

2025-02-15 Thread Andres Freund
Hi, On 2025-02-14 09:52:24 -0800, Jacob Champion wrote: > On Fri, Feb 14, 2025 at 8:53 AM Andres Freund wrote: > > commit 70291a3c66e > > Author: Michael Paquier > > Date: 2024-11-07 12:11:27 +0900 > > > > Improve handling of empty query results in BackgroundPsql::query() > > > > commit ba

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Andres Freund
On 2025-02-15 15:53:20 -0500, Tom Lane wrote: > Andres Freund writes: > > Are you planning to push / backpatch, or should I? > > I can take care of it, if you like. That'd be appreciated!

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Andres Freund
Hi, On 2025-02-15 15:52:01 -0500, Tom Lane wrote: > The v5 patch seems Good Enough(TM) to me. Agreed. > We can refine it later perhaps; I don't think something like the above would > affect anything that external code should care about. I don't really think it's worth spending cycles on this a

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Tom Lane
Andres Freund writes: > Are you planning to push / backpatch, or should I? I can take care of it, if you like. regards, tom lane

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Jeff Davis
On Sat, 2025-02-15 at 15:43 -0500, Andres Freund wrote: > It seems completely infeasible to me to to implement the "single > error" > approach in a minor version. +1, keep with the behavior in the proposed patches. Even in the next major version, I'd be inclined to try to move toward interfaces t

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Tom Lane
Andres Freund writes: > On 2025-02-15 12:35:45 -0800, Jeff Davis wrote: >> I am not suggesting a change, but there's a minor point about the >> behavior of the replacement that I'd like to highlight: >> Unicode discusses a choice[1]: "An ill-formed subsequence consisting of >> more than one code u

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Andres Freund
Hi, On 2025-02-15 12:35:45 -0800, Jeff Davis wrote: > I am not suggesting a change, but there's a minor point about the > behavior of the replacement that I'd like to highlight: > > Unicode discusses a choice[1]: "An ill-formed subsequence consisting of > more than one code unit could be treated

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Jeff Davis
On Fri, 2025-02-14 at 17:27 -0800, Noah Misch wrote: > I'm attaching a WIP patch from Andres Freund. I am not suggesting a change, but there's a minor point about the behavior of the replacement that I'd like to highlight: Unicode discusses a choice[1]: "An ill-formed subsequence consisting of mo

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Jeff Davis
Expanding on the reasoning a bit: This discussion is only relevant only when the application meets the following conditions: A. Sends invalidly-encoded input to an escaping routine. Many languages protect against this, such as python and rust. But other languages, like C, Go, and Ruby do not.

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Andres Freund
Hi, On 2025-02-15 14:12:06 -0500, Tom Lane wrote: > On closer inspection, PQescapeInternal already issues only one > error message, since it does "return NULL" after detecting the > first error. So this makes PQescapeStringInternal behave more > like that. This looks good to me. I looked throug

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Tom Lane
Noah Misch writes: > On 2025-02-15 13:08:03 -0500, Tom Lane wrote: >> The other thing that was discussed in the security thread was >> modifying PQescapeStringInternal and PQescapeInternal to produce >> no more than one complaint about invalid multibyte characters, >> on the grounds that input tha

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Tom Lane
Andres Freund writes: > It seems that nobody is arguing against the "just skip one byte" behaviour, so > I'm inclined to push this fairly soon, even if Noah's "24 hours" haven't quite > elapsed. A few more cycles in the buildfarm wouldn't hurt. Agreed. I thought there would be more discussion,

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Noah Misch
On Sat, Feb 15, 2025 at 01:23:51PM -0500, Andres Freund wrote: > On 2025-02-15 13:08:03 -0500, Tom Lane wrote: > > I studied the v3 patch a little and realized that it only fixes the > > behavior for the case of a complete-but-invalid multibyte character. > > If we have an incomplete character at t

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Andres Freund
Hi, On 2025-02-15 13:08:03 -0500, Tom Lane wrote: > I studied the v3 patch a little and realized that it only fixes the > behavior for the case of a complete-but-invalid multibyte character. > If we have an incomplete character at the end of the string, the > whole thing still gets dropped. Surel

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Tom Lane
I studied the v3 patch a little and realized that it only fixes the behavior for the case of a complete-but-invalid multibyte character. If we have an incomplete character at the end of the string, the whole thing still gets dropped. Surely that's not what we want if we are going to adopt this beh

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-15 Thread Christoph Berg
Re: Andres Freund > I think that'd be *really* helpful. Of course that does require somebody > watching and raising an alarm... > > Do you have ongoing package builds for sid or such? What I am doing anyway is to trigger the regression test of each package once a month (randomly distributed over

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-15 Thread Andres Freund
Hi, On 2025-02-15 17:55:12 +0100, Christoph Berg wrote: > Re: Andres Freund > > I don't think that common uses of PQescapeIdentifier/Literal are likely to > > catch the problem, so it's perhaps not too surprising it wasn't caught. > > Which, > > I guess, shows that we really need more explicit ed

Re: New buildfarm animals with FIPS mode enabled

2025-02-15 Thread Tom Lane
Mark Wong writes: > That's my fault. I did a sloppy job copying configs etc from the s390x > fips animals and forgot about the OS versions, branches, etc. Peter > Eisentraut reminded me I think I cleaned that all up. Cool, thanks. regards, tom lane

Re: New buildfarm animals with FIPS mode enabled

2025-02-15 Thread Mark Wong
Hi Tom, On 2/14/25 10:01 AM, Tom Lane wrote: I see that somebody decided to crank up some animals running RHEL8 and RHEL9 with FIPS mode turned on. The RHEL9 animals pass on v17 and master, but not older branches; the RHEL8 animals pass nowhere. This is unsurprising given that the v17-era comm

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-15 Thread Christoph Berg
Re: Andres Freund > > What's missing in the PG regression tests to see that problem? > > Well, the expanded tests added as part of the fix would catch it, but I agree, > it's a problem this wasn't caught beforehand. Oh sorry, I was actually skimming the git log to see if there is a test, but then

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-15 Thread Andres Freund
Hi, On 2025-02-15 13:33:54 +0100, Christoph Berg wrote: > Re: Andres Freund > > > > The fprintf suggests that since 5dc1e42b4 PQescapeIdentifier ignores > > > > its len. > > > > > > Ugh, yes. Need something like the attached. > > > > I just pushed this fix, together with an expansion of test_e

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Andrew Dunstan
On 2025-02-14 Fr 8:27 PM, Noah Misch wrote: The security team has a question below about how best to proceed with a recent behavior change. Commit 5dc1e42b4fa6a4434afa7d7cdcf0291351a7b873 for this week's CVE-2025-1094 changed how PQescapeString()[1] reacts to input that is not valid in the cli

Re: Virtual generated columns

2025-02-15 Thread Dean Rasheed
On Fri, 14 Feb 2025 at 10:59, Peter Eisentraut wrote: > > On 13.02.25 14:06, jian he wrote: > > I didn't solve the out join semantic issue. > > i am wondering, can we do the virtual generated column expansion in > > the rewrite stage as is, > > and wrap the expressions in PHVs if the virtual gener

Re: pg17.3 PQescapeIdentifier() ignores len

2025-02-15 Thread Christoph Berg
Re: Andres Freund > > > The fprintf suggests that since 5dc1e42b4 PQescapeIdentifier ignores its > > > len. > > > > Ugh, yes. Need something like the attached. > > I just pushed this fix, together with an expansion of test_escape.c. With the > expanded test both uses of strlen() are detected.

Re: psql: Add tab completion for ALTER USER RESET

2025-02-15 Thread Robins Tharakan
Hi Tomas, Thanks for taking a look - apologies for the delay here. On Tue, 10 Dec 2024 at 09:09, Tomas Vondra wrote: > > 1) Does it make sense to still show "ALL" when the query returns > nothing? Not sure if we already have a way to handle this. > +1 - "ALL" is pointless when there is nothing

Re: Expanding HOT updates for expression and partial indexes

2025-02-15 Thread Matthias van de Meent
On Thu, 13 Feb 2025 at 19:46, Burd, Greg wrote: > > Attached find an updated patchset v5 that is an evolution of v4. > > Changes v4 to v5 are: > * replaced GUC with table reloption called "expression_checks" (open to other > name ideas) > * minimal documentation updates to README.HOT to address c

Re: WAL-logging facility for pgstats kinds

2025-02-15 Thread Cédric Villemain
On 12/02/2025 01:50, Michael Paquier wrote: On Mon, Feb 10, 2025 at 11:43:30AM -0500, Andres Freund wrote: Because I saw this being moved to the new CF: I continue to *strenuously* object to this design. As outlined upthread, I think it's going into the completely wrong direction. Right. F

Re: Proposal - Allow extensions to set a Plan Identifier

2025-02-15 Thread Andrei Lepikhov
On 14/2/2025 08:21, Michael Paquier wrote: On Thu, Feb 13, 2025 at 11:10:27AM -0600, Sami Imseih wrote: I don't think direct setting of values is a good idea. We will need an API similar to pgstat_report_query_id which ensures we are only reporting top level planIds -and- in the case of multiple

Re: Get rid of WALBufMappingLock

2025-02-15 Thread Alexander Korotkov
Hi! On Fri, Feb 14, 2025 at 4:11 PM Yura Sokolov wrote: > 14.02.2025 17:09, Yura Sokolov пишет: > > 14.02.2025 13:24, Alexander Korotkov пишет: > >> On Fri, Feb 14, 2025 at 11:45 AM Pavel Borisov > >> wrote: > >>> On Fri, 14 Feb 2025 at 00:59, Alexander Korotkov > >>> wrote: > On Thu, Fe

Re: explain analyze rows=%.0f

2025-02-15 Thread Andrei Lepikhov
On 13/2/2025 21:42, Robert Haas wrote: On Thu, Feb 13, 2025 at 4:05 AM Ilia Evdokimov wrote: 1. Documentation (v9-0001-Clarify-display-of-rows-as-decimal-fractions-DOC.patch) One thing that bothers me is that the documentation explains how to compute total time, but it does not clarify how t

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Laurenz Albe
On Fri, 2025-02-14 at 17:27 -0800, Noah Misch wrote: > Commit 5dc1e42b4fa6a4434afa7d7cdcf0291351a7b873 for this week's CVE-2025-1094 > changed how PQescapeString()[1] reacts to input that is not valid in the > client encoding.  Before that commit, the function would ignore encoding > problems excep