Re: Statistics Import and Export: difference in statistics dumped

2025-03-05 Thread Jeff Davis
On Wed, 2025-03-05 at 15:22 +0530, Ashutosh Bapat wrote: > Hmm. Updating the statistics without consuming more CPU is more > valuable when autovacuum is off it improves query plans with no extra > efforts. But if adding a new mode is some significant work, riding it > on top of autovacuum=off might

Re: Add contrib/pg_logicalsnapinspect

2025-03-05 Thread Masahiko Sawada
On Wed, Mar 5, 2025 at 3:10 PM Tom Lane wrote: > > Bertrand Drouvot writes: > > yeah makes sense. Done in the attached, and bonus point I realized that the > > test could be simplified (so, removing useless steps in passing). > > Just a side note: tayra showed two instances of this failure today

Re: track generic and custom plans in pg_stat_statements

2025-03-05 Thread Ilia Evdokimov
Hi, Thank you for your patch. It is really useful for tracking the history of generic and custom plan usage. At first glance, I have the following suggestions for improvement: 1. Is there any reason for the double check of cplan != NULL? It seems unnecessary, and we could simplify it to: -

Re: Statistics Import and Export

2025-03-05 Thread Nathan Bossart
On Wed, Mar 05, 2025 at 08:17:53PM -0500, Andres Freund wrote: > Right now --statistics more than doubles the number of queries that pg_dump > issues. That's oviously noticeable locally, but it's going to be really > noticeable when dumping across the network. > > I think we need to do more to les

Re: optimize file transfer in pg_upgrade

2025-03-05 Thread Nathan Bossart
On Wed, Mar 05, 2025 at 03:40:52PM -0500, Greg Sabino Mullane wrote: > I've seen various failures, but they always get caught quite early. > Certainly early enough to easily abort, fix perms/mounts/etc., then retry. > I think your instinct is correct that this reversion is more trouble than > its w

Re: Update Unicode data to Unicode 16.0.0

2025-03-05 Thread Nathan Bossart
On Wed, Mar 05, 2025 at 03:34:06PM -0800, Jeff Davis wrote: > On Wed, 2025-03-05 at 14:33 -0600, Nathan Bossart wrote: >> +   report_status(PG_WARNING, "warning"); >> +   pg_log(PG_WARNING, "Your installation contains >> relations that may be affected by a new version of Uni

Re: log_min_messages per backend type

2025-03-05 Thread Fujii Masao
On 2025/03/05 9:33, Euler Taveira wrote: > +    Valid BACKENDTYPE values are ARCHIVER, > +    AUTOVACUUM, BACKEND, > +    BGWORKER, BGWRITER, > +    CHECKPOINTER, LOGGER, > +    SLOTSYNCWORKER, WALRECEIVER, > +    WALSENDER, WALSUMMARIZER, and > +    WALWRITER. W

Re: Statistics Import and Export

2025-03-05 Thread Nathan Bossart
On Wed, Mar 05, 2025 at 08:54:35PM -0500, Corey Huinker wrote: > * The stats data is kinda heavy (most common value lists, most common > elements lists, esp for high stattargets), which would be a considerable > memory impact and some of those stats might not even be needed (example, > index stats

Re: jsonb_strip_nulls with arrays?

2025-03-05 Thread Florents Tselai
> On 6 Mar 2025, at 2:10 AM, Ian Lawrence Barwick wrote: > > Hi > > 2025年3月1日(土) 2:58 Florents Tselai : >> Please add this to the next Commitfest at >> https://commitfest.postgresql.org/52/ >> >> >> Added ; thanks >> https://commitfest.postgresql.org/patch/5260/ > > I see this was committ

Re: Monitoring gaps in XLogWalRcvWrite() for the WAL receiver

2025-03-05 Thread Michael Paquier
On Wed, Mar 05, 2025 at 08:04:44AM +, Bertrand Drouvot wrote: > On Wed, Mar 05, 2025 at 12:35:26PM +0900, Michael Paquier wrote: >> Perhaps there's a point in backpatching a portion of what's in the >> attached patch (the wait event?), but I am not planning to bother much >> with the stable bra

Re: track generic and custom plans in pg_stat_statements

2025-03-05 Thread Sami Imseih
> > Please see v2 > oops, had to fix a typo in meson.build. Please see v3. -- Sami v3-0001-add-plan_cache-counters-to-pg_stat_statements.patch Description: Binary data

Re: JSON_VALUE() behavior when RETURNING bytea (expected base64 decoding)

2025-03-05 Thread David G. Johnston
On Wednesday, March 5, 2025, Shay Rojansky wrote: > > SELECT JSON_VALUE(jsonb '"AQID"', '$' RETURNING bytea); -- Expected > 0x010203, got AQID > I get \x41514944 which is precisely what I would expect since it what this query results in as well: select 'AQID'::bytea; David J.

Re: JSON_VALUE() behavior when RETURNING bytea (expected base64 decoding)

2025-03-05 Thread Shay Rojansky
> > >> SELECT JSON_VALUE(jsonb '"AQID"', '$' RETURNING bytea); -- Expected >> 0x010203, got AQID >> > > I get \x41514944 which is precisely what I would expect since it what this > query results in as well: > > select 'AQID'::bytea; > If the behavior of RETURNING is meant to be identical to that o

Re: Orphaned users in PG16 and above can only be managed by Superusers

2025-03-05 Thread Ashutosh Sharma
Thanks, Nathan, for reviewing the patch. Below are my comments inline: On Thu, Mar 6, 2025 at 1:43 AM Nathan Bossart wrote: > > > * The patch alleges to only block DROP ROLE commands when there exists > _both_ admins of the target role and roles for which the target role is > an admin. Howev

Re: Orphaned users in PG16 and above can only be managed by Superusers

2025-03-05 Thread Ashutosh Sharma
Hi Robert, Thanks for the review comments. On Thu, Mar 6, 2025 at 2:10 AM Robert Haas wrote: > > On Wed, Mar 5, 2025 at 3:13 PM Nathan Bossart > wrote: > > * The patch alleges to only block DROP ROLE commands when there exists > > _both_ admins of the target role and roles for which the targ

Re: Hook for Selectivity Estimation in Query Planning

2025-03-05 Thread Andrei Lepikhov
On 5/3/2025 19:50, Aleksander Alekseev wrote: Andrei, Matthias, Could you explain why you think the Pluggable TOASTer proposal was similar? [...] I merely pointed out that adding hooks without any particular value for the Postgres users was criticized before, see for instance: Thank you for y

Re: Statistics Import and Export

2025-03-05 Thread Corey Huinker
> > One fairly easy win would be to stop issuing getAttributeStats() for > non-expression indexes. In most cases that'll already drastically cut down > on > the extra queries. That does seem like an easy win, especially since we're already using indexprs for some filters. I am concerned that, dow

Re: Statistics Import and Export

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 22:00:42 -0500, Corey Huinker wrote: > On Wed, Mar 5, 2025 at 9:18 PM Andres Freund wrote: > > On 2025-03-05 20:54:35 -0500, Corey Huinker wrote: > > > It's been considered and not ruled out, with a "let's see how the simple > > > thing works, first" approach. Considerations are

Re: Statistics Import and Export

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 20:54:35 -0500, Corey Huinker wrote: > It's been considered and not ruled out, with a "let's see how the simple > thing works, first" approach. Considerations are: > > * pg_stats is keyed on schemaname + tablename (which can also be indexes) > and we need to use that because of t

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-05 Thread Ajin Cherian
On Fri, Feb 28, 2025 at 11:34 PM Shubham Khanna wrote: > > > The attached Patch contains the suggested changes. > > Thanks and regards, > Shubham Khanna. Some comments: 1. + + -a + --all + + + For all source server non-template databases create subscriptions for +

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-05 Thread Peter Smith
Hi Shubham. Some review comments for patch v13-0001. == GENERAL 1. --cleanup-existing-publications I've never liked this proposed switch name much. e.g. why say "cleanup" instead of "drop"? What is the difference? Saying drop is very explicit about what the switch will do. e.g. why say "ex

Add arbitrary xid and mxid to pg_resetwal

2025-03-05 Thread Daniil Davydov
Hi, I prepared a patch that will allow us to set arbitrary values in -m and -x options for pg_resetwal. For now, it is not possible to specify a value that does not fit into existing SLRU segments, and main idea of this patch (for REL_17_STABLE) is to create such segments inside pg_resetwal's main(

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-05 Thread Michael Paquier
On Wed, Mar 05, 2025 at 08:55:55PM -0500, Andres Freund wrote: > Once we've slept for 10+ seconds without reaching the target, sleeping for > 100us is way too short a sleep and just wastes CPU cycles. A decent portion > of the CPU time when running under valgrind is wasted just due to way too > tig

Re: track generic and custom plans in pg_stat_statements

2025-03-05 Thread Sami Imseih
Thanks for the review! > I could see EXPLAIN being somewhat useful (especially for non-interactive > things like auto_explain), so a weak +1 on that. I'll make this a follow-up to this patch. > Definitely not useful for pg_stat_database IMHO. agree as well. I did not have strong feelings about

Re: Statistics Import and Export

2025-03-05 Thread Corey Huinker
> > Apologies if this has already been considered upthread, but would it be > possible to use one query to gather all the required information into a > sorted table? At a glance, it looks to me like it might be feasible. I > had a lot of luck with reducing the number per-object queries with that

Re: Add Pipelining support in psql

2025-03-05 Thread Michael Paquier
On Wed, Mar 05, 2025 at 03:25:12PM +0100, Daniel Verite wrote: > Anthonin Bonnefoy wrote: >> I do see the idea to make it easier to convert existing scripts into >> using pipelining. The main focus of the initial implementation was >> more on protocol regression tests with psql, so that's not

Re: JSON_VALUE() behavior when RETURNING bytea (expected base64 decoding)

2025-03-05 Thread Greg Sabino Mullane
It looks like your bytea_output is set to 'escape', which would explain what you are seeing. Try adding this in first: SET bytea_output = hex; SELECT JSON_VALUE(jsonb '"AQID"', '$' RETURNING bytea); That (hex) is the default value, so you must be setting it to escape somewhere. You can see where

Re: Statistics Import and Export

2025-03-05 Thread Andres Freund
Hi, On 2025-02-25 21:29:56 -0500, Corey Huinker wrote: > On Tue, Feb 25, 2025 at 9:00 PM Jeff Davis wrote: > > > On Mon, 2025-02-24 at 09:54 -0500, Andres Freund wrote: > > > Have you compared performance of with/without stats after these > > > optimizations? > > > > On unoptimized build with ass

Re: JSON_VALUE() behavior when RETURNING bytea (expected base64 decoding)

2025-03-05 Thread David G. Johnston
On Wednesday, March 5, 2025, Shay Rojansky wrote: > >>> SELECT JSON_VALUE(jsonb '"AQID"', '$' RETURNING bytea); -- Expected >>> 0x010203, got AQID >>> >> >> I get \x41514944 which is precisely what I would expect since it what >> this query results in as well: >> >> select 'AQID'::bytea; >> > > I

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 16:19:04 -0800, Jacob Champion wrote: > On Wed, Mar 5, 2025 at 9:28 AM Andres Freund wrote: > > Unrelated to the change in this patch, but tests really shouldn't use > > while(1) > > loops without a termination condition. If something is wrong, the test will > > hang indefinite

Re: support fast default for domain with constraints

2025-03-05 Thread jian he
hi. rearrange the patch. v3-0001 and v3-0002 is preparare patches. v3-0001 add function: ExecPrepareExprSafe and ExecInitExprSafe. v3-0002 add function: DomainHaveVolatileConstraints v3-0003 tests and apply fast default for domain with constraints. v3-0003 table with empty rows aligned with mast

Re: what's going on with lapwing?

2025-03-05 Thread Julien Rouhaud
On Tue, Mar 04, 2025 at 10:18:49AM -0500, Tom Lane wrote: > > But yeah, I thought we were overdue for a buildfarm release. > I'm pleased to see that Andrew just pushed one. FWIW I installed the client version 19.1 this morning and forced a run on HEAD and lapwing is back to green.

Re: Statistics Import and Export

2025-03-05 Thread Corey Huinker
On Wed, Mar 5, 2025 at 9:18 PM Andres Freund wrote: > Hi, > > On 2025-03-05 20:54:35 -0500, Corey Huinker wrote: > > It's been considered and not ruled out, with a "let's see how the simple > > thing works, first" approach. Considerations are: > > > > * pg_stats is keyed on schemaname + tablename

Re: Refactoring postmaster's code to cleanup after child exit

2025-03-05 Thread Noah Misch
On Tue, Mar 04, 2025 at 05:50:34PM -0500, Andres Freund wrote: > On 2024-12-09 00:12:32 +0100, Tomas Vondra wrote: > > [23:48:44.444](1.129s) ok 3 - reserved_connections limit > > [23:48:44.445](0.001s) ok 4 - reserved_connections limit: matches > > process ended prematurely at > > /home/user/work/

Hook for Selectivity Estimation in Query Planning

2025-03-05 Thread Andrei Lepikhov
Hi, I would like to discuss the introduction of a hook for evaluating the selectivity of an expression when searching for an optimal query plan. This topic has been brought up in various discussions, for example, in [1]. Currently, extensions that interact with the optimiser can only add the

RE: Selectively invalidate caches in pgoutput module

2025-03-05 Thread Hayato Kuroda (Fujitsu)
Dear Hou, Thanks for reading the thread! > > Attached patchset implemented with the approach. > > 0001 contains only part to avoid whole cache invalidation, for ADD/SET/DROP > > command. > > I think the changes proposed in 0001 are reasonable. Basically, any > modifications to pg_publication_rel

Re: Draft for basic NUMA observability

2025-03-05 Thread Jakub Wartak
On Tue, Mar 4, 2025 at 5:02 PM Bertrand Drouvot wrote: > > Cool! Attached is v7 > Thanks for the new version! ... and another one: 7b ;) > > > === 2 [..] > > Well, I've made query_numa a parameter there simply to avoid that code > > duplication in the first place, look at those TupleDescInitEnt

Re: doc: expand note about pg_upgrade's --jobs option

2025-03-05 Thread Daniel Gustafsson
> On 4 Mar 2025, at 19:08, Nathan Bossart wrote: > The attached patch replaces the "et cetera" with those two general categories. LGTM. -- Daniel Gustafsson

Re: Add contrib/pg_logicalsnapinspect

2025-03-05 Thread Bertrand Drouvot
Hi, On Wed, Mar 05, 2025 at 02:42:15PM +0530, Amit Kapila wrote: > On Wed, Mar 5, 2025 at 12:47 PM Bertrand Drouvot > wrote: > > > > Agree, PFA a patch doing so. > > > > It would be better if you could add a few comments atop the > permutation line to explain the working of the test. yeah makes

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2025-03-05 Thread Heikki Linnakangas
On 28/02/2025 03:53, Peter Geoghegan wrote: On Sat, Feb 8, 2025 at 8:47 AM Michail Nikolaev wrote: Just some commit messages + few cleanups. I'm worried about this: +These longer pin lifetimes can cause buffer exhaustion with messages like "no +unpinned buffers available" when the index has

RE: Selectively invalidate caches in pgoutput module

2025-03-05 Thread Hayato Kuroda (Fujitsu)
Dear hackers, I did a performance testing with HEAD and v2-0001, and confirmed that it could improve performance around 20% in the typical case. Workload == I emulated a scenario that there are many tables to be published and only one table is stop and resume publishing. In HEAD, ALTER PUBLIC

Allow database owners to CREATE EVENT TRIGGER

2025-03-05 Thread Steve Chavez
Hello hackers, Currently PostgreSQL only allows creating event triggers for superusers, this prevents usage on PostgreSQL service providers, which do not grant superuser access. This patch allows database owners to create event triggers, while preventing privilege escalation. Unlike superuser ev

Re: making EXPLAIN extensible

2025-03-05 Thread Andrei Lepikhov
On 4/3/2025 22:23, Robert Haas wrote: On Tue, Mar 4, 2025 at 1:53 PM Jeff Davis wrote: I don't expect there to be zillions of extensions that only add new and exciting explain options. Instead, it seems more likely that all TableAM and CustomScan extensions will have 1-3 new explain options, an

Re: Add regression test checking combinations of (object,backend_type,context) in pg_stat_io

2025-03-05 Thread Michael Paquier
On Wed, Mar 05, 2025 at 07:34:16AM +, Bertrand Drouvot wrote: > That would mean changing the test each time pgstat_tracks_io_object() is > modified in such a way that the output is changed. That's a good thing as > the writer will need to double check if the new output makes sense according > t

Re: Monitoring gaps in XLogWalRcvWrite() for the WAL receiver

2025-03-05 Thread Michael Paquier
On Wed, Mar 05, 2025 at 08:04:44AM +, Bertrand Drouvot wrote: > On Wed, Mar 05, 2025 at 12:35:26PM +0900, Michael Paquier wrote: > Also for sync? sync looks fine as issue_xlog_fsync() is being called in > XLogWalRcvFlush(), no? Yes, we're OK for the sync data aggregated in the WAL receiver on

Re: per backend WAL statistics

2025-03-05 Thread Xuneng Zhou
Subject: Clarification Needed on WAL Pending Checks in Patchset Hi, Thank you for the patchset. I’ve spent some time learning and reviewing it and have 2 comments. I'm new to PostgreSQL hacking, so please bear with me if I make mistakes or say something that seems trivial. I noticed that in pat

Re: Wrong results with subquery pullup and grouping sets

2025-03-05 Thread Dean Rasheed
On Wed, 5 Mar 2025 at 09:09, Richard Guo wrote: > > I noticed the adjacent code that sets wrap_non_vars to true if we > are considering an appendrel child subquery, and I doubt this is > necessary. > > /* > * If we are dealing with an appendrel member then anything that's not a > * simple Var

Re: Incorrect result of bitmap heap scan.

2025-03-05 Thread Matthias van de Meent
On Sun, 2 Mar 2025 at 01:35, Tom Lane wrote: > > Peter Geoghegan writes: > > Is everybody in agreement about committing and back patching this fix, > > which simply disables the optimization altogether? > > I myself don't see a better way, but thought I'd ask before proceeding > > with review and

Re: Considering fractional paths in Append node

2025-03-05 Thread Alexander Korotkov
On Wed, Mar 5, 2025 at 8:32 AM Andrei Lepikhov wrote: > On 5/3/2025 03:27, Alexander Korotkov wrote: > > On Mon, Mar 3, 2025 at 1:04 PM Andrei Lepikhov wrote: > >>> 2. As usage of root->tuple_fraction RelOptInfo it has been criticized, > >>> do you think we could limit this to some simple cases?

Re: Separate GUC for replication origins

2025-03-05 Thread Amit Kapila
On Wed, Mar 5, 2025 at 12:42 PM Masahiko Sawada wrote: > > On Tue, Mar 4, 2025 at 10:42 PM Amit Kapila wrote: > > > > On Wed, Mar 5, 2025 at 6:24 AM Euler Taveira wrote: > > > > > > On Sat, Mar 1, 2025, at 10:08 AM, Amit Kapila wrote: > > > > > > On Thu, Feb 13, 2025 at 6:48 AM Masahiko Sawada

Re: Wrong results with subquery pullup and grouping sets

2025-03-05 Thread Richard Guo
On Wed, Mar 5, 2025 at 11:02 AM Richard Guo wrote: > It seems to me that simple Var expressions in a subquery's target list > also need to retain their separate identity in order to match grouping > set columns after subquery pullup, not just non-var expressions. I noticed the adjacent code that

Re: [RFC] Lock-free XLog Reservation from WAL

2025-03-05 Thread Yura Sokolov
05.03.2025 08:39, Zhou, Zhiguo пишет: > > > On 2/23/2025 8:03 PM, Yura Sokolov wrote: >> 14.02.2025 11:41, Zhou, Zhiguo пишет: >>> >>> >>> On 2/11/2025 9:25 AM, Japin Li wrote: On Mon, 10 Feb 2025 at 22:12, "Zhou, Zhiguo" wrote: > On 2/5/2025 4:32 PM, Japin Li wrote: >> On Mon, 27 J

Re: Statistics Import and Export: difference in statistics dumped

2025-03-05 Thread Ashutosh Bapat
On Tue, Mar 4, 2025 at 11:45 PM Jeff Davis wrote: > > On Tue, 2025-03-04 at 10:28 +0530, Ashutosh Bapat wrote: > > > > > > > What solution are you suggesting? The only one that comes to mind > > > is > > > moving everything to SECTION_POST_DATA, which is possible, but it > > > seems > > > like a b

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-05 Thread Nisha Moond
On Tue, Mar 4, 2025 at 8:05 PM Shubham Khanna wrote: > > The attached Patch contains the suggested changes. > Hi Shubham, Here are few comments for 040_pg_createsubscriber.pl 1) +# Run pg_createsubscriber on node S using '--cleanup-existing-publications'. +# --verbose is used twice to show more

Re: Log connection establishment timings

2025-03-05 Thread Bertrand Drouvot
Hi, On Tue, Mar 04, 2025 at 06:25:42PM -0500, Melanie Plageman wrote: > Attached v9 implements log_connections as an enum with a new third > value "timings". > > On Mon, Mar 3, 2025 at 11:14 AM Bertrand Drouvot > wrote: > > > > > > On Fri, Feb 28, 2025 at 05:52:35PM -0500, Melanie Plageman wrote

Re: Separate GUC for replication origins

2025-03-05 Thread Peter Eisentraut
On 11.02.25 21:25, Euler Taveira wrote: Here is another patch that only changes the GUC name to max_replication_origin_sessions. I think the naming and description of this is still confusing. What does this name mean? There is (I think) no such thing as a "replication origin session". So wh

Re: support fast default for domain with constraints

2025-03-05 Thread jian he
On Wed, Mar 5, 2025 at 11:13 AM Tom Lane wrote: > > This patch appears to summarily throw away a couple of > backwards-compatibility concerns that the previous round > took care to preserve: > > * not throwing an error if the default would fail the domain > constraints, but the table is empty so t

Re: Patch: Cover POSITION(bytea,bytea) with tests

2025-03-05 Thread Ilia Evdokimov
On 27.02.2025 17:40, Aleksander Alekseev wrote: Hi, This function is currently not covered by any tests. The proposed patch fixes this. -- Best regards, Aleksander Alekseev Hi, +1. The 'POSISTION' tests include 'text', 'bits' types, but not 'bytea'. -- Best regards, Ilia Evdokimov, Tanto

Re: doc: expand note about pg_upgrade's --jobs option

2025-03-05 Thread Magnus Hagander
On Wed, Mar 5, 2025 at 11:00 AM Daniel Gustafsson wrote: > > On 4 Mar 2025, at 19:08, Nathan Bossart > wrote: > > > The attached patch replaces the "et cetera" with those two general > categories. > > LGTM. > Another option that I think would also work is to just cut down the details to just "T

Re: per backend WAL statistics

2025-03-05 Thread Bertrand Drouvot
Hi, On Wed, Mar 05, 2025 at 05:45:57PM +0800, Xuneng Zhou wrote: > Subject: Clarification Needed on WAL Pending Checks in Patchset > > Hi, > > Thank you for the patchset. I’ve spent some time learning and reviewing it > and have 2 comments. Thanks for looking at it! > I noticed that in patches

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Tom Lane
Andres Freund writes: > On 2025-03-05 11:19:46 -0500, Tom Lane wrote: >> However, we seem to be moving towards a situation where each type of CI run >> is a special snowflake that differs in multiple dimensions from other types. >> That might make it difficult to figure out which dimension is resp

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 08:16:45 -0800, Jacob Champion wrote: > From efc9fc3b3993601e9611131f229fbcaf4daa46f1 Mon Sep 17 00:00:00 2001 > From: Michael Paquier > Date: Wed, 5 Mar 2025 13:30:43 +0900 > Subject: [PATCH 1/2] Fix race condition in pre-auth test > > --- > src/test/authentication/t/007_pre_

Re: making EXPLAIN extensible

2025-03-05 Thread Jeff Davis
It would be good to clarify whether this is for (a) experimenting with explain options that might be useful in core some day; or (b) special developer-only options that would never be useful in core; or (c) production-grade explain options specific to an extension. On Tue, 2025-03-04 at 16:23 -050

Re: making EXPLAIN extensible

2025-03-05 Thread Robert Haas
On Wed, Mar 5, 2025 at 12:52 PM Jeff Davis wrote: > It would be good to clarify whether this is for (a) experimenting with > explain options that might be useful in core some day; or (b) special > developer-only options that would never be useful in core; or (c) > production-grade explain options

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2025-03-05 Thread Matthias van de Meent
On Wed, 5 Mar 2025 at 10:04, Heikki Linnakangas wrote: > > On 28/02/2025 03:53, Peter Geoghegan wrote: > > On Sat, Feb 8, 2025 at 8:47 AM Michail Nikolaev > > wrote: > >> Just some commit messages + few cleanups. > > > > I'm worried about this: > > > > +These longer pin lifetimes can cause buffer

Re: Hook for Selectivity Estimation in Query Planning

2025-03-05 Thread Aleksander Alekseev
Andrei, Matthias, > Could you explain why you think the Pluggable TOASTer proposal was similar? > [...] I merely pointed out that adding hooks without any particular value for the Postgres users was criticized before, see for instance: https://www.postgresql.org/message-id/20230206104917.sipa7nz

Re: Should work_mem be stable for a prepared statement?

2025-03-05 Thread James Hunter
On Fri, Feb 28, 2025 at 2:34 PM Tom Lane wrote: > > Sami Imseih writes: > > However, I think any GUC that can influence the planner > > should be considered for consistency in behavior. > > It was mentioned above with the enable_* GUCs, but another > > one I can think of is the max_parallel_worke

Re: making EXPLAIN extensible

2025-03-05 Thread Tom Lane
Robert Haas writes: > OK. It sounds to me like there is a good amount of support for going > forward with something like what I have, even though some people might > also like other things. What I feel is currently lacking is some > review of the actual code. Would anyone like to do that? Here's

Re: explain plans for foreign servers

2025-03-05 Thread Jeff Davis
On Wed, 2025-02-26 at 13:13 -0600, Sami Imseih wrote: > 1/ The use of NOTICE to propagate the explain plan. > I see the message content is checked, but this does not look robust > and could lead to > some strange results if another ExecutorRun hook emits a similar > notice message. Fundamentally,

Re: Hook for Selectivity Estimation in Query Planning

2025-03-05 Thread Aleksander Alekseev
Hi, > I would like to discuss the introduction of a hook for evaluating the > selectivity of an expression when searching for an optimal query plan. > This topic has been brought up in various discussions, for example, in [1]. > > [...] As I vaguely recall recent proposals like this ("Pluggable T

Re: is git.postgresql.org working fine?

2025-03-05 Thread Aleksander Alekseev
Hi, > > Is it only me? > > I've had trouble in the past with cloning on old/slow machines --- it > seems to time out after awhile. But AFAIR the symptom was an error > message not "stalling", so maybe that's unrelated. > > Anyway, I tried it just now and didn't observe any problem: > > [...] For

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Peter Geoghegan
On Wed, Mar 5, 2025 at 11:06 AM Andres Freund wrote: > Post-commit issues due to debug_parallel_query=regress seem rather common, > surely not helped by CI/cfbot not flagging them. I wonder if we ought to make > one of the CI tasks use debug_parallel_query=regress, to avoid that problem? That wou

Re: Enhance file_fdw to report processed and skipped tuples in COPY progress

2025-03-05 Thread Fujii Masao
On 2024/11/30 15:23, Kirill Reshke wrote: On Fri, 11 Oct 2024 at 06:53, Fujii Masao wrote: However, this issue already exists without the proposed patch. Since file_fdw already reports progress partially, querying multiple file_fdw tables can lead to inaccurate or confusing progress reports.

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-05 Thread Jacob Champion
On Wed, Mar 5, 2025 at 5:47 AM Jacob Champion wrote: > > So while we're at it, should we add a > `backend_type = 'client backend'` filter to stop that from flaking in > the future? That would further align this query with the > wait_for_event() implementation. More concretely: here's a squashable

Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Andres Freund
Hi, In https://postgr.es/m/CAH2-WzkYXSnM60ZNo-vQLxFoGzHLHFD0x%3DiPHF6VGxiZmWUuwQ%40mail.gmail.com Peter wrote: On 2025-03-05 09:37:05 -0500, Peter Geoghegan wrote: > Committed just now. Thanks again. But since had to revert, due to BF issues: commit d00107cd63e780753aa25563fa37603369997d0c Aut

Re: Allow database owners to CREATE EVENT TRIGGER

2025-03-05 Thread Isaac Morland
On Wed, 5 Mar 2025 at 10:28, Tom Lane wrote: > I wrote: > > Or in other words: not-superuser to superuser is far from the only > > type of privilege escalation that we need to prevent. > > After reflecting on that for a moment: maybe say that an event trigger > fires for queries that are run by a

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Tom Lane
Andres Freund writes: > Post-commit issues due to debug_parallel_query=regress seem rather common, > surely not helped by CI/cfbot not flagging them. I wonder if we ought to make > one of the CI tasks use debug_parallel_query=regress, to avoid that problem? Yeah, it certainly seems like a test co

Re: doc: expand note about pg_upgrade's --jobs option

2025-03-05 Thread Nathan Bossart
On Wed, Mar 05, 2025 at 09:35:27AM -0600, Nathan Bossart wrote: > On Wed, Mar 05, 2025 at 01:52:40PM +0100, Magnus Hagander wrote: >> Another option that I think would also work is to just cut down the details >> to just "The --jobs option allows multiple CPU cores to be >> used". > > That's fine

Re: making EXPLAIN extensible

2025-03-05 Thread Robert Haas
On Wed, Mar 5, 2025 at 12:20 PM Jeff Davis wrote: > I didn't look into the technical details to see what might be required > to allow that kind of collaboration, and I am not suggesting you > redesign the entire feature around that idea. OK. It sounds to me like there is a good amount of support

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 12:29:15 -0500, Tom Lane wrote: > Andres Freund writes: > > I guess we could be add a "standardized" section at the top of each task > > describing their oddities? Not sure it's worth it. > > I think this does need to be documented somewhere/somehow, just so > that people don't

Re: new commitfest transition guidance

2025-03-05 Thread Daniel Gustafsson
> On 4 Mar 2025, at 15:10, jian he wrote: > On Tue, Feb 4, 2025 at 2:39 PM Daniel Gustafsson wrote: >> They will be on the wiki shortly, I've taken notes of all discussions and am >> busy tidying them up to be able to publish them once the participants have >> proofread to ensure noone is misatt

should num_custom_plans be reset after plan invalidation?

2025-03-05 Thread Sami Imseih
Hi, While examining plan caches, I noticed that when a generic plan is invalidated, the next execution of the prepared statement still results in a generic plan. This is of course with the default plan_cache_mode. This behavior might go unnoticed since plan cache invalidations are relatively unco

Re: making EXPLAIN extensible

2025-03-05 Thread Jeff Davis
On Tue, 2025-03-04 at 16:23 -0500, Robert Haas wrote: > But, I'm doubtful that > letting unrelated extensions try to share the same option name is > that > thing. This sub-discussion started because we were wondering whether to prefix the options. I'm just pointing out that, even if there is a co

Re: Expanding HOT updates for expression and partial indexes

2025-03-05 Thread Burd, Greg
Hello, I've rebased and updated the patch a bit. The biggest change is that the performance penalty measured with v1 of this patch is essentially gone in v10. The overhead was due to re-creating IndexInfo information unnecessarily, which I found existed in the estate. I've added a few fields

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-03-05 Thread Daniel Gustafsson
> On 3 Mar 2025, at 16:23, Daniel Gustafsson wrote: > The attached 0002 also contains documentation touchups and comments etc. 0001 > is your patch from v6. I managed to misunderstand skip blocks in TAP tests in the 0002, so the attached version fixes that. It has been failing on Debian in CI

Re: Allow LISTEN on patterns

2025-03-05 Thread Greg Sabino Mullane
Does not seem like a bug to me. Just the normal auto-lowercase encountered in every other SQL command. See: greg=# select * from pg_listening_channels(); pg_listening_channels --- (0 rows) greg=# listen foo; LISTEN greg=# select * from pg_listening_channels(); pg_listening_c

Re: Allow LISTEN on patterns

2025-03-05 Thread Tom Lane
Trey Boudreau writes: > I didn’t see any past references to the pg_notify() ‘anomaly’: > LISTEN FOO; > NOTIFY FOO, ‘BAR’; -- notification delivered > PERFORM pg_notify(‘FOO’, ‘BAR’); -- notification NOT delivered > PERFORM pg_notify(‘foo’, ‘BAR’); -- notification delivered > Can we come to some

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Tom Lane
Andres Freund writes: > On 2025-03-05 12:29:15 -0500, Tom Lane wrote: >> I think this does need to be documented somewhere/somehow, just so >> that people don't waste time focusing on "it's failing on FreeBSD" >> when the actual cause is some other thing we happened to load >> onto that task. > 0

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Peter Geoghegan
On Wed, Mar 5, 2025 at 1:34 PM Andres Freund wrote: > Pushed both patches. Thanks! -- Peter Geoghegan

Re: Make tuple deformation faster

2025-03-05 Thread Jeff Davis
On Thu, 2025-03-06 at 01:07 +1300, David Rowley wrote: > I've attached the results. The 3990x with clang looks good, but the > rest are mostly slower. I am still curious why. If it's due to compiler misoptimization, is that kind of thing often misoptimized, or is there something we're doing in pa

track generic and custom plans in pg_stat_statements

2025-03-05 Thread Sami Imseih
Hi, Currently, there is little visibility for queries that are being executed using generic or custom plans. There is pg_prepared_statements which show generic_plans and custom_plans as of d05b172a760, but this information is backend local and not very useful to a DBA that wishes to track this inf

Re: per backend WAL statistics

2025-03-05 Thread Bertrand Drouvot
Hi, On Wed, Mar 05, 2025 at 09:18:16AM -0500, Andres Freund wrote: > Hi, > > On 2025-03-05 13:03:07 +, Bertrand Drouvot wrote: > > But yeah, if 0002 in [1] does not go in, then your concern is valid, so > > adding > > the extra check in the attached. > > This crashes in cfbot: > > https://

Re: Allow LISTEN on patterns

2025-03-05 Thread Trey Boudreau
> On Mar 5, 2025, at 10:42 AM, Tom Lane wrote: > > Anyway, I encourage reading some of the past threads on this > topic. > I didn’t see any past references to the pg_notify() ‘anomaly’: LISTEN FOO; NOTIFY FOO, ‘BAR’; -- notification delivered PERFORM pg_notify(‘FOO’, ‘BAR’); -- notification N

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 11:19:46 -0500, Tom Lane wrote: > Andres Freund writes: > > Post-commit issues due to debug_parallel_query=regress seem rather common, > > surely not helped by CI/cfbot not flagging them. I wonder if we ought to > > make > > one of the CI tasks use debug_parallel_query=regress,

Re: Non-text mode for pg_dumpall

2025-03-05 Thread Mahendra Singh Thalor
Thanks Alvaro for feedback and review. On Wed, 5 Mar 2025 at 20:42, Álvaro Herrera wrote: > > Disclaimer: I didn't review these patches fully. > > On 2025-Mar-05, Mahendra Singh Thalor wrote: > > > On Wed, 5 Mar 2025 at 01:02, Álvaro Herrera wrote: > > > > > A database name containing a newline

Re: new commitfest transition guidance

2025-03-05 Thread Álvaro Herrera
On 2025-Mar-05, Daniel Gustafsson wrote: > I would avoid using Google for finding content on the wiki, the search > function > on the wiki itself is generally more reliable. Searching for FOSDEM 2025 > returns the following as the top result: > > https://wiki.postgresql.org/wiki/FOSDEM/PGDa

Re: Should we add debug_parallel_query=regress to CI?

2025-03-05 Thread Andres Freund
Hi, On 2025-03-05 13:10:00 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2025-03-05 12:29:15 -0500, Tom Lane wrote: > >> I think this does need to be documented somewhere/somehow, just so > >> that people don't waste time focusing on "it's failing on FreeBSD" > >> when the actual cause is

Re: explain plans for foreign servers

2025-03-05 Thread Tom Lane
Jeff Davis writes: > Ideally, we'd have EXPLAIN ANALYZE return two result sets, kind of like > how a query with a semicolon returns two result sets. That changes the > expected message flow for EXPLAIN ANALYZE, though, so we'd need a new > option so we are sure the client is expecting it (is this

Re: Interrupts vs signals

2025-03-05 Thread Andres Freund
Hi, On 2025-02-28 22:24:56 +0200, Heikki Linnakangas wrote: > I noticed that the ShutdownLatchSupport() function is unused. The first > patch removes it. Looks like that's the case since commit 80a8f95b3bca6a80672d1766c928cda34e979112 Author: Andres Freund Date: 2021-08-13 05:49:26 -0700

Re: explain plans for foreign servers

2025-03-05 Thread Sami Imseih
>> What if we do something like a new EXPLAIN option which returns all >> the rows >> back to the client, and then writes out the plan to some local >> memory. > That's another idea, but I am starting to think returning two result > sets from EXPLAIN ANALYZE would be generally useful. I did not t

  1   2   >