Re: BitmapHeapScan streaming read user and prelim refactoring

2025-03-22 Thread Andres Freund
Hi, On 2025-03-22 16:14:11 -0400, Andres Freund wrote: > On 2025-03-22 22:00:00 +0200, Alexander Lakhin wrote: > > @@ -24,14 +24,14 @@ > >  SELECT count(*) FROM bmscantest WHERE a = 1 AND b = 1; > >   count > >  --- > > -    23 > > +    18 > >  (1 row) > > Is it possible that this is the bug

Re: Update LDAP Protocol in fe-connect.c to v3

2025-03-22 Thread Tom Lane
Andrew Jackson writes: > Currently the LDAP usage in fe-connect.c does not explicitly set the > protocol version to v3. This causes issues with many LDAP servers as they > will often require clients to use the v3 protocol and disallow any use of > the v2 protocol. This is the first complaint I ca

Re: Proposal - Allow extensions to set a Plan Identifier

2025-03-22 Thread Sami Imseih
> On Sat, Mar 22, 2025 at 11:50:06PM +0100, Andrei Lepikhov wrote: > > planId actually looks less controversial than queryId or plan_node_id. At > > the same time, it is not free from the different logic that extensions may > > incorporate into this value: I can imagine, for example, the attempt of

Re: Add estimated hit ratio to Memoize in EXPLAIN to explain cost adjustment

2025-03-22 Thread Ilia Evdokimov
After considering the opinions expressed in this discussion, I tend to agree more with David. If we add info about estimated unique keys and estimated capacity, then any additional information - such as evict_ratio and hit_ratio - can also be calculated, as EXPLAIN ANALYZE provides all the nece

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Michael Paquier
On Sat, Mar 22, 2025 at 09:04:19PM -0400, Tom Lane wrote: > Right. I'm arguing that that's good. The proposed patch already > obscures the difference between similar table names in different > (temp) schemas, and I'm suggesting that taking that a bit further > would be fine. > > Note that if the

Re: Add Postgres module info

2025-03-22 Thread Tom Lane
I spent awhile reviewing the v5 patch, and here's a proposed v6. Some notes: * I didn't like depending on offsetof(Pg_magic_struct, module_extra) to determine which parts of the struct are checked for compatibility. It just seems way too easy to break that with careless insertion of new fields, an

Update LDAP Protocol in fe-connect.c to v3

2025-03-22 Thread Andrew Jackson
Currently the LDAP usage in fe-connect.c does not explicitly set the protocol version to v3. This causes issues with many LDAP servers as they will often require clients to use the v3 protocol and disallow any use of the v2 protocol. Further the other usage of LDAP in postgres (in `backend/libpq/au

Re: Proposal - Allow extensions to set a Plan Identifier

2025-03-22 Thread Michael Paquier
On Fri, Mar 21, 2025 at 09:25:24AM -0400, Sami Imseih wrote: >> planner() is the sole place in the core code where the planner hook >> can be called. Shouldn't we have at least a call to >> pgstat_report_plan_id() after planning a query? At least that should >> be the behavior I'd expect, where a

Re: meson and check-tests

2025-03-22 Thread Rustam ALLAKOV
Hi, please note, same file `/src/tools/testwrap` on the same line number is being changed in this patch [1] in the same commitfest. [1] https://commitfest.postgresql.org/patch/5602/ Regards, Rustam Allakov

Re: speedup COPY TO for partitioned table.

2025-03-22 Thread vignesh C
On Tue, 11 Mar 2025 at 18:24, jian he wrote: > > after my change: > > COPY TO can be used only with plain tables, not views, and does not > copy rows from child tables, > however COPY TO can be used with partitioned tables. > For example, in a table inheritance hierarchy, COPY table T

Re: Next commitfest app release is planned for March 18th

2025-03-22 Thread Jelte Fennema-Nio
On Fri, 21 Mar 2025 at 18:53, Tom Lane wrote: > > Peter Eisentraut writes: > > - If I'm the committer for a patch but not a reviewer, and the patch is > > in "needs review" status, then the patch is formally speaking not > > actionable by me and should not be under "Patches that are ready for > >

Improve error reporting for few options in pg_createsubscriber

2025-03-22 Thread vignesh C
Hi, Currently, error reports for database, publication, subscription, and replication slots do not include the option name. This has been addressed by including the option name in the error messages, ensuring consistency similar to remove option. Additionally, pg_log_error and exit(1) have been re

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-22 Thread Shubham Khanna
On Sat, Mar 22, 2025 at 6:23 PM vignesh C wrote: > > On Fri, 21 Mar 2025 at 18:59, Shubham Khanna > wrote: > > > > > > During the recent testing, I observed that the tests were failing when > > using wait_for_slot_catchup(). To address this, I reverted to using > > wait_for_subscription_sync(), w

Re: Next commitfest app release is planned for March 18th

2025-03-22 Thread Tom Lane
Peter Geoghegan writes: > On Sat, Mar 22, 2025 at 7:15 AM Jelte Fennema-Nio wrote: >> Yeah I think we need a way to "star" a patch (not just for committers, >> but for anyone). > Right. In my view "starring" something should definitely not appear in > the "History Log" of the patch; it should be

Re: Remove redundant if-else in EXPLAIN by using ExplainPropertyText

2025-03-22 Thread David Rowley
On Fri, 21 Mar 2025 at 09:24, Ilia Evdokimov wrote: > Thanks to David [0], we found that the if and else branches contained > equivalent code. Since ExplainPropertyText already handles non-text > formats, the extra condition is unnecessary. > > I reviewed other files related to EXPLAIN where simil

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-03-22 Thread Alexander Lakhin
Hello Melanie, 15.03.2025 16:43, Melanie Plageman wrote: On Thu, Mar 13, 2025 at 5:41 PM Melanie Plageman wrote: Overall, I feel pretty good about merging this once Thomas merges the read stream patches. This was committed in 944e81bf99db2b5b70b, 2b73a8cd33b745c, and c3953226a07527a1e2. I've

Re: Parallel heap vacuum

2025-03-22 Thread Andres Freund
Hi, On 2025-03-20 01:35:42 -0700, Masahiko Sawada wrote: > One plausible solution would be that we don't use ReadStream in > parallel heap vacuum cases but directly use > table_block_parallelscan_xxx() instead. It works but we end up having > two different scan methods for parallel and non-paralle

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-03-22 Thread Andres Freund
Hi, On 2025-03-22 22:00:00 +0200, Alexander Lakhin wrote: > 15.03.2025 16:43, Melanie Plageman wrote: > > On Thu, Mar 13, 2025 at 5:41 PM Melanie Plageman > > wrote: > > > Overall, I feel pretty good about merging this once Thomas merges the > > > read stream patches. > > This was committed in 94

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-03-22 Thread Jelte Fennema-Nio
On Wed, 19 Mar 2025 at 14:15, torikoshia wrote: > BTW based on your discussion, I thought this patch could not be merged > anytime soon. Does that align with your understanding? Yeah, that aligns with my understanding. I don't think it's realistic to get this merged before the code freeze, but I

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Tom Lane
Michael Paquier writes: > On Sat, Mar 22, 2025 at 10:43:00AM +0100, Christoph Berg wrote: >> Are we at the point where the patch is already Ready for Committer? > I'll think a bit more about how to document all that. Anyway, yes, > I'm OK with the per-field custom_query_jumble, so let's move on

Re: making EXPLAIN extensible

2025-03-22 Thread Robert Haas
On Sat, Mar 22, 2025 at 4:46 AM Andrei Lepikhov wrote: > I skimmed through the code and tested how it works. > It looks good and has no apparent architectural dependencies. > But I haven't scrutinised it line-by-line and do not intend to do so. > I wanna say I hate the design of this module. Havin

Re: Reduce TupleHashEntryData struct size by half

2025-03-22 Thread Jeff Davis
On Tue, 2025-03-04 at 17:28 -0800, Jeff Davis wrote: > My results (with wide tables): > >     GROUP BY  EXCEPT >    master:  2151    1732 >    entire v8 series:    2054    1740 I'm not sure what I did with the EXCEPT test, above, perhaps I had f

Bug - DoS - Handler function lookups consider non-handler functions

2025-03-22 Thread David G. Johnston
Today: create extension tsm_system_rows ; create schema s1; create function s1.system_rows(internal) returns void language c as 'tsm_system_rows.so', 'tsm_system_rows_handler'; set search_path to s1,public,pg_catalog; select count(*) from pg_class tablesample system_rows(0); ERROR: function syste

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-03-22 Thread David G. Johnston
On Fri, Mar 21, 2025 at 10:23 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > Then this would benefit from the new function I suggest creating since it > apparently has the same, IMO, bug. > > Concretely like I posted here: https://www.postgresql.org/message-id/cakfquwybtck+uw-byfchhp8

Re: Bug - DoS - Handler function lookups consider non-handler functions

2025-03-22 Thread Tom Lane
"David G. Johnston" writes: > create extension tsm_system_rows ; > create schema s1; > create function s1.system_rows(internal) returns void language c as > 'tsm_system_rows.so', 'tsm_system_rows_handler'; > set search_path to s1,public,pg_catalog; > select count(*) from pg_class tablesample syste

Re: Next commitfest app release is planned for March 18th

2025-03-22 Thread Peter Geoghegan
On Sat, Mar 22, 2025 at 10:49 AM Tom Lane wrote: > > Personally, I think it'd be most useful for the dashboard to show all > > open patches. > > Surely not "all"? We have that display already. Should be more like > "patches that I have expressed an interest in or have reason to take > current or

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Tom Lane
I wrote: > So my feeling is: if we think this is the behavior we want, let's do > it across the board. I suggest that we simply drop the relid from the > jumble and use the table alias (that is, eref->aliasname) instead. I experimented with this trivial fix (shown in-line to keep the cfbot from t

Re: Proposal: Progressive explain

2025-03-22 Thread Robert Haas
On Wed, Mar 19, 2025 at 6:53 PM Rafael Thofehrn Castro wrote: > The strategy I used here is to use a MemoryContextCallback > (ProgressiveExplainReleaseFunc), configured in the memory context > where the query is being executed, being responsible for calling > ProgressiveExplainCleanup() if the que

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-03-22 Thread Andres Freund
Hi, On 2025-03-22 16:42:42 -0400, Andres Freund wrote: > Hm, it's clearly related to the all-visible path, but I think the bug is > actually independent of what was reported there. The bug you reported is > perfectly reproducible, without any concurrency, after all. > > Here's a smaller reproduce

Re: Using read_stream in index vacuum

2025-03-22 Thread Andrey Borodin
> On 22 Mar 2025, at 00:23, Melanie Plageman wrote: > > > I've committed the btree and gist read stream users. Cool! Thanks! > I think we can > come back to the test after feature freeze and make sure it is super > solid. +1. > On 22 Mar 2025, at 02:54, Melanie Plageman wrote: > > On F

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-22 Thread Michael Paquier
On Thu, Mar 20, 2025 at 06:16:44PM +0900, Ryo Kanbayashi wrote: > Sorry, I found a miss on 006_service.pl. > Fixed patch is attached... Please note that the commit fest app needs all the patches of a a set to be posted in the same message. In this case, v2-0001 is not going to get automatic test

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-03-22 Thread Antonin Houska
Alvaro Herrera wrote: > I rebased this patch series; here's v09. No substantive changes from v08. > I made sure the tree still compiles after each commit. Thanks. > I did look at 0002 again (and renamed the members of the new struct by > adding a p_ prefix, as well as fixing the references to

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Christoph Berg
Re: To Michael Paquier > > >> +#define JUMBLE_CUSTOM(nodetype, item) \ > > >> +_jumble##nodetype##_##item(jstate, expr, expr->item) > > > > In this one, I want to mean that we require a custom per-field > > function to look like that: > > _jumbleNodefoo_field(JumbleState *jstate, NodeFoo *

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Christoph Berg
Re: Michael Paquier > >> + * Note that the argument types are enforced for the per-field custom > >> + * functions. > >> + */ > >> +#define JUMBLE_CUSTOM(nodetype, item) \ > >> + _jumble##nodetype##_##item(jstate, expr, expr->item) > > In this one, I want to mean that we require a custom per-fiel

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-03-22 Thread Antonin Houska
Robert Haas wrote: > On Thu, Mar 20, 2025 at 2:09 PM Antonin Houska wrote: > > Robert Haas wrote: > > > Is there a README or a long comment in here someplace that is a good > > > place to read to understand the overall design of this feature? > > > > I tried to explain how it works in the commi

Re: Adding extension default version to \dx

2025-03-22 Thread Jelte Fennema-Nio
On Tue, 25 Feb 2025 at 17:11, Nathan Bossart wrote: > > On Tue, Feb 25, 2025 at 04:42:37PM +0100, Magnus Hagander wrote: > > Thanks goes to both you and the previous responders - I did manage to mute > > this thread away and missed the early replies, but got Jeltes the other day > > which brought

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-22 Thread vignesh C
On Fri, 21 Mar 2025 at 18:59, Shubham Khanna wrote: > > > During the recent testing, I observed that the tests were failing when > using wait_for_slot_catchup(). To address this, I reverted to using > wait_for_subscription_sync(), which was employed previously and has > proven to be more reliable

Re: RFC: Additional Directory for Extensions

2025-03-22 Thread Andrew Dunstan
On 2025-03-21 Fr 12:42 PM, Tom Lane wrote: Matheus Alcantara writes: On Thu, Mar 20, 2025 at 7:38 PM Andrew Dunstan wrote: (wondering if this another of these cases where the "path includes postgres" thing bites us, and we're looking in the wrong place) Nope, testing shows it's not that, so

Re: making EXPLAIN extensible

2025-03-22 Thread Andrei Lepikhov
On 3/21/25 20:54, Robert Haas wrote: On Fri, Mar 21, 2025 at 2:37 PM Tom Lane wrote: Here's v9, which also adds 'SET debug_parallel_query = off' to the pg_overexplain tests, per CI, because the test results are not (and cannot realistically be made) stable under under that option. I skimmed thr

Re: Parallel heap vacuum

2025-03-22 Thread Melanie Plageman
On Thu, Mar 20, 2025 at 4:36 AM Masahiko Sawada wrote: > > When testing the multi passes of table vacuuming, I found an issue. > With the current patch, both leader and parallel workers process stop > the phase 1 as soon as the shared TidStore size reaches to the limit, > and then the leader resum

Re: Next commitfest app release is planned for March 18th

2025-03-22 Thread Peter Geoghegan
On Sat, Mar 22, 2025 at 7:15 AM Jelte Fennema-Nio wrote: > Yeah I think we need a way to "star" a patch (not just for committers, > but for anyone). After creating this initial version of this dashboard > I realized it's also possible to "subscribe to updates" for a patch, > which means you'll get

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-22 Thread Ryo Kanbayashi
On Sat, Mar 22, 2025 at 4:46 PM Michael Paquier wrote: > > On Thu, Mar 20, 2025 at 06:16:44PM +0900, Ryo Kanbayashi wrote: > > Sorry, I found a miss on 006_service.pl. > > Fixed patch is attached... > > Please note that the commit fest app needs all the patches of a a set > to be posted in the sam

Re: Proposal - Allow extensions to set a Plan Identifier

2025-03-22 Thread Michael Paquier
On Sat, Mar 22, 2025 at 11:50:06PM +0100, Andrei Lepikhov wrote: > planId actually looks less controversial than queryId or plan_node_id. At > the same time, it is not free from the different logic that extensions may > incorporate into this value: I can imagine, for example, the attempt of > uniqu

Re: pg_stat_statements and "IN" conditions

2025-03-22 Thread Sami Imseih
> Sure, it's important and I'm planning to tackle this next. If you want, > we can collaborate on a patch for that. I spent some time looking some more at this, and I believe all that needs to be done is check for a PRAM node with a type of PARAM_EXTERN. During planning the planner turns the Para

Re: AIO v2.5

2025-03-22 Thread Noah Misch
On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > Attached v2.11, with the following changes: > - Added an error check for FileStartReadV() failing > > FileStartReadV() actually can fail, if the file can't be re-opened. I > thought it'd be important for the error message to dif

Re: Update LDAP Protocol in fe-connect.c to v3

2025-03-22 Thread Andrew Jackson
> This is the first complaint I can recall hearing about that, so exactly which ones are "many"? I've tested a 2 before figuring out about the v3 issue. lldap[0] and the docker image osixia/docker-openldap[1]. - lldap gives the following error message when I attempt to connect without the patch "

Re: Update Unicode data to Unicode 16.0.0

2025-03-22 Thread Jeremy Schneider
On Fri, 21 Mar 2025 13:45:24 -0700 Jeff Davis wrote: > > Maybe we should actually move in the direction of having encodings > > that are essentially specific versions of Unicode. Instead of just > > having UTF-8 that accepts whatever, you could have UTF-8.v16.0.0 or > > whatever, which would only

Re: Proposal - Allow extensions to set a Plan Identifier

2025-03-22 Thread Andrei Lepikhov
On 3/22/25 06:48, Michael Paquier wrote: On Fri, Mar 21, 2025 at 09:25:24AM -0400, Sami Imseih wrote: planner() is the sole place in the core code where the planner hook can be called. Shouldn't we have at least a call to pgstat_report_plan_id() after planning a query? At least that should be

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Tom Lane
Michael Paquier writes: > Alias.aliasname is not qualified, so it means that we'd begin to > assign the same query ID even if using two relations from two schemas > depending on what search_path assigns, no? Right. I'm arguing that that's good. The proposed patch already obscures the difference

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-22 Thread Michael Paquier
On Sat, Mar 22, 2025 at 12:24:43PM -0400, Tom Lane wrote: > I experimented with this trivial fix (shown in-line to keep the cfbot > from thinking this is the patch-of-record): > > What's happening there is that there's an ALTER TABLE ADD COLUMN in > the test, so the executions after the first one

Re: AIO v2.5

2025-03-22 Thread Noah Misch
On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > Attached v2.11 > Subject: [PATCH v2.11 05/27] aio: Add io_method=io_uring Apart from some isolated cosmetic points, this is ready to commit: > + ereport(ERROR, > + errcode(err

Re: Test to dump and restore objects left behind by regression

2025-03-22 Thread Ashutosh Bapat
On Fri, Mar 21, 2025 at 6:04 PM Alvaro Herrera wrote: > > On 2025-Mar-21, Ashutosh Bapat wrote: > > > On Thu, Mar 20, 2025 at 8:37 PM vignesh C wrote: > > > > Should the copyright be only 2025 in this case: > > > The patch was posted in 2024 to this mailing list. So we better > > protect the copy