Re: hash_search_with_hash_value is high in "perf top" on a replica

2025-02-10 Thread Jakub Wartak
Hi Thomas! On Tue, Feb 4, 2025 at 10:22 PM Thomas Munro wrote: > > On Sun, Feb 2, 2025 at 3:44 AM Ants Aasma wrote: > > The other direction is to split off WAL decoding, buffer lookup and maybe > > even pinning to a separate process from the main redo loop. > > Hi Ants, > [..] > An assumption I

Re: Fix outdated code comments in nodeAgg.c

2025-02-10 Thread Richard Guo
On Sun, Feb 9, 2025 at 11:15 PM Zhang Mingli wrote: > By reading the codes in nodeAgg.c, I found that some comments are outdated. > The comments referred to the function lookup_hash_entry(), which has been > removed in commit b563594. > I've adjusted the comments to refer to the correct function

Re: Sample rate added to pg_stat_statements

2025-02-10 Thread Ilia Evdokimov
Hi hackers, Since current patch is in the commitfest with the status 'Ready for committer', I’d like to summarize what it does, the problems it addresses, and answer the key questions raised in the discussion thread. Enabling pg_stat_statements can cause a performance drop due to two main re

Re: branch-free tuplesort partitioning

2025-02-10 Thread John Naylor
with only two distinct steps: https://www.postgresql.org/message-id/PH7P220MB1533DA211DF219996760CBB7D9EB2%40PH7P220MB1533.NAMP220.PROD.OUTLOOK.COM -- John Naylor Amazon Web Services branchless-lomuto-20250210.ods Description: application/vnd.oasis.opendocument.spreadsheet

Re: Adjust tuples estimate for appendrels

2025-02-10 Thread Richard Guo
Thanks for all the reviews. Attached is an updated patch that resolves the review comments. Thanks Richard v2-0001-Adjust-tuples-estimate-for-appendrels.patch Description: Binary data

Re: outdated comment in table_tuple_update definition

2025-02-10 Thread Ashutosh Bapat
On Fri, Feb 7, 2025 at 3:08 AM Daniel Gustafsson wrote: > > > On 6 Feb 2025, at 20:00, Sergei Kornilov wrote: > > > I found that the parameter type was changed by commit 19d8e23 (Ignore BRIN > > indexes when checking for HOT updates), but the comment was not updated. > > Looks like oversight. >

Re: Showing applied extended statistics in explain Part 2

2025-02-10 Thread Andrei Lepikhov
On 8/2/2025 20:50, Tomas Vondra wrote: On 1/24/25 11:17, Andrei Lepikhov wrote: On 11/1/24 12:22, Tatsuro Yamada wrote: I often use SQL Server to compare execution plans generated by PostgreSQL, and I appreciate how they display the usage of extended statistics. They clearly identify which sta

Re: Conflict detection for update_deleted in logical replication

2025-02-10 Thread Amit Kapila
On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote: > > On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote: > > > > > > I'm not really sure that these behaviors are the expected behavior of > > > users who set max_conflict_retention_duration to some subscriptions. > > > Or I might have set the wro

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-10 Thread vignesh C
On Tue, 4 Feb 2025 at 18:31, vignesh C wrote: > > On Thu, 30 Jan 2025 at 17:32, Shlok Kyal wrote: > > > > @@ -1428,6 +1427,12 @@ check_foreign_tables_in_schema(Oid schemaid) > > errdetail("foreign table \"%s\" is a partition of > > partitioned table \"%s\"", > >

Re: should we have a fast-path planning for OLTP starjoins?

2025-02-10 Thread Tomas Vondra
On 2/10/25 08:29, Richard Guo wrote: > On Mon, Feb 10, 2025 at 9:32 AM Tomas Vondra wrote: >> E.g. imagine we have a join of 8 relations, with F (fact), dimensions D1 >> and D2, and then some artibrary tables T1, T2, T3, T4, T5. And let's say >> deconstruct_recurse() sees them in this particula

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Amit Kapila
On Mon, Feb 10, 2025 at 11:33 AM Peter Smith wrote: > > Some review comments for v72-0001. > > == > GENERAL > > My preference was to just keep the enum as per v70 for the *actual* > cause, and introduce a separate set of bit flags for *possible* causes > to be checked. This creates a clear cod

RE: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Zhijie Hou (Fujitsu)
On Monday, February 10, 2025 2:10 PM Amit Kapila wrote: > > On Sat, Feb 8, 2025 at 12:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > 3. > > > > + if (cause & RS_INVAL_HORIZON) > > + { > > + if (!SlotIsLogical(s)) > > +

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Nisha Moond
On Mon, Feb 10, 2025 at 6:12 PM vignesh C wrote: > > On Mon, 10 Feb 2025 at 17:33, Nisha Moond wrote: > > > > Here are the v73 patches incorporating the comments above and the > > subsequent comments from [1]. > > - patch 002 is rebased on 001 with no new changes. > > Few comments: > 1) For some

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-10 Thread Andres Freund
Hi, On 2025-02-09 21:06:02 +0100, Jelte Fennema-Nio wrote: > On Sun, 9 Feb 2025 at 19:05, Tom Lane wrote: > > > > Andres Freund writes: > > > I'm somewhat against this patch, as it's fairly fundamentally incompatible > > > with AIO. There's no real way to get information in this manner if the IO

support virtual generated column not null constraint

2025-02-10 Thread jian he
hi. Virtual generated columns committed, https://git.postgresql.org/cgit/postgresql.git/commit/?id=83ea6c54025bea67bcd4949a6d58d3fc11c3e21b This patch is for implementing not null constraints on virtual generated columns. NOT NULL constraints on virtual generated columns mean that if we INSERT a

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-10 Thread Shubham Khanna
On Mon, Feb 10, 2025 at 7:05 AM Peter Smith wrote: > > Some review comments for v4-0001. > > == > Commit message > > 1. > This patch enhances the 'pg_createsubscriber' utility by adding the > '--all-databases' option, which automatically fetches all non-template > databases on the source serve

Re: explain analyze rows=%.0f

2025-02-10 Thread Matheus Alcantara
Thanks for the new patch version. -- v7-0001-Clarify-display-of-rows-and-loops-as-decimal-fraction.patch > +if (nloops > 1 && planstate->instrument->ntuples < nloops) > +appendStringInfo(es->str," rows=%.2f loops=%.2f)", rows, > nloops); > Sorry, but why do the

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-10 Thread Shubham Khanna
On Mon, Feb 10, 2025 at 6:58 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > > Fixed the given comments. The attached patch at [1] contains the > > suggested changes. > > Thanks for updates. I registered the thread to the commitfest entry [1]. > Few comments. > > 01. fetch_source_databases

Re: Doc: Move standalone backup section, mention -X argument

2025-02-10 Thread Álvaro Herrera
On 2024-Jun-28, David G. Johnston wrote: > A documentation comment came in [1] causing me to review some of our backup > documentation and I left the current content and location of the standalone > backups was odd. I propose to move it to a better place, under file system > backups. Even before

Re: Expanding HOT updates for expression and partial indexes

2025-02-10 Thread Burd, Greg
On Feb 9, 2025, at 1:14 AM, Laurenz Albe wrote: > > I think that the goal of this patch is interesting and desirable. Thanks for taking a look at it. Which version did you prefer, v3 or v4? > The greatest concern for me is the performance impact. Agreed, I’m still looking for ways to minimize

Re: Enhancing Memory Context Statistics Reporting

2025-02-10 Thread torikoshia
On 2025-02-03 21:47, Rahila Syed wrote: Hi, Just idea; as an another option, how about blocking new requests to the target process (e.g., causing them to fail with an error or returning NULL with a warning) if a previous request is still pending? Users can simply retry the request if it

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Nisha Moond
On Sat, Feb 8, 2025 at 12:28 PM Zhijie Hou (Fujitsu) wrote: > > On Friday, February 7, 2025 9:06 PM Nisha Moond > wrote: > > > > Attached v72 patches, addressed the above comments as well as Vignesh's > > comments in [2]. > > - There are no new changes in patch-002. > > Thanks for updating the

Re: Fix assert failure when decoding XLOG_PARAMETER_CHANGE on primary

2025-02-10 Thread Amit Kapila
On Fri, Feb 7, 2025 at 11:30 PM Masahiko Sawada wrote: > > On Thu, Feb 6, 2025 at 9:30 PM Amit Kapila wrote: > > > > True but it sounds like there is more harm than benefit. It seems > > reasonable to do this on HEAD. Shall we think of doing it differently > > in HEAD and back-branches or let's r

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-10 Thread torikoshia
On 2025-02-10 05:06, Jelte Fennema-Nio wrote: Thanks for reviewing the patch and comments! Fixed issues you pointed out and attached v2 patch. On Sun, 9 Feb 2025 at 19:05, Tom Lane wrote: Andres Freund writes: > I'm somewhat against this patch, as it's fairly fundamentally incompatible > wi

Fix 035_standby_logical_decoding.pl race conditions

2025-02-10 Thread Bertrand Drouvot
Hi hackers, Please find attached a patch to $SUBJECT. In rare circumstances (and on slow machines) it is possible that a xl_running_xacts is emitted and that the catalog_xmin of a logical slot on the standby advances past the conflict point. In that case, no conflict is reported and the test fai

Re: Doc: Move standalone backup section, mention -X argument

2025-02-10 Thread David G. Johnston
On Monday, February 10, 2025, Álvaro Herrera wrote: > On 2024-Jun-28, David G. Johnston wrote: > > > A documentation comment came in [1] causing me to review some of our > backup > > documentation and I left the current content and location of the > standalone > > backups was odd. I propose to m

Re: Showing applied extended statistics in explain Part 2

2025-02-10 Thread Tomas Vondra
On 2/10/25 10:09, Andrei Lepikhov wrote: > On 8/2/2025 20:50, Tomas Vondra wrote: >> >> >> On 1/24/25 11:17, Andrei Lepikhov wrote: >>> On 11/1/24 12:22, Tatsuro Yamada wrote: >>> I often use SQL Server to compare execution plans generated by >>> PostgreSQL, and I appreciate how they display the

Re: describe special values in GUC descriptions more consistently

2025-02-10 Thread Nathan Bossart
On Mon, Feb 10, 2025 at 09:13:26AM +1100, Peter Smith wrote: > +1 for improving consistency. Thanks for reviewing. > 1. IMO all places wording as "XXX means to YYY" should be just "XXX > means YYY" (e.g. remove the "to") > > e.g. "-1 means to wait forever." => "-1 means wait forever." > e.g. ""-

Re: describe special values in GUC descriptions more consistently

2025-02-10 Thread David G. Johnston
On Mon, Feb 10, 2025 at 9:02 AM Nathan Bossart wrote: > On Mon, Feb 10, 2025 at 09:13:26AM +1100, Peter Smith wrote: > > +1 for improving consistency. > > Thanks for reviewing. > > > 1. IMO all places wording as "XXX means to YYY" should be just "XXX > > means YYY" (e.g. remove the "to") > > > >

Re: explain analyze rows=%.0f

2025-02-10 Thread Ilia Evdokimov
On 10.02.2025 18:32, Matheus Alcantara wrote: Thanks for the new patch version. -- v7-0001-Clarify-display-of-rows-and-loops-as-decimal-fraction.patch +if (nloops > 1 && planstate->instrument->ntuples < nloops) +appendStringInfo(es->str," rows=%.2f loops=%.

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-10 Thread vignesh C
On Mon, 10 Feb 2025 at 20:36, Shubham Khanna wrote: > > The attached patch contains the suggested changes. If a new database is created on the primary server while pg_createsubscriber is running, the subscription will not be created for the new database. To reproduce this issue, follow these step

Re: WAL-logging facility for pgstats kinds

2025-02-10 Thread Andres Freund
Hi, On 2025-01-14 12:54:36 +0900, Michael Paquier wrote: > On Fri, Jan 10, 2025 at 01:46:53PM +0900, Michael Paquier wrote: > > I'd rather use RecoveryInProgress() here even if XLogInsertAllowed() > > is a synonym of that, minus the update of LocalXLogInsertAllowed for > > the local process. > >

Re: Remove useless casts to (char *)

2025-02-10 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > On 06.02.25 12:49, Dagfinn Ilmari Mannsåker wrote: >> I have only skimmed the patches, but one hunk jumped out at me: >> Peter Eisentraut writes: >> >>> diff --git a/src/backend/libpq/pqcomm.c b/src/backend/libpq/pqcomm.c >>> index 1bf27d93cfa..937a2b02a4f 100644 >>>

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-02-10 Thread Jacob Champion
On Thu, Feb 6, 2025 at 12:35 AM Michael Paquier wrote: > /* Update app name to current GUC setting */ > + /* TODO: ask the list: maybe do this before setting STATE_UNDEFINED? > */ > if (application_name) > pgstat_report_appname(application_name); > > Historic

Small memory fixes for pg_createsubcriber

2025-02-10 Thread Ranier Vilela
Hi. Per Coverity. Coverity has some reports about pg_createsubcriber. CID 1591322: (#1 of 1): Resource leak (RESOURCE_LEAK) 10. leaked_storage: Variable dbname going out of scope leaks the storage it points to. Additionally there are several calls that are out of the ordinary, according to the

Re: Expanding HOT updates for expression and partial indexes

2025-02-10 Thread Matthias van de Meent
On Thu, 6 Feb 2025 at 23:24, Burd, Greg wrote: > > Attached find a patch that expands the cases where heap-only tuple (HOT) > updates are possible without changing the basic semantics of HOT. This is > accomplished by examining expression indexes for changes to determine if > indexes require up

Re: Support NOT VALID / VALIDATE constraint options for named NOT NULL constraints

2025-02-10 Thread Rushabh Lathia
Hi Alvaro, I have incorporated the suggested changes, and here is the latest version of the patch: - Added more test cases to the regression suite. - Included tests in the pg_dump test. - Left objects with *INVALID NOT NULL* for pg_upgrade. - Fixed an issue where recursion to child ta

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Melanie Plageman
On Sun, Feb 9, 2025 at 9:27 AM Tomas Vondra wrote: > > > 2) ryzen > > > This "new" machine has multiple types of storage. The cached results (be > it in shared buffers or in page cache) are not very interesting. 0003 > helps a bit (~15%), but other than that it's just random noise. > > Th

Re: Fix punctuation errors in PostgreSQL documentation

2025-02-10 Thread John Naylor
On Mon, Feb 10, 2025 at 8:18 AM David G. Johnston wrote: > > On Saturday, February 8, 2025, 斉藤登 wrote: >> >> >> I found some issues in the original English documentation that I >> believe need to be corrected. > > > I concur on all of them. Thanks for the patch! I will push this after the upcomi

Re: Conflict detection for update_deleted in logical replication

2025-02-10 Thread Dilip Kumar
On Mon, Feb 10, 2025 at 2:45 PM Amit Kapila wrote: > > On Mon, Feb 10, 2025 at 10:26 AM Dilip Kumar wrote: > > > > On Fri, Feb 7, 2025 at 11:17 AM Amit Kapila wrote: > > > > > > > > > I'm not really sure that these behaviors are the expected behavior of > > > > users who set max_conflict_retenti

Re: [RFC] Lock-free XLog Reservation from WAL

2025-02-10 Thread Zhou, Zhiguo
On 2/5/2025 4:32 PM, Japin Li wrote: On Mon, 27 Jan 2025 at 17:30, "Zhou, Zhiguo" wrote: On 1/26/2025 10:59 PM, Yura Sokolov wrote: 24.01.2025 12:07, Japin Li пишет: On Thu, 23 Jan 2025 at 21:44, Japin Li wrote: On Thu, 23 Jan 2025 at 15:03, Yura Sokolov wrote: 23.01.2025 11:46, Japin

Re: Separate memory contexts for relcache and catcache

2025-02-10 Thread Melih Mutlu
Hi, I rebased and updated the patch to address some concerns raised before and see if anyone is still interested in this. I believe that there is a general consensus around RelCacheContext and CatCacheContext, considering that these two caches are fairly used. For the rest, I followed Ashutosh's

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread vignesh C
On Mon, 10 Feb 2025 at 17:33, Nisha Moond wrote: > > Here are the v73 patches incorporating the comments above and the > subsequent comments from [1]. > - patch 002 is rebased on 001 with no new changes. Few comments: 1) For some reason SlotInvalidationCauses was with PGDLLIMPORT, this is remove

Re: BF member drongo doesn't like 035_standby_logical_decoding.pl

2025-02-10 Thread Bertrand Drouvot
Hi, On Mon, Jan 27, 2025 at 07:13:01AM +, Bertrand Drouvot wrote: > On Fri, Jan 24, 2025 at 02:44:21PM -0500, Andres Freund wrote: > > If we could trigger VACUUM in a transaction on the primary this would be > > easy, but we can't. > > Another idea that I had ([1]) was to make use of injecti

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Tomas Vondra
On 2/10/25 19:02, Melanie Plageman wrote: > On Sun, Feb 9, 2025 at 9:27 AM Tomas Vondra wrote: >> >> >> 2) ryzen >> >> >> This "new" machine has multiple types of storage. The cached results (be >> it in shared buffers or in page cache) are not very interesting. 0003 >> helps a bit (~15%)

Re: Expanding HOT updates for expression and partial indexes

2025-02-10 Thread Burd, Greg
Apologies for not being clear, this preserves the current behavior for summarizing indexes allowing for HOT updates while also updating the index. No degradation here that I’m aware of, indeed the tests that ensure that behavior are unchanged and pass. -greg > On Feb 10, 2025, at 12:17 PM, Ma

Re: NOT ENFORCED constraint feature

2025-02-10 Thread Álvaro Herrera
On Mon, Feb 10, 2025, at 7:03 AM, Amul Sul wrote: > Attached patch implemented this behaviour. To achieve this, we have to > revert (see 0007) some committed code and relax the restriction that > the NOT ENFORCED constraint must also be NOT VALID. Now, NOT ENFORCED > and NOT VALID are independent s

Re: describe special values in GUC descriptions more consistently

2025-02-10 Thread Peter Smith
On Tue, Feb 11, 2025 at 3:22 AM David G. Johnston wrote: > > On Mon, Feb 10, 2025 at 9:02 AM Nathan Bossart > wrote: >> >> On Mon, Feb 10, 2025 at 09:13:26AM +1100, Peter Smith wrote: >> > +1 for improving consistency. >> >> Thanks for reviewing. >> >> > 1. IMO all places wording as "XXX means t

Re: should we have a fast-path planning for OLTP starjoins?

2025-02-10 Thread Robert Haas
On Fri, Feb 7, 2025 at 3:09 PM Tomas Vondra wrote: > I don't think that's quite true. The order of dimension joins does not > matter because the joins do not affect the join size at all. The size of > |F| has nothing to do with that, I think. We'll do the same number of > lookups against the dimen

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Robert Haas
On Mon, Feb 10, 2025 at 1:11 PM Tomas Vondra wrote: > Certainly for the "localized" regressions, and cases when bitmapheapscan > would not be picked. The eic=1 case makes me a bit more nervous, because > it's default and affects NVMe storage. Would be good to know why is > that, or perhaps conside

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Melanie Plageman
On Mon, Feb 10, 2025 at 4:24 PM Robert Haas wrote: > > On Mon, Feb 10, 2025 at 1:11 PM Tomas Vondra wrote: > > Certainly for the "localized" regressions, and cases when bitmapheapscan > > would not be picked. The eic=1 case makes me a bit more nervous, because > > it's default and affects NVMe st

Re: Virtual generated columns

2025-02-10 Thread Richard Guo
On Mon, Feb 10, 2025 at 1:16 PM Zhang Mingli wrote: > I believe virtual columns should behave like stored columns, except they > don't actually use storage. > Virtual columns are computed when the table is read, and they should adhere > to the same rules of join semantics. > I agree with Richard

Re: Statistics Import and Export

2025-02-10 Thread Jeff Davis
On Sun, 2025-02-09 at 22:00 -0500, Corey Huinker wrote: > > 0001 - I've added pg_locks tests for a regular index and a > partitioned index. Committed 0001, the fix for importing stats. Regards, Jeff Davis

Re: Inconsistency between Compression and Storage for Foreign Tables

2025-02-10 Thread Sami Imseih
> This form sets the storage mode for a column. See the similar form of ALTER > TABLE for more details. > Note that the storage mode has no effect unless the table's foreign-data > wrapper chooses to pay attention to it. Hi, It looks like cb1ca4d [1], from nearly 10 years ago, allowed storage o

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-10 Thread Andres Freund
Hi, On 2025-02-10 18:30:56 -0500, Andres Freund wrote: > On 2025-02-10 23:52:17 +0100, Jelte Fennema-Nio wrote: > > On Mon, 10 Feb 2025 at 14:31, Andres Freund wrote: > > > But this will also not work with AIO w/ Buffered IO. Which we hope to use > > > much > > > more commonly. > > > > To be cle

Re: describe special values in GUC descriptions more consistently

2025-02-10 Thread Peter Smith
On Tue, Feb 11, 2025 at 9:25 AM Nathan Bossart wrote: > > On Tue, Feb 11, 2025 at 08:29:28AM +1100, Peter Smith wrote: > > +1 for this. Your wording examples below look good to me.. > > Okay, how does this look? > > -- v2 mostly looked good to me. Just a couple of questions. ~~~ 1. {"shared_m

Re: explain analyze rows=%.0f

2025-02-10 Thread Ilia Evdokimov
On 10.02.2025 23:43, Matheus Alcantara wrote: When the total number of returned tuples is less than the number of loops currently shows 'rows = 0'. This can mislead users into thinking that no rows were returned at all, even though some might have appeared occasionally. I think that this can h

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Melanie Plageman
On Mon, Feb 10, 2025 at 1:02 PM Melanie Plageman wrote: > > It'll be hard to look into all of these, so I think I'll focus on > trying to reproduce something with eic=1 that I can reproduce on my > machine. So far, I can reproduce a regression with the following and > the data file attached. > > #

Re: Statistics Import and Export

2025-02-10 Thread Jeff Davis
On Sun, 2025-02-09 at 22:00 -0500, Corey Huinker wrote: > > 0002 - I've done some documentation rewording, mostly wording changes > where behaviors surrounding data-only dumps are actually meant for > any dump that has all schema excluded. Comments on v45-0002: * Why is generate_old_dump() passi

Re: explain analyze rows=%.0f

2025-02-10 Thread Matheus Alcantara
Em seg., 10 de fev. de 2025 às 13:38, Ilia Evdokimov escreveu: > > -- v7-0001-Clarify-display-of-rows-and-loops-as-decimal-fraction.patch > >> +if (nloops > 1 && planstate->instrument->ntuples < nloops) > >> +appendStringInfo(es->str," rows=%.2f loops=%.2f)", >

Re: Track the amount of time waiting due to cost_delay

2025-02-10 Thread Nathan Bossart
On Tue, Feb 04, 2025 at 10:14:48AM +, Bertrand Drouvot wrote: > On Mon, Feb 03, 2025 at 02:05:51PM -0600, Nathan Bossart wrote: >> Barring objections, I am planning to commit this one soon. I might move >> the addition of analyze_delay_point() to its own patch, but otherwise I >> think it look

Re: pull-up subquery if JOIN-ON contains refs to upper-query

2025-02-10 Thread Ilia Evdokimov
On 09.02.2025 18:14, Alena Rybakina wrote: Hi! I found another example where the transformation worked incorrectly and reconsidered the idea. As for conversion of exists_sublink_to_ANY, we need to get the flattened implicit-AND list of clauses and pull out the chunks of the WHERE clause that

Re: Improving the latch handling between logical replication launcher and worker processes.

2025-02-10 Thread Alexander Lakhin
Hello, 04.09.2024 16:53, Heikki Linnakangas wrote: On 04/09/2024 14:24, vignesh C wrote: I agree that this approach is more simple than the other approach. How about something like the attached patch to handle the same. I haven't looked at these new patches from the last few days, but please

Re: describe special values in GUC descriptions more consistently

2025-02-10 Thread Nathan Bossart
On Tue, Feb 11, 2025 at 08:29:28AM +1100, Peter Smith wrote: > +1 for this. Your wording examples below look good to me.. Okay, how does this look? -- nathan >From 25f62fa18e5e6ffd0fbd48e9f9bdd5474caaf23c Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Fri, 7 Feb 2025 16:06:18 -0600 Subject

Re: Proposal: "query_work_mem" GUC, to distribute working memory to the query's individual operators

2025-02-10 Thread James Hunter
On Fri, Jan 24, 2025 at 5:48 PM Jeff Davis wrote: > > On Fri, 2025-01-24 at 17:04 -0800, James Hunter wrote: > > Generating "high memory" vs. "low memory" paths would be tricky, > > because the definition of "high" vs. "low" depends on the entire path > > tree, not just on a single path node. So I

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-02-10 Thread Jacob Champion
On Mon, Feb 10, 2025 at 8:23 AM Jacob Champion wrote: > The test is supposed to enforce that, but I see that it's not for some > reason. That's concerning. I'll investigate, thanks for pointing it > out. Bad regex escaping on my part; fixed in v8. Thanks for the report! While debugging, I also n

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-10 Thread Sami Imseih
> This fixes the long comments in plannodes.h to make it easier to add the > attribute annotation. It made the most sense to make this the first patch > in the set. > A commit that happened last Friday made also this to have conflict. Thanks for committing v5-0001. I am not sure why there is comm

Re: 2025-02-13 release announcement draft

2025-02-10 Thread Erik Rijkers
Op 2/11/25 om 01:09 schreef Jonathan S. Katz: Hi, Attached is a draft of the 2025-02-13 update release announcement. Please provide feedback no later than 2025-02-13 0:00 UTC. worst cas worst case Erik

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-10 Thread Peter Smith
Hi Shubham. Responding with a blanket statement "Fixed the given comments" makes it difficult to be sure what was done when I come to confirm the changes. Often embedded questions go unanswered, and if changes are *not* made, then I can't tell if there was a good reason for rejection or was the co

Re: speedup COPY TO for partitioned table.

2025-02-10 Thread Melih Mutlu
Hi, jian he , 27 Oca 2025 Pzt, 04:47 tarihinde şunu yazdı: > in the > > find_all_inheritors->find_inheritance_children->find_inheritance_children_extended > > find_inheritance_children_extended we have > """ > if (numoids > 1) > qsort(oidarr, numoids, sizeof(Oid), oid_cmp); > """ > >

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-02-10 Thread Robert Haas
On Wed, Feb 5, 2025 at 12:23 PM Melanie Plageman wrote: > I started getting worried thinking about this. If you have a cursor > for select * from a table and fetch forward far enough, couldn't > vacuum fail to get cleanup locks on a whole range of blocks? I don't think so. A given scan holds at m

Re: NOT ENFORCED constraint feature

2025-02-10 Thread Isaac Morland
On Mon, 10 Feb 2025 at 13:48, Álvaro Herrera wrote: I think this proposed state of affairs is problematic. Current queries > that assume that pg_constraint.convalidated means that a constraint is > validated would be broken. My suggestion at this point is that instead of > adding a separate boo

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-10 Thread Sami Imseih
Another thought that I have is that If we mention that extensions can use these jumbling ( or whatever the final name is ) functions outside of core, it makes sense to actually show an example of this. What do you think? -- Sami

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Melanie Plageman
On Mon, Feb 10, 2025 at 4:22 PM Melanie Plageman wrote: > > I don't really know what to do about this. The behavior of master > parallel bitmap heap scan can be emulated with the patch by increasing > effective_io_concurrency. But, IIRC we didn't want to do that for some > reason? > Not only does

Re: AIO v2.3

2025-02-10 Thread Thomas Munro
On Thu, Jan 23, 2025 at 5:29 PM Andres Freund wrote: +/* caller will issue more io, don't submit */ +#define READ_BUFFERS_MORE_MORE_MORE (1 << 3) > - Heikki doesn't love pgaio_submit_staged(), suggested pgaio_kick_staged() or > such. I don't love that name though. Problem statement: You want to

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-10 Thread Michael Paquier
On Mon, Feb 10, 2025 at 02:02:10PM -0600, Sami Imseih wrote: > Thanks for committing v5-0001. I am not sure why there is comment > that is not correctly indented. Attached is a fix for that Thanks, fixed. The reason behind that is likely that I have fat fingers. -- Michael signature.asc Descrip

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-02-10 Thread Andres Freund
Hi, On 2025-02-10 16:24:25 -0500, Robert Haas wrote: > On Mon, Feb 10, 2025 at 1:11 PM Tomas Vondra wrote: > > Certainly for the "localized" regressions, and cases when bitmapheapscan > > would not be picked. The eic=1 case makes me a bit more nervous, because > > it's default and affects NVMe st

2025-02-13 release announcement draft

2025-02-10 Thread Jonathan S. Katz
Hi, Attached is a draft of the 2025-02-13 update release announcement. Please provide feedback no later than 2025-02-13 0:00 UTC. Thanks, Jonathan The PostgreSQL Global Development Group has released an update to all supported versions of PostgreSQL, including 17.3, 16.7, 15.11, 14.16, and 13

Allow io_combine_limit up to 1MB

2025-02-10 Thread Thomas Munro
Hi, Tomas queried[1] the limit of 256kB (or really 32 blocks) for io_combine_limit. Yeah, I think we should increase it and allow experimentation with larger numbers. Note that real hardware and protocols have segment and size limits that can force the kernel to split your I/Os, so it's not at a

Re: Proposal to CREATE FOREIGN TABLE LIKE

2025-02-10 Thread Sami Imseih
> Patch V2 addressed the comments. Overall this LGTM. I still see a "no real storage" in v2 that should be removed from the documentation. + Foreign tables have no real storage in PostgreSQL. + Inapplicable options: INCLUDING INDEXES, INCLUDING STORAGE, I think the test coverage to check for th

RE: Improve CRC32C performance on SSE4.2

2025-02-10 Thread Devulapalli, Raghuveer
Hi John, > I'm highly suspicious of these numbers because they show this version > is about 20x faster than "scalar", so relatively speaking 3x faster > than the AVX-512 proposal? Apologies for this. I was suspicious of this too and looks like I had unintentionally set the scalar version I wrote

Re: describe special values in GUC descriptions more consistently

2025-02-10 Thread David G. Johnston
On Mon, Feb 10, 2025 at 4:53 PM Peter Smith wrote: > On Tue, Feb 11, 2025 at 9:25 AM Nathan Bossart > wrote: > > > > On Tue, Feb 11, 2025 at 08:29:28AM +1100, Peter Smith wrote: > > > +1 for this. Your wording examples below look good to me.. > > > > Okay, how does this look? > > > > -- > > v2 m

Re: speedup COPY TO for partitioned table.

2025-02-10 Thread David Rowley
On Tue, 11 Feb 2025 at 08:10, Melih Mutlu wrote: > jian he , 27 Oca 2025 Pzt, 04:47 tarihinde şunu > yazdı: >> so the find_all_inheritors output order is deterministic? > > You're right that order in find_all_inheritors is deterministic. But it's not > always the same with the order of SELECT ou

Re: 2025-02-13 release announcement draft

2025-02-10 Thread Josef Šimánek
út 11. 2. 2025 v 1:10 odesílatel Jonathan S. Katz napsal: > > Hi, > > Attached is a draft of the 2025-02-13 update release announcement. > Please provide feedback no later than 2025-02-13 0:00 UTC. tables and indexs that prevent potential corruption tables and indexes that prevent potential corru

Re: 2025-02-13 release announcement draft

2025-02-10 Thread David Rowley
On Tue, 11 Feb 2025 at 13:10, Jonathan S. Katz wrote: > Attached is a draft of the 2025-02-13 update release announcement. > Please provide feedback no later than 2025-02-13 0:00 UTC. "indexs" -> "indexes" Should psql have `'s? David

Re: 2025-02-13 release announcement draft

2025-02-10 Thread Josef Šimánek
út 11. 2. 2025 v 1:10 odesílatel Jonathan S. Katz napsal: > > Hi, > > Attached is a draft of the 2025-02-13 update release announcement. > Please provide feedback no later than 2025-02-13 0:00 UTC. noncompatible -> incompatible > Thanks, > > Jonathan

Re: AIO v2.3

2025-02-10 Thread Andres Freund
Hi, On 2025-02-06 11:50:04 +0100, Jakub Wartak wrote: > Hi Andres, OK, so I've hastily launched AIO v2.3 (full, 29 patches) > patchset probe run before going for short vacations and here results > are attached*. Thanks for doing that work! > TLDR; in terms of SELECTs the master vs aioworkers lo

Re: dblink: Add SCRAM pass-through authentication

2025-02-10 Thread Jacob Champion
On Wed, Jan 22, 2025 at 6:10 AM Matheus Alcantara wrote: > The attached patch enables SCRAM authentication for dblink connections when > using dblink_fdw without requiring a plain-text password on user mapping > properties. The implementation is very similar to what was implemented on > postgres_f

Re: Expanding HOT updates for expression and partial indexes

2025-02-10 Thread Nathan Bossart
On Mon, Feb 10, 2025 at 06:17:42PM +0100, Matthias van de Meent wrote: > I have serious doubts about the viability of any proposal working to > implement PHOT/WARM in PostgreSQL, as they seem to have an inherent > nature of fundamentally breaking the TID lifecycle: > We won't be able to clean up de

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-10 Thread Andres Freund
Hi, On 2025-02-10 23:52:17 +0100, Jelte Fennema-Nio wrote: > On Mon, 10 Feb 2025 at 14:31, Andres Freund wrote: > > But this will also not work with AIO w/ Buffered IO. Which we hope to use > > much > > more commonly. > > To be clear, here you mean worker based AIO right? Because it would > wor

Re: Proposal: "query_work_mem" GUC, to distribute working memory to the query's individual operators

2025-02-10 Thread James Hunter
I hope to have an initial patch-set for a prototype, within the next couple of weeks. But I wanted to add some design comments to this thread, first, to solicit feedback, etc. — First, some bookkeeping: Peter Geoghegan pointed me, offline, to Oracle’s 2002 paper [1] on how they managed SQL executi

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Peter Smith
Hi Nisha. Some review comments about v74-0001 == src/backend/replication/slot.c 1. /* Maximum number of invalidation causes */ -#define RS_INVAL_MAX_CAUSES RS_INVAL_WAL_LEVEL - -StaticAssertDecl(lengthof(SlotInvalidationCauses) == (RS_INVAL_MAX_CAUSES + 1), - "array length mismatch"); +#def

Re: read stream on amcheck

2025-02-10 Thread jian he
On Fri, Jan 3, 2025 at 1:53 AM Matheus Alcantara wrote: > > > Yeap, I agree. Attached a v2 fixed. > hi. some minor issue i found. +#include "storage/block.h" no need, since "#include "storage/bufmgr.h" already included it. do we need to add ``CHECK_FOR_INTERRUPTS()`` in heapam_read_stream_next_

Re: Test to dump and restore objects left behind by regression

2025-02-10 Thread Ashutosh Bapat
On Thu, Feb 6, 2025 at 11:32 AM Michael Paquier wrote: > > On Wed, Feb 05, 2025 at 03:28:04PM +0900, Michael Paquier wrote: > > Hmm. I was reading through the patch and there is something that > > clearly stands out IMO: the new compare_dumps(). It is in Utils.pm, > > and it acts as a wrapper of

Re: 2025-02-13 release announcement draft

2025-02-10 Thread Ilia Evdokimov
On 11.02.2025 03:09, Jonathan S. Katz wrote: Hi, Attached is a draft of the 2025-02-13 update release announcement. Please provide feedback no later than 2025-02-13 0:00 UTC. Thanks, Jonathan you may simply shutdown PostgreSQL you may simply shut down PostgreSQL -- Best regards, Ilia E

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Amit Kapila
On Tue, Feb 11, 2025 at 8:49 AM Peter Smith wrote: > > > InvalidatePossiblyObsoleteSlot: > > 2. > + if (possible_causes & RS_INVAL_IDLE_TIMEOUT) > + { > + /* > + * Assign the current time here to avoid system call overhead > + * while holding the spinlock in subsequent code. > + */ > + now = GetCu

Re: Non-text mode for pg_dumpall

2025-02-10 Thread Mahendra Singh Thalor
Thanks Jian. On Tue, 4 Feb 2025 at 07:35, jian he wrote: > > hi. > > just a quick response for v15. > > the pg_restore man page says option --list as "List the table of > contents of the archive". > but > $BIN10/pg_restore --format=directory --list --file=1.sql dir10 > also output the contents of

Re: Improve CRC32C performance on SSE4.2

2025-02-10 Thread John Naylor
On Tue, Feb 11, 2025 at 7:25 AM Devulapalli, Raghuveer wrote: > I ran the same benchmark drive_crc32c with the postgres infrastructure and > found that your v2 sse42 version from corsix is slower than > pg_comp_crc32c_sse42 in master branch when buffer is < 128 bytes. That matches my findings a

RE: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-10 Thread Zhijie Hou (Fujitsu)
On Monday, February 10, 2025 8:03 PM Nisha Moond wrote: > > On Sat, Feb 8, 2025 at 12:28 PM Zhijie Hou (Fujitsu) > wrote: > > > > > 3. > > > > + if (cause & RS_INVAL_HORIZON) > > + { > > + if (!SlotIsLogical(s)) > > +

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-02-10 Thread Shubham Khanna
On Fri, Feb 7, 2025 at 7:46 AM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for updating the patch. > > Previously you told that you had a plan to extend the patch to drop other > replication > objects [1], but I think it is not needed. pg_createsubscriber has already > been > abl

  1   2   >