Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-11 Thread Nisha Moond
On Tue, Feb 11, 2025 at 11:42 AM Zhijie Hou (Fujitsu) wrote: > > On Monday, February 10, 2025 8:03 PM Nisha Moond > wrote: > > > > On Sat, Feb 8, 2025 at 12:28 PM Zhijie Hou (Fujitsu) > > > > wrote: > > > > > > > > 3. > > > > > > + if (cause & RS_INVAL_HORIZON) > > > +

Re: proposal - plpgsql - support standard syntax for named arguments for cursors

2025-02-11 Thread Pavel Stehule
Hi so 8. 2. 2025 v 22:25 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > so 8. 2. 2025 v 20:25 odesílatel Tom Lane napsal: > >> Is there any reason to think that that's actually in the standard? > > > I think the possibility to use named arguments in OPEN statements is a > > PostgreSQL

Fix possible resource leak (src/bin/pg_basebackup/pg_receivewal.c)

2025-02-11 Thread Ranier Vilela
Hi. Per Coverity. CID 1591288: (#1 of 1): Resource leak (RESOURCE_LEAK) 10. leaked_storage: Variable sysidentifier going out of scope leaks the storage it points to. Trivial patch attached. best regards, Ranier Vilela fix-resource-leak-pg_receivewal.patch Description: Binary data

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-11 Thread Nisha Moond
On Tue, Feb 11, 2025 at 8:49 AM Peter Smith wrote: > > Hi Nisha. > > Some review comments about v74-0001 > > == > src/backend/replication/slot.c > > 1. > /* Maximum number of invalidation causes */ > -#define RS_INVAL_MAX_CAUSES RS_INVAL_WAL_LEVEL > - > -StaticAssertDecl(lengthof(SlotInvalida

Re: NOT ENFORCED constraint feature

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-10, Isaac Morland wrote: > I'm having a lot of trouble understanding the operational distinction > between your 'u' and 'U'. If it's not enforced, it cannot be assumed to be > valid, regardless of whether it was valid in the past. I'm not sure what I > think of a single character vs. 2

Re: pull-up subquery if JOIN-ON contains refs to upper-query

2025-02-11 Thread Alena Rybakina
On 10.02.2025 23:51, Ilia Evdokimov wrote: On 09.02.2025 18:14, Alena Rybakina wrote: Hi! I found another example where the transformation worked incorrectly and reconsidered the idea. As for conversion of exists_sublink_to_ANY, we need to get the flattened implicit-AND list of clauses and p

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-11 Thread Jelte Fennema-Nio
On Tue, 11 Feb 2025 at 16:36, Andres Freund wrote: > Shrug. It means that it'll not work in what I hope will be the default > mechanism before long. I just can't get excited for that. In all likelihood > it'll result in bug reports that I'll then be on the hook to fix. My assumption was that io_

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-02-11 Thread Andres Freund
Hi, On 2025-02-10 14:30:15 -0500, Robert Haas wrote: > On Wed, Feb 5, 2025 at 12:23 PM Melanie Plageman > wrote: > > I started getting worried thinking about this. If you have a cursor > > for select * from a table and fetch forward far enough, couldn't > > vacuum fail to get cleanup locks on a w

Re: Proposal to CREATE FOREIGN TABLE LIKE

2025-02-11 Thread Sami Imseih
> + Foreign tables have no real storage in PostgreSQL. > + Inapplicable options: INCLUDING INDEXES, > INCLUDING STORAGE, > > Oh, I corrected another one in the code comments, but I forgot about this one. > Done in patch v3. I attached v4 with some slight modifications to the wording, otherwise thi

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-11 Thread Nathan Bossart
On Tue, Feb 11, 2025 at 03:22:49PM +0100, Álvaro Herrera wrote: > I find this proposed patch a bit strange and I feel it needs more > explanation. > > When this thread started, Bharath justified his patches saying that a > slot that's inactive for a very long time could be problematic because > of

Re: Introduce XID age and inactive timeout based replication slot invalidation

2025-02-11 Thread Álvaro Herrera
Hello, I find this proposed patch a bit strange and I feel it needs more explanation. When this thread started, Bharath justified his patches saying that a slot that's inactive for a very long time could be problematic because of XID wraparound. Fine, that sounds a reasonable feature. If you wa

Re: Move wal_buffers_full to WalUsage (and report it in pgss/explain)

2025-02-11 Thread Ilia Evdokimov
Hi, Thank you for your work! 1. Perhaps In EXPLAIN you forget to check that usage->wal_buffers_full > 0: if ((usage->wal_records > 0) || (usage->wal_fpi > 0) || (usage->wal_bytes > 0)) 2. I have a small suggestion for pg_stat_statements: would it make sense to move wal_buffers_full next to

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 17:00:36 +0100, Jelte Fennema-Nio wrote: > On Tue, 11 Feb 2025 at 16:36, Andres Freund wrote: > > Shrug. It means that it'll not work in what I hope will be the default > > mechanism before long. I just can't get excited for that. In all likelihood > > it'll result in bug repor

Re: Orphaned users in PG16 and above can only be managed by Superusers

2025-02-11 Thread Ashutosh Sharma
Hi Robert, On Tue, Feb 4, 2025 at 10:54 PM Robert Haas wrote: > > On Thu, Jan 30, 2025 at 8:45 AM Ashutosh Sharma wrote: > > Imagine a superuser creates role u1. Since the superuser is creating > > u1, it won't have membership in any role. Now, suppose u1 creates a > > new role, u2. In this case

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-02-11 Thread Shlok Kyal
On Tue, 11 Feb 2025 at 09:51, Shubham Khanna wrote: > > On Fri, Feb 7, 2025 at 7:46 AM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Shubham, > > > > Thanks for updating the patch. > > > > Previously you told that you had a plan to extend the patch to drop other > > replication > > objects [1],

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-02-11 Thread Andres Freund
On 2025-02-05 12:23:29 -0500, Melanie Plageman wrote: > Attached v16 implements the logic to not count pages we failed to > freeze because of cleanup lock contention as eager freeze failures. That looks good to me.

Re: Small memory fixes for pg_createsubcriber

2025-02-11 Thread Euler Taveira
On Mon, Feb 10, 2025, at 1:31 PM, Ranier Vilela wrote: > Coverity has some reports about pg_createsubcriber. > > CID 1591322: (#1 of 1): Resource leak (RESOURCE_LEAK) > 10. leaked_storage: Variable dbname going out of scope leaks the storage it > points to. > Additionally there are several calls

[PATCH] snowball: fix potential NULL dereference

2025-02-11 Thread Коротков Максим
Hi all, I found the case of potential NULL pointer dereference. In snowball/libstemmer/api.c if we transfer control to the SN_create_env() function by using the error label when there is a memory allocation error of z->p or z->S, we can then dereference the NULL pointer z->S in the function SN_cl

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Sami Imseih
I have only looked at 0001, but I am wondering why query_id_const_merge is a pg_stat_statements GUC rather than a core GUC? The dependency of pg_stat_statements to take advantage of this useful feature does not seem right. For example if the user does not have pg_stat_statements enabled, but are

Re: [PATCH] snowball: fix potential NULL dereference

2025-02-11 Thread Tom Lane
=?utf-8?q?=D0=9A=D0=BE=D1=80=D0=BE=D1=82=D0=BA=D0=BE=D0=B2_=D0=9C=D0=B0=D0=BA=D1=81=D0=B8=D0=BC?= writes: > I found the case of potential NULL pointer dereference. > In snowball/libstemmer/api.c if we transfer control to the SN_create_env() > function > by using the error label when there is a m

Re: DOCS - inactive_since field readability

2025-02-11 Thread Amit Kapila
On Fri, Feb 7, 2025 at 12:05 PM Peter Smith wrote: > > > I've had a go at rewording this paragraph in the 0002 patch. > The change in 0001 looks odd after seeing it in HTML format. We should either add one empty line between two paragraphs otherwise it doesn't appear good. Did you see multi-parag

Re: Test to dump and restore objects left behind by regression

2025-02-11 Thread Ashutosh Bapat
Hi Michael, On Sun, Feb 9, 2025 at 1:25 PM Michael Paquier wrote: > > On Fri, Feb 07, 2025 at 07:11:25AM +0900, Michael Paquier wrote: > > Okay, thanks for the feedback. We have been relying on diff -u for > > the parts of the tests touched by 0001 for some time now, so if there > > are no obje

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-11 Thread Shlok Kyal
On Wed, 5 Feb 2025 at 14:14, Álvaro Herrera wrote: > > On 2025-Feb-05, vignesh C wrote: > > > We can maintain the behavior you suggested when the > > PUBLISH_VIA_PARTITION_ROOT option is set to false. However, when > > PUBLISH_VIA_PARTITION_ROOT is true, the table data is copied from the > > root

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-02-11 Thread Shlok Kyal
On Mon, 10 Feb 2025 at 16:11, vignesh C wrote: > > On Tue, 4 Feb 2025 at 18:31, vignesh C wrote: > > > > On Thu, 30 Jan 2025 at 17:32, Shlok Kyal wrote: > > > > > > @@ -1428,6 +1427,12 @@ check_foreign_tables_in_schema(Oid schemaid) > > > errdetail("foreign table \"%s\" is a

Re: Proposal to CREATE FOREIGN TABLE LIKE

2025-02-11 Thread Zhang Mingli
On Feb 11, 2025 at 08:14 +0800, Sami Imseih , wrote: > > Patch V2 addressed the comments. > > > > Overall this LGTM. > > > > I still see a "no real storage" in v2 that should be removed > > from the documentation. > > > > + Foreign tables have no real storage in PostgreSQL. > > + Inapplicable optio

Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Jelte Fennema-Nio
The default open file limit of 1024 is extremely low, given modern resources and kernel architectures. The reason that this hasn't changed change is because doing so would break legacy programs that use the select(2) system call in hard to debug ways. So instead programs that want to opt-in to a hi

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-11, Sami Imseih wrote: > I have only looked at 0001, but I am wondering why > query_id_const_merge is a pg_stat_statements GUC > rather than a core GUC? I was wondering the same thing and found the explanation here: https://postgr.es/m/ztmuctymis3n3...@paquier.xyz > Other extensions

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-11, Dmitry Dolgov wrote: > > On Tue, Feb 11, 2025 at 10:49:59AM GMT, Sami Imseih wrote: > > I have only looked at 0001, but I am wondering why > > query_id_const_merge is a pg_stat_statements GUC > > rather than a core GUC? > > It was moved from being a core GUC into a pg_stat_stateme

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 18:45:13 +0100, Jelte Fennema-Nio wrote: > On Tue, 11 Feb 2025 at 17:19, Andres Freund wrote: > > Yes, at least initially: > > Ah, then I understand your point of view much better. Still I think we > could easily frame it as: If you enable io_uring, you also get these > additio

Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Tom Lane
Jelte Fennema-Nio writes: > The default open file limit of 1024 is extremely low, given modern > resources and kernel architectures. The reason that this hasn't changed > change is because doing so would break legacy programs that use the > select(2) system call in hard to debug ways. So instead p

Re: pgbench with partitioned tables

2025-02-11 Thread Melanie Plageman
On Tue, Feb 11, 2025 at 3:10 AM Sergey Tatarintsev wrote: > > 08.02.2025 17:32, Álvaro Herrera пишет: > > On 2025-Feb-07, Melanie Plageman wrote: > > > >> Okay, I've stared at this a bit, and it seems basically fine the way > >> it is (I might add a bit more whitespace, clean up the commit message

Re: explain analyze rows=%.0f

2025-02-11 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 11, 2025 at 12:14 PM Andrei Lepikhov wrote: >> I support the idea in general, but I believe it should be expanded to >> cover all cases of parameterised plan nodes. Each rescan iteration may >> produce a different number of tuples, and rounding can obscure import

Re: PATCH: Disallow a netmask of zero unless the IP is also all zeroes

2025-02-11 Thread Daniel Gustafsson
> On 11 Feb 2025, at 21:25, Tom Lane wrote: > I'm a bit distressed to realize that hba.c isn't using cidr_in. > Maybe we should try to share code instead of duplicating yet more. +1. I have a note along these lines on my never-shrinking TODO, I think it would be great if we took a stab at that.

Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 21:04:25 +0100, Tomas Vondra wrote: > I agree the defaults may be pretty low for current systems, but do we > want to get into the business of picking a value and overriding whatever > value is set by the sysadmin? I don't think a high hard limit should be > seen as an implicit p

Re: Expanding HOT updates for expression and partial indexes

2025-02-11 Thread Matthias van de Meent
On Mon, 10 Feb 2025 at 20:11, Burd, Greg wrote: > > On Feb 10, 2025, at 12:17 PM, Matthias van de Meent > > wrote: > > > >> > >> I have a few concerns with the patch, things I’d greatly appreciate your > >> thoughts on: > >> > >> First, I pass an EState along the update path to enable running t

Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Tom Lane
Andres Freund writes: > My suggestion would be to redefine max_files_per_process as the number of > files we try to be able to open in backends. I.e. set_max_safe_fds() would > first count the number of already open fds (since those will largely be > inherited by child processes) and then check if

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-02-11 Thread Masahiko Sawada
On Fri, Jan 24, 2025 at 11:16 AM Masahiko Sawada wrote: > > On Thu, Jan 23, 2025 at 3:24 AM Ashutosh Bapat > wrote: > > > > On Thu, Jan 23, 2025 at 6:16 AM Masahiko Sawada > > wrote: > > > > > > On Fri, Jan 10, 2025 at 12:33 AM Masahiko Sawada > > > wrote: > > > > > > > > On Thu, Jan 9, 2025

Re: Proposal: "query_work_mem" GUC, to distribute working memory to the query's individual operators

2025-02-11 Thread Jeff Davis
On Tue, 2025-02-11 at 10:39 -0800, James Hunter wrote: > * The Path would store "nbytes" (= the optimizer's estimate of how > much working memory a given Path will use), to allow for future > optimizer logic to consider memory usage when choosing the best Path. > > * The Plan would store a copy of

Re: Skip collecting decoded changes of already-aborted transactions

2025-02-11 Thread Masahiko Sawada
On Mon, Feb 3, 2025 at 10:41 AM Masahiko Sawada wrote: > > On Wed, Jan 29, 2025 at 11:12 PM Peter Smith wrote: > > > > On Tue, Jan 28, 2025 at 9:26 PM Masahiko Sawada > > wrote: > > > > > > On Mon, Jan 27, 2025 at 7:01 PM Peter Smith wrote: > > > > > > ... > > > > > > To be honest, I didn't un

Re: Proposal: allow non-masked IPs inside of pg_hba.conf

2025-02-11 Thread Greg Sabino Mullane
> > This too would work better if hba.c were sharing cidr_in's logic: +1, a two-for-one solution. Cheers, Greg -- Crunchy Data - https://www.crunchydata.com Enterprise Postgres Software Products & Tech Support

Re: PATCH: Disallow a netmask of zero unless the IP is also all zeroes

2025-02-11 Thread Greg Sabino Mullane
On Tue, Feb 11, 2025 at 3:25 PM Tom Lane wrote: > More generally, should we reject if the netmask causes *any* nonzero > IP bits to be ignored? Our CIDR type already imposes that rule: > Yeah, I like that idea a lot. That's a great DETAIL message. Cheers, Greg -- Crunchy Data - https://www.cr

Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 16:18:37 -0500, Tom Lane wrote: > Andres Freund writes: > > And when using something like io_uring for AIO, it'd allow to > > max_files_per_process in addition to the files requires for the io_uring > > instances. > > Not following? Surely we'd not be configuring that so early

Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Tomas Vondra
On 2/11/25 21:18, Tom Lane wrote: > Tomas Vondra writes: >> I did run into bottlenecks due to "too few file descriptors" during a >> recent experiments with partitioning, which made it pretty trivial to >> get into a situation when we start trashing the VfdCache. I have a >> half-written draft

Re: describe special values in GUC descriptions more consistently

2025-02-11 Thread Peter Smith
I don't have an opinion about the ssl_crl stuff. Everything else looks good to me. == Kind Regards, Peter Smith. Fujitsu Australia.

Re: should we have a fast-path planning for OLTP starjoins?

2025-02-11 Thread Richard Guo
On Mon, Feb 10, 2025 at 5:35 PM Tomas Vondra wrote: > On 2/10/25 08:29, Richard Guo wrote: > > Hmm, I'm still a little concerned about whether the resulting joins > > are legal. Suppose we have a join pattern like the one below. > > > > F left join > > (D1 inner join T on true) on F.b = D1.b >

Re: Convert macros to static inline functions

2025-02-11 Thread Peter Eisentraut
On 31.01.25 14:29, Maxim Orlov wrote: Great job! I've been working on the 64 XIDs patch for years, and I've never liked this place.  On the other hand, as we know, inlining does not always work since it only suggests to the compiler to do it. After all, many of these calls are used in pretty "ho

Re: Move wal_buffers_full to WalUsage (and report it in pgss/explain)

2025-02-11 Thread Bertrand Drouvot
Hi, On Thu, Feb 06, 2025 at 10:27:17AM +, Bertrand Drouvot wrote: > === Patch's structure > > The patch series is made of 3 "small" sub-patches: > > 0001: to move wal_buffers_full to WalUsage > 0002: to report wal_buffers_full in pg_stat_statements > 0003: to report wal_buffers_full in expla

Re: Track the amount of time waiting due to cost_delay

2025-02-11 Thread Bertrand Drouvot
Hi, On Mon, Feb 10, 2025 at 02:52:46PM -0600, Nathan Bossart wrote: > Here is what I have prepared for commit. Other expanding the commit > messages, I've modified 0001 to just add a parameter to > vacuum_delay_point() to indicate whether this is a vacuum or analyze. I > was worried that adding

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-11 Thread Shubham Khanna
On Tue, Feb 11, 2025 at 6:30 AM Peter Smith wrote: > > Hi Shubham. > > Responding with a blanket statement "Fixed the given comments" makes > it difficult to be sure what was done when I come to confirm the > changes. Often embedded questions go unanswered, and if changes are > *not* made, then I

Re: Virtual generated columns

2025-02-11 Thread Richard Guo
On Tue, Feb 11, 2025 at 10:34 AM Richard Guo wrote: > Yeah, I also feel that the virtual generated columns should adhere to > outer join semantics, rather than being unconditionally replaced by > the generation expressions. But maybe I'm wrong. > > If that's the case, this incorrect-result issue

Re: pgbench with partitioned tables

2025-02-11 Thread Sergey Tatarintsev
08.02.2025 17:32, Álvaro Herrera пишет: On 2025-Feb-07, Melanie Plageman wrote: Okay, I've stared at this a bit, and it seems basically fine the way it is (I might add a bit more whitespace, clean up the commit message, etc). So I'm interested in committing it. I will admit that having never co

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-11 Thread Jelte Fennema-Nio
On Tue, 11 Feb 2025 at 00:53, Andres Freund wrote: > > I mostly meant worker based AIO, yes. I haven't checked how accurately these > > are kept for io_uring. I would hope they are... > > It does look like it is tracked. nice! > > The thing is that you'd often get completely misleading stats. So

Re: Non-text mode for pg_dumpall

2025-02-11 Thread Mahendra Singh Thalor
On Tue, 11 Feb 2025 at 20:40, jian he wrote: > > hi. > review based on v16. > > because of > https://postgr.es/m/cafc+b6pwqisl+3rvlxn9vhc8aonp4ov9c6u+bvd6kmwmdbd...@mail.gmail.com > > in copy_global_file_to_out_file, now it is: > if (strcmp(outfile, "-") == 0) > OPF = stdout; > I am co

Re: AIO v2.3

2025-02-11 Thread Robert Haas
On Tue, Feb 11, 2025 at 12:11 PM James Hunter wrote: > I like this idea. If we want to submit a batch, then just submit a batch. Sounds good to me, too. -- Robert Haas EDB: http://www.enterprisedb.com

Re: AIO v2.3

2025-02-11 Thread James Hunter
On Mon, Feb 10, 2025 at 2:40 PM Thomas Munro wrote: > ... > Problem statement: You want to be able to batch I/O submission, ie > make a single call to ioring_enter() (and other mechanisms) to start > several I/Os, but the code that submits is inside StartReadBuffers() > and the code that knows how

Re: explain analyze rows=%.0f

2025-02-11 Thread Andrei Lepikhov
On 8/2/2025 04:28, Ilia Evdokimov wrote: On 08.02.2025 00:01, Matheus Alcantara wrote: Just for reference I'm trying to apply based on commit fb056564ec5. You are right, because two commits were appeared after creating v6-patch on partition_prune.out and patch v6 must not have applied on master

Re: explain analyze rows=%.0f

2025-02-11 Thread Robert Haas
On Tue, Feb 11, 2025 at 12:14 PM Andrei Lepikhov wrote: > I support the idea in general, but I believe it should be expanded to > cover all cases of parameterised plan nodes. Each rescan iteration may > produce a different number of tuples, and rounding can obscure important > data. > > For exampl

Re: Using Expanded Objects other than Arrays from plpgsql

2025-02-11 Thread Tom Lane
Andrey Borodin writes: > On 7 Feb 2025, at 02:05, Tom Lane wrote: >> Do you have any further comments on this patch? > No, all steps of the patch set look good to me. Pushed then. Thanks for reviewing! regards, tom lane

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-11 Thread Jelte Fennema-Nio
On Tue, 11 Feb 2025 at 17:19, Andres Freund wrote: > Yes, at least initially: Ah, then I understand your point of view much better. Still I think we could easily frame it as: If you enable io_uring, you also get these additional fancy stats. Also afaict the items don't have to mean that > 1) it

Re: postgresql.conf.sample ordering for IO, worker related GUCs

2025-02-11 Thread Andres Freund
Hi, On 2025-01-31 18:21:54 -0500, Andres Freund wrote: > On January 31, 2025 5:22:43 PM EST, Robert Treat wrote: > >On Fri, Jan 31, 2025 at 10:25 AM Andres Freund wrote: > >> > >> Hi, > >> > >> On 2025-01-30 21:24:05 -0500, Andres Freund wrote: > >> > On January 30, 2025 8:55:36 PM EST, Tom Lane

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Sami Imseih
I do not have an explanation from the patch yet, but I have a test that appears to show unexpected results. I only tested a few datatypes, but from what I observe, some merge as expected and others do not; i.e. int columns merge correctly but bigint do not. """ show pg_stat_statements.query_id_con

Re: Expanding HOT updates for expression and partial indexes

2025-02-11 Thread Matthias van de Meent
On Tue, 11 Feb 2025 at 00:20, Nathan Bossart wrote: > > On Mon, Feb 10, 2025 at 06:17:42PM +0100, Matthias van de Meent wrote: > > I have serious doubts about the viability of any proposal working to > > implement PHOT/WARM in PostgreSQL, as they seem to have an inherent > > nature of fundamentall

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
Hello I noticed something that surprised me at first, but on further looking it should have been obvious: setting pg_stat_statements.query_id_const_merge affects the query ID for all readers of it, not just pg_stat_statement. This is good because it preserves the property that pg_stat_activity ent

Re: Move wal_buffers_full to WalUsage (and report it in pgss/explain)

2025-02-11 Thread Bertrand Drouvot
Hi, On Tue, Feb 11, 2025 at 03:30:09PM +0300, Ilia Evdokimov wrote: > Hi, > > Thank you for your work! Thanks for the review! > 1. Perhaps In EXPLAIN you forget to check that usage->wal_buffers_full > 0: > > if ((usage->wal_records > 0) || (usage->wal_fpi > 0) || (usage->wal_bytes > > 0)) I d

Re: RFC: Allow EXPLAIN to Output Page Fault Information

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 09:59:43 +0100, Jelte Fennema-Nio wrote: > On Tue, 11 Feb 2025 at 00:53, Andres Freund wrote: > > > The thing is that you'd often get completely misleading stats. Some of > > > the IO > > > will still be done by the backend itself, so there will be a non-zero > > > value. But i

Re: NOT ENFORCED constraint feature

2025-02-11 Thread Isaac Morland
On Tue, 11 Feb 2025 at 08:36, Álvaro Herrera wrote: > On 2025-Feb-10, Isaac Morland wrote: > > > I'm having a lot of trouble understanding the operational distinction > > between your 'u' and 'U'. If it's not enforced, it cannot be assumed to > be > > valid, regardless of whether it was valid in

Re: should we have a fast-path planning for OLTP starjoins?

2025-02-11 Thread Tomas Vondra
On 2/10/25 22:36, Robert Haas wrote: > On Fri, Feb 7, 2025 at 3:09 PM Tomas Vondra wrote: >> I don't think that's quite true. The order of dimension joins does not >> matter because the joins do not affect the join size at all. The size of >> |F| has nothing to do with that, I think. We'll do the

Re: Non-text mode for pg_dumpall

2025-02-11 Thread jian he
hi. review based on v16. because of https://postgr.es/m/cafc+b6pwqisl+3rvlxn9vhc8aonp4ov9c6u+bvd6kmwmdbd...@mail.gmail.com in copy_global_file_to_out_file, now it is: if (strcmp(outfile, "-") == 0) OPF = stdout; I am confused, why "-" means stdout. ``touch ./- `` command works fine. i

Re: explain analyze rows=%.0f

2025-02-11 Thread Matheus Alcantara
Em seg., 10 de fev. de 2025 às 18:14, Ilia Evdokimov escreveu: > Sorry for missing your question earlier. If you notice in the code above, the > variable(average) 'rows' is defined as: > > double rows = planstate->instrument->ntuples / nloops; > > This represents the total rows divided by the num

Re: should we have a fast-path planning for OLTP starjoins?

2025-02-11 Thread Tomas Vondra
On 2/11/25 10:28, Richard Guo wrote: > On Mon, Feb 10, 2025 at 5:35 PM Tomas Vondra wrote: >> On 2/10/25 08:29, Richard Guo wrote: >>> Hmm, I'm still a little concerned about whether the resulting joins >>> are legal. Suppose we have a join pattern like the one below. >>> >>> F left join >>> (

Re: Removing unneeded self joins

2025-02-11 Thread Alena Rybakina
Hi! Thank you for the work with this subject, I think it is really important. On 10.02.2025 22:58, Alexander Korotkov wrote: Hi! On Mon, Feb 10, 2025 at 7:19 AM Andrei Lepikhov wrote: On 9/2/2025 18:41, Alexander Korotkov wrote: Regarding adjust_relid_set() and replace_relid(). I think the

Re: Eagerly scan all-visible pages to amortize aggressive vacuum

2025-02-11 Thread Melanie Plageman
On Mon, Feb 10, 2025 at 8:06 PM Andres Freund wrote: > > On 2025-02-04 12:44:22 -0500, Melanie Plageman wrote: > > On Mon, Feb 3, 2025 at 9:09 PM Andres Freund wrote: > > > > + /* > > > > + * Now calculate the eager scan start block. Start at a random > > > > spot > > > > + * somew

Re: Proposal: "query_work_mem" GUC, to distribute working memory to the query's individual operators

2025-02-11 Thread Jeff Davis
On Mon, 2025-02-10 at 19:09 -0800, James Hunter wrote: > I think it makes sense to split the work into two parts: one part > that > improves SQL execution, and a second part that improves the > optimizer, > to reflect the improvements to execution. I like the idea to store the value of work_mem in

Re: describe special values in GUC descriptions more consistently

2025-02-11 Thread Nathan Bossart
Thank you for the reviews. On Mon, Feb 10, 2025 at 05:25:42PM -0700, David G. Johnston wrote: > On Mon, Feb 10, 2025 at 4:53 PM Peter Smith wrote: >> {"shared_memory_size_in_huge_pages", PGC_INTERNAL, PRESET_OPTIONS, >> gettext_noop("Shows the number of huge pages needed for the main >> share

Re: pg_stat_statements and "IN" conditions

2025-02-11 Thread Álvaro Herrera
On 2025-Feb-11, Sami Imseih wrote: > I do not have an explanation from the patch yet, but I have a test > that appears to show unexpected results. I only tested a few datatypes, > but from what I observe, some merge as expected and others do not; > i.e. int columns merge correctly but bigint do no

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-11 Thread Melanie Plageman
On Thu, Feb 6, 2025 at 1:06 PM Melanie Plageman wrote: > > On Wed, Feb 5, 2025 at 5:26 PM Melanie Plageman > wrote: > > > > Yes, looking at these results, I also feel good about it. I've updated > > the commit metadata in attached v14, but I could use a round of review > > before pushing it. > >

RE: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-11 Thread Hayato Kuroda (Fujitsu)
Dear Shubham, Thanks for updating the patch! I feel the patch has good shape. Here is a small comment. ``` + /* Error if no databases were found on the source server */ + if (num_rows == 0) + { + pg_log_error("no databases found on the source server"); +

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-11 Thread Julien Rouhaud
On Wed, Feb 12, 2025 at 09:08:00AM +0900, Michael Paquier wrote: > Wikipedia seems to agree with you that "fingerprint" would fit for > this purpose, though: > https://en.wikipedia.org/wiki/Fingerprint_(computing) > > Has anybody any comments about that? That would be a large renaming, > but in th

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-11 Thread Greg Sabino Mullane
On Tue, Feb 11, 2025 at 7:08 PM Michael Paquier wrote: > On Mon, Feb 10, 2025 at 02:02:10PM -0600, Sami Imseih wrote: > > I am OK with moving away from "jumble" in-lieu of something else, but my > thoughts are we should actually call this process "fingerprint" > I agree fingerprint is the right

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-11 Thread Sami Imseih
> Of course some people may want to keep the current behavior, if they have > limited number of temp tables or similar, so I had a GUC for that. I don't > think that the community would really welcome such GUC for core-postgres, > especially since it wouldn't be pg_stat_statements specific. FWIW,

Re: Skip collecting decoded changes of already-aborted transactions

2025-02-11 Thread Peter Smith
Hi. Here are some minor comments for the v18* patch set. // Patch v18-0001 1.1. Commit message A previously reported typo still exists: /noticeble/noticeable/ // Patch v18-0002 2.1 +#define RBTXN_PREPARE_STATUS_FLAGS (RBTXN_IS_PREPARED | RBTXN_SKIPPED_PREPARE | RBTXN_SENT_PR

RE: pg_rewind with --write-recovery-conf option doesn't write dbname to primary_conninfo value.

2025-02-11 Thread Hayato Kuroda (Fujitsu)
Dear Sawada-san, Thanks for updating the patch! > I've attached the updated patch. I address all comments I got so far > and added a small regression test. > > It makes sense to me that we move GetDbnameFromConnectionOptions() to > recovery_gen.c since this function is currently used only with >

Re: EvictUnpinnedBuffer and buffer free list

2025-02-11 Thread Ashutosh Bapat
Thanks a lot Melanie for a very detailed response, a good reference to pin. On Fri, Jan 31, 2025 at 8:20 PM Melanie Plageman wrote: > > I don't have an explicit issue with EvictUnpinnedBuffer() putting > buffers on the freelist -- it seems like that could be fine. But since > it is for testing/d

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-02-11 Thread Ajin Cherian
On Wed, Jan 29, 2025 at 9:31 AM Peter Smith wrote: Hi Ajin, Some review comments for patch v12-0001. == Commit message 1. Track transactions which have snapshot changes with a new flag RBTXN_HAS_SNAPSHOT_CHANGES ~ The commit message only says *what* it

EquivalenceClass and custom_read_write

2025-02-11 Thread Ashutosh Bapat
Hi All, In pathnodes.h typedef struct EquivalenceClass { pg_node_attr(custom_read_write, no_copy_equal, no_read, no_query_jumble) Because of custom_read_write attribute, I expect _outEquivalenceClass to be present in outfuncs.c and _readEquivalenceClass to be present in readfuncs.c. I find the fir

Re: Track the amount of time waiting due to cost_delay

2025-02-11 Thread Bertrand Drouvot
Hi, On Tue, Feb 11, 2025 at 04:42:26PM -0600, Nathan Bossart wrote: > On Tue, Feb 11, 2025 at 08:51:15AM +, Bertrand Drouvot wrote: > > On Mon, Feb 10, 2025 at 02:52:46PM -0600, Nathan Bossart wrote: > >> Off-list, I've asked Bertrand to gauge the feasibility of adding this > >> information to

Re: Fix punctuation errors in PostgreSQL documentation

2025-02-11 Thread John Naylor
On Mon, Feb 10, 2025 at 6:34 PM John Naylor wrote: > Thanks for the patch! I will push this after the upcoming minor releases. This is done. -- John Naylor Amazon Web Services

Re: pg_rewind with --write-recovery-conf option doesn't write dbname to primary_conninfo value.

2025-02-11 Thread Masahiko Sawada
On Mon, Feb 3, 2025 at 12:36 PM Masahiko Sawada wrote: > > On Sun, Feb 2, 2025 at 9:50 PM Hayato Kuroda (Fujitsu) > wrote: > > > > Dear Sawada-san, > > > > > I think it's a good idea to support it at least on HEAD. I've attached > > > a patch for that. > > > > +1. I've confirmed that pg_rewind an

Re: Test to dump and restore objects left behind by regression

2025-02-11 Thread Michael Paquier
On Tue, Feb 11, 2025 at 12:19:33PM +0530, Ashutosh Bapat wrote: > Sorry for replying late here. The refactored code in > 002_compare_backups.pl has a potential to cause confusion even without > this refactoring. The differences in tablespace paths are adjusted in > compare_files() and not in the ac

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-11 Thread Michael Paquier
On Mon, Feb 10, 2025 at 02:14:09PM -0600, Sami Imseih wrote: > Another thought that I have is that If we mention that extensions can use > these jumbling ( or whatever the final name is ) functions outside of > core, it makes > sense to actually show an example of this. What do you think? Not sure

Re: describe special values in GUC descriptions more consistently

2025-02-11 Thread Daniel Gustafsson
> On 11 Feb 2025, at 19:11, Nathan Bossart wrote: > I thought about this one a bit, and I actually came to the opposite > conclusion. IMHO it's reasonably obvious that an empty string means that > the file isn't loaded, so there's not much point in stating it in the GUC > description. Instead,

Re: Track the amount of time waiting due to cost_delay

2025-02-11 Thread Nathan Bossart
On Tue, Feb 11, 2025 at 08:51:15AM +, Bertrand Drouvot wrote: > On Mon, Feb 10, 2025 at 02:52:46PM -0600, Nathan Bossart wrote: >> Off-list, I've asked Bertrand to gauge the feasibility of adding this >> information to the autovacuum logs and to VACUUM/ANALYZE (VERBOSE). IMHO >> those are natu

Re: Bump soft open file limit (RLIMIT_NOFILE) to hard limit on startup

2025-02-11 Thread Tom Lane
Tomas Vondra writes: > On 2/11/25 21:18, Tom Lane wrote: >> I think what we actually would like to know is how often we have to >> close an open FD in order to make room to open a different file. >> Maybe that's the same thing you mean by "cache miss", but it doesn't >> seem like quite the right t

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-02-11 Thread Peter Smith
On Tue, Feb 11, 2025 at 9:16 PM Shubham Khanna wrote: > > > #13. Unanswered question "How are tests expecting this even passing?". > > Was a reason identified? IOW, how can we be sure the latest tests > > don't have a similar problem? > > > > In the v4-0001 patch [1], the tests were mistakenly usi

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Andres Freund
Hi, On 2025-02-11 13:12:17 +1300, Thomas Munro wrote: > Tomas queried[1] the limit of 256kB (or really 32 blocks) for > io_combine_limit. Yeah, I think we should increase it and allow > experimentation with larger numbers. Note that real hardware and > protocols have segment and size limits that

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-11 Thread Julien Rouhaud
On Wed, Feb 12, 2025 at 10:59:04AM +0900, Michael Paquier wrote: > On Wed, Feb 12, 2025 at 09:20:53AM +0800, Julien Rouhaud wrote: > > > > FTR my main motivation was to be able to deal with queries referencing > > temporary relations, as if your application creates a lot of those it > > basically

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Andres Freund
Hi, On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > How about just maintaining it in a new variable > effective_io_combine_limit, whenever either of them is assigned? Yea, that's probably the least bad way. I wonder if we should just name that variable io_combine_limit and have the GUC be _r

Re: Allow io_combine_limit up to 1MB

2025-02-11 Thread Thomas Munro
On Wed, Feb 12, 2025 at 3:22 PM Andres Freund wrote: > On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > > How about just maintaining it in a new variable > > effective_io_combine_limit, whenever either of them is assigned? > > Yea, that's probably the least bad way. > > I wonder if we should ju

Re: [PATCH] Optionally record Plan IDs to track plan changes for a query

2025-02-11 Thread Julien Rouhaud
On Tue, Feb 11, 2025 at 08:57:46PM -0600, Sami Imseih wrote: > > Of course some people may want to keep the current behavior, if they have > > limited number of temp tables or similar, so I had a GUC for that. I don't > > think that the community would really welcome such GUC for core-postgres, >

Re: Small memory fixes for pg_createsubcriber

2025-02-11 Thread Michael Paquier
On Tue, Feb 11, 2025 at 01:32:32PM -0300, Euler Taveira wrote: > There is no bug. They are the same behind the scenes. I'm fine changing it. It > is a new code and it wouldn't cause a lot of pain to backpatch patches in the > future. On consistency grounds, and as this is documented in fe-exec.c a

  1   2   >