Re: Consistently use macro HeapTupleIsValid to check the validity of tuples in tablecmds.c

2025-04-09 Thread Michael Paquier
On Wed, Apr 09, 2025 at 05:43:24PM +0300, Heikki Linnakangas wrote: > Agreed. I use both, depending on which mood I'm in. Same here, extended to OidIsValid(), HeapTupleIsValid(), XLogRecPtr, etc., and I tend to prefer such macros, except if consistency of the surroundings matter most. FWIW, I thi

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
> One possible bad behaviour that this could cause is if > autovacuum_vacuum_insert_scale_factor was set lower than > autovacuum_vacuum_scale_factor is that we could end up performing a > vacuum for inserts when all we've got is dead tuples from aborted > inserts. Thanks for pointing this out! >

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-09 Thread David Rowley
On Thu, 10 Apr 2025 at 16:57, Amit Langote wrote: > > On Thu, Apr 10, 2025 at 12:03 PM David Rowley wrote: > > -Most operations on EquivalenceClasses should ignore child members. > > +Most operations on EquivalenceClasses needn't look at child members. > > > > Would that be ok? > > Yeah, I think

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
>> >> What I am saying is n_ins_since_vacuum should not account for aborted >> >> inserts. >> > >> > It does and from what I can see it should. You need to explain why it >> > should not. More importantly, convincingly enough to change five year old >> > behavior. >> >> n_ins_since_vacuum was

Re: Possible api miuse bms_next_member

2025-04-09 Thread David Rowley
On Thu, 10 Apr 2025 at 02:18, Tom Lane wrote: > If we did want to do something about this warning, rather than > hacking up the call sites I'd be inclined to invent something like > "bms_first_member()" which does the same thing but additionally > asserts on empty input. Not really convinced it's

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-04-09 Thread Thomas Munro
On Wed, Apr 9, 2025 at 1:46 PM James Hunter wrote: > On Mon, Apr 7, 2025 at 7:34 PM Thomas Munro wrote: > > On Thu, Feb 13, 2025 at 1:40 PM Melanie Plageman > > wrote: > > > Thomas mentioned this to me off-list, and I think he's right. We > > > likely need to rethink the way parallel bitmap heap

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David Rowley
On Thu, 10 Apr 2025 at 10:54, Sami Imseih wrote: > Fair enough, and I think I got my answers from this thread. This > design decision was not > discussed in the thread for b07642dbcd8. This discussion is mostly settled down now, but... I also went through that thread to see if it was mentioned a

Re: sync_standbys_defined read/write race on startup

2025-04-09 Thread Michael Paquier
On Wed, Apr 09, 2025 at 06:21:50PM +0300, Maksim.Melnikov wrote: > Test 041_checkpoint_at_promote.pl is really good example > of using injection points, but anyway, I still don't see > how injection points can help us here. In failed test case > we started postgres, immediately open psql connection

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-09 Thread Amit Langote
On Thu, Apr 10, 2025 at 12:03 PM David Rowley wrote: > On Wed, 9 Apr 2025 at 17:38, Amit Langote wrote: > > Still, maybe a tiny tweak to the last line could help steer readers > > right without diving into storage. How about: > > > > Most operations on EquivalenceClasses should ignore child membe

Re: Streaming relation data out of order

2025-04-09 Thread Andres Freund
Hi, On 2025-04-09 19:17:00 +1200, Thomas Munro wrote: > One weird issue, not just with reordering, is that read_stream.c's > distance cools off too easily with some simple test patterns. Start > at 1, double on miss, subtract one on hit, repeat, and you can be > trapped alternating between 1 and

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Mark Dilger
> > Yes, there is a bug. Accounting rows inserted as part of an aborted > transaction in > n_ins_since_vacuum is not correct, since the same rows are being > accounted for with n_dead_tup. > If I create a table with autovacuum_enabled=false, insert rows (some of which abort), and check the stats,

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-04-09 Thread Daniel Gustafsson
> On 9 Apr 2025, at 20:41, Jacob Champion > wrote: > > Hello, > > On Thu, Apr 3, 2025 at 8:51 AM Daniel Gustafsson wrote: >> Committed, after another round of testing and looking. > > I think we may want to consider marking sslkeylogfile as a debug > option (that is, opt->dispchar = "D") in f

Re: Sample rate added to pg_stat_statements

2025-04-09 Thread Ilia Evdokimov
Since at the moment these changes are not a priority, and it's more important to address [0] for pg_stat_statements, I'm marking this patch as "Returned with Feedback". [0]: https://www.postgresql.org/message-id/Zz0MFPq1s4WFxxpL%40paquier.xyz -- Best regards, Ilia Evdokimov, Tantor Labs LLC.

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
> If we went with your suggestion, I think the final n_ins_since_vacuum column > would be 2. Do you think the n_tup_ins should also be 2? n_ins_since_vacuum should be 2 and n_tup_ins should be 10. A user tracks how many inserts they performed with n_tup_ins to measure load/activity on the d

Re: explain analyze rows=%.0f

2025-04-09 Thread Ilia Evdokimov
On 31.03.2025 23:59, Ilia Evdokimov wrote: We definitely shouldn’t remove the row counts < 1 check, since there are many places in the planner where we divide by rows. This mechanism was added specifically to prevent division by zero. Also, allowing rows estimates below 1 can sometimes make

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Mark Dilger
> The issue I see is that n_ins_since_vacuum should only > reflect the number of newly inserted rows that are eligible for > freezing, as described > in pgstat_report_vacuum [0] > > [0] > https://github.com/postgres/postgres/blob/master/src/backend/utils/activity/pgstat_relation.c#L238-L247 For t

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David G. Johnston
On Wed, Apr 9, 2025, 11:57 Sami Imseih wrote: > > Forget original purpose, is there presently a bug or not? > > Yes, there is a bug. Accounting rows inserted as part of an aborted > transaction in > n_ins_since_vacuum is not correct, since the same rows are being > accounted for with n_dead_tup.

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
> So why is it important we not account for the aborted insert in both > n_ins_since_vacuum and n_dead_tup? I am saying n_dead_tup should continue to account for n_dead_tup. I am not saying it should not. What I am saying is n_ins_since_vacuum should not account for aborted inserts. > When would

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David G. Johnston
On Wed, Apr 9, 2025, 12:39 Sami Imseih wrote: > > They will differ because n_tup_ins keeps increasing, while > n_ins_since_vacuum is > reset after a vacuum. The issue I see is that n_ins_since_vacuum should > only > reflect the number of newly inserted rows that are eligible for > freezing, as de

Summarizing indexes allowing single-phase VACUUM?

2025-04-09 Thread Matthias van de Meent
Hi, With PG16, we got Index AM -level indications for summarizing indexes, improving HOT applicability. Given that these indexes explicitly never store TIDs and thus don't really need the cleanup part of vacuum's index cleanup, we could well decide to not count those indexes to the indexes we nee

Re: Logging parallel worker draught

2025-04-09 Thread Melanie Plageman
On Mon, Feb 3, 2025 at 12:37 AM Sami Imseih wrote: > > Besides that, I think this is ready for committer. I started looking at this, and I like the idea. A few comments: I don't understand what 0002 is. For starters, the commit message says something about pg_stat_database, and there are no chan

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David G. Johnston
On Wed, Apr 9, 2025, 12:56 Sami Imseih wrote: > > What I am saying is n_ins_since_vacuum should not account for aborted > inserts. > It does and from what I can see it should. You need to explain why it should not. More importantly, convincingly enough to change five year old behavior. You sh

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
On Wed, Apr 9, 2025 at 4:23 PM Mark Dilger wrote: >> >> In other words, the reason n_ins_since_vacuum was introduced is to freeze >> (committed) rows, so it should not need to track dead rows to do what it >> intends >> to do. > > > Wouldn't that result in the rather strange behavior that 1 milli

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-04-09 Thread Yura Sokolov
08.04.2025 13:29, Aya Iwata (Fujitsu) wrote: > Hi Yura san, > > >> I just don't get, why it should be "in-memory"? All the same things you >> describe further, but storing in paged index on-disk with caching >> through shared_buffers - why this way it wouldn't work? > > We make the columnar stor

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Jelte Fennema-Nio
On Thu, Apr 10, 2025, 07:08 Jacob Champion wrote: > Christoph noted that this was also confusing from the packaging side, > earlier, and Daniel proposed -Doauth-client/--with-oauth-client as the > feature switch name instead. > +1 Next up: staticlibs. > I think your suggestion of not using any

Re: Draft for basic NUMA observability

2025-04-09 Thread Tomas Vondra
On 4/9/25 17:14, Andres Freund wrote: > Hi, > > On 2025-04-09 16:33:14 +0200, Tomas Vondra wrote: >> From e1f093d091610d70fba72b2848f25ff44899ea8e Mon Sep 17 00:00:00 2001 >> From: Tomas Vondra >> Date: Tue, 8 Apr 2025 23:31:29 +0200 >> Subject: [PATCH 1/2] Cleanup of pg_numa.c >> >> This moves/r

Re: Cleaning up ERRCODE usage in our XML code

2025-04-09 Thread Tom Lane
Peter Eisentraut writes: > On 14.09.24 21:14, Tom Lane wrote: >> +Section: Class 10 - XQuery Error >> + >> +# recent SQL versions define quite a few codes in this class, but for now >> +# we are only using this generic one >> +10608EERRCODE_INVALID_ARGUMENT_FOR_XQUERY

Re: Possible api miuse bms_next_member

2025-04-09 Thread Matthias van de Meent
On Wed, 9 Apr 2025 at 15:01, Ranier Vilela wrote: > > Hi. > > Per Coverity. > > CID 1608872: (#1 of 1): Improper use of negative value (NEGATIVE_RETURNS) > 32. negative_returns: bms_next_member(child_joinrel->relids, -1) is passed to > a parameter that cannot be negative.[show details] > > CID 16

Re: Cleaning up ERRCODE usage in our XML code

2025-04-09 Thread Chapman Flack
On 04/09/25 10:03, Tom Lane wrote: > I think I stole it from DB2: > > 10608 An error was encountered in the argument of an XQuery function or > operator. Interesting. ISO does seem to reserve a class 10 for XQuery errors, but not to define any subclass codes within it. Every other SQL/XML error

n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
Hi, I came across what appears to be incorrect behavior in the pg_stat_all_tables.n_ins_since_vacuum counter after a rollback. As shown below, the first two inserts of 1,000 rows were rolled back, yet they are still counted toward n_ins_since_vacuum.Consequently, they influence the vacuum insert

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
>> What I am saying is n_ins_since_vacuum should not account for aborted >> inserts. > > It does and from what I can see it should. You need to explain why it should > not. More importantly, convincingly enough to change five year old behavior. n_ins_since_vacuum was introduced to trigger auto

Add missing PGDLLIMPORT markings

2025-04-09 Thread Peter Eisentraut
I ran src/tools/mark_pgdllimport.pl and it detected a few new global variables with missing markings. See attached patch. Please point out if any of these should not be marked or if they are special cases in some other way. I'm Cc'ing various people involved with that new code. Btw., this n

Re: Add missing PGDLLIMPORT markings

2025-04-09 Thread Robert Haas
On Wed, Apr 9, 2025 at 11:28 AM Andres Freund wrote: > FWIW, the AIO ones really don't make sense to make public - the only reason > for those variables to exists is so they can be put into an array of > callbacks. There's no way an extension could ever benefit from them. But I > guess we don't r

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-04-09 Thread Jacob Champion
Hello, On Thu, Apr 3, 2025 at 8:51 AM Daniel Gustafsson wrote: > Committed, after another round of testing and looking. I think we may want to consider marking sslkeylogfile as a debug option (that is, opt->dispchar = "D") in fe-connect.c. Besides being a true "debug option", this would also pre

Re: Draft for basic NUMA observability

2025-04-09 Thread Bertrand Drouvot
Hi, On Tue, Apr 08, 2025 at 12:46:16PM +0200, Tomas Vondra wrote: > > > On 4/8/25 01:26, Shinoda, Noriyoshi (SXD Japan FSI) wrote: > > Hi, > > > > Thanks for developing this great feature. > > The manual says that the 'size' column of the pg_shmem_allocations_numa > > view is 'int4', but the

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Andres Freund
Hi, On 2025-04-09 17:05:39 -0500, Sami Imseih wrote: > On Wed, Apr 9, 2025 at 4:23 PM Mark Dilger > wrote: > >> > >> In other words, the reason n_ins_since_vacuum was introduced is to freeze > >> (committed) rows, so it should not need to track dead rows to do what it > >> intends > >> to do. >

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David G. Johnston
On Wednesday, April 9, 2025, Sami Imseih wrote: > > > What is the use case for that behavior? Perhaps you have one, but until > you make it explicit, it is hard for others to get behind your proposal. > > The point is to ensure that the pg_stats fields that autovacuum uses > are supplied the cor

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
> Except there isn’t some singular provably correct value here. Today’s > behavior (considering dead tuples) > is not intrinsically wrong nor correct, and neither is what you propose > (ignoring the dead tuples). > The fact that those dead tuples get removed regardless is a point in favor of >

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Jacob Champion
On Wed, Apr 9, 2025 at 10:44 AM Christoph Berg wrote: > > Re: Jacob Champion > > > How about ifdef-ing away the dlopen call if --with-libcurl is not > > > specified. > > > > This sounds like a pretty decent, simple way to go. Christoph, does > > that ring any alarm bells from your perspective? >

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-09 Thread Nathan Bossart
On Wed, Apr 09, 2025 at 01:20:29PM +0900, Michael Paquier wrote: > I guess that's the consensus, then. No objections to the removal here. That would look like the attached. There are still a couple of other known TOAST snapshot issues I need to revisit, but this sidesteps a few of them. But this

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-04-09 Thread Peter Smith
On Wed, Apr 9, 2025 at 8:08 PM Amit Kapila wrote: > > On Wed, Apr 9, 2025 at 7:48 AM Peter Smith wrote: > > > > I was looking at the recent push for the pg_createsubscription "--all" > > option [1], because I was absent for several weeks prior. > > > > I found some minor non-functional issues as

Re: Add missing PGDLLIMPORT markings

2025-04-09 Thread Andres Freund
Hi, On 2025-04-09 12:02:52 +0200, Peter Eisentraut wrote: > I ran src/tools/mark_pgdllimport.pl and it detected a few new global > variables with missing markings. See attached patch. Please point out if > any of these should not be marked or if they are special cases in some other > way. I'm C

Re: Possible api miuse bms_next_member

2025-04-09 Thread Tom Lane
Matthias van de Meent writes: > On Wed, 9 Apr 2025 at 15:01, Ranier Vilela wrote: >> Coverity has two new reports about use of the function *bms_next_member*. >> I think that he is right. > I don't know much about the planner, but I would expect a RelOptInfo's > relids field to always contain at

Re: Draft for basic NUMA observability

2025-04-09 Thread Andres Freund
Hi, On 2025-04-09 17:28:31 +0200, Tomas Vondra wrote: > On 4/9/25 17:14, Andres Freund wrote: > > I'd mention that the includes of postgres.h/fmgr.h is what caused missing > > build-time dependencies and via that failures on buildfarm member dogfish. > > > > Not really, I also need to include "c

Re: tab complete for COPY populated materialized view TO

2025-04-09 Thread Kirill Reshke
On Wed, 9 Apr 2025 at 13:23, jian he wrote: > > hi. > > we allow the "COPY table TO" command to copy rows from materialized > views in [1]. > The attached patch is to add a tab complete for it. > > [1] > https://git.postgresql.org/cgit/postgresql.git/commit/?id=534874fac0b34535c9a5ab9257d6574f784

Re: Draft for basic NUMA observability

2025-04-09 Thread Tomas Vondra
On 4/9/25 14:07, Tomas Vondra wrote: > ... > > OK, here are two patches, where 0001 adds the missingdeps check to the > Debian meson build. It just adds that to the build script. > > 0002 leaves the NUMA stuff in src/port (i.e. it's no longer moved to > src/backend/port). It still needs to includ

PostgreSQL 18 Release Management Team & Feature Freeze

2025-04-09 Thread Heikki Linnakangas
Hi, We are pleased to announce the Release Management Team (RMT) (cc'd) for the PostgreSQL 18 release: - Tomas Vondra - Nathan Bossart - Heikki Linnakangas You can find information about the responsibilities of the RMT here: https://wiki.postgresql.org/wiki/Release_Management_Team We have no

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Jelte Fennema-Nio
On Wed, Apr 9, 2025, 10:58 Jacob Champion wrote: > Is it acceptable/desirable for a build, which has not been configured > --with-libcurl, to still pick up a compatible OAuth implementation > installed by the distro? If so, we can go with a "bare" dlopen(). If > that's not okay, I think we will p

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
> Forget original purpose, is there presently a bug or not? Yes, there is a bug. Accounting rows inserted as part of an aborted transaction in n_ins_since_vacuum is not correct, since the same rows are being accounted for with n_dead_tup. > Between the two options the one where we count dead tupl

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David G. Johnston
On Wed, Apr 9, 2025 at 11:32 AM Sami Imseih wrote: > The purpose of b07642dbcd8d is to trigger autovacuum for append only/mainly > workloads that don't generate dead tuples. > > Why does counting or not counting the dead tuples matter? Forget original purpose, is there presently a bug or not? B

Re: Draft for basic NUMA observability

2025-04-09 Thread Andres Freund
Hi, On 2025-04-07 19:59:59 +0200, Tomas Vondra wrote: > This reminds me whether it's fine to have "os_page_num" as int. Should > we make it bigint, perhaps? Yes, that's better. Seems very unlikely anybody will encounter this in the next few years, but it's basically free to use the larger range h

Re: Draft for basic NUMA observability

2025-04-09 Thread Andres Freund
Hi, On 2025-04-09 16:33:14 +0200, Tomas Vondra wrote: > From e1f093d091610d70fba72b2848f25ff44899ea8e Mon Sep 17 00:00:00 2001 > From: Tomas Vondra > Date: Tue, 8 Apr 2025 23:31:29 +0200 > Subject: [PATCH 1/2] Cleanup of pg_numa.c > > This moves/renames some of the functions defined in pg_numa.c

Re: Build macOS shared modules as dylib rather than bundle

2025-04-09 Thread Tom Lane
Peter Eisentraut writes: > ... But this change triggers a new variant of this issue: > https://www.postgresql.org/message-id/e1o4hov-001oyi...@gemulon.postgresql.org > With the changed linker options, the symbol search order appears to > be different, and so hash_search() gets found in the OS libr

Re: sync_standbys_defined read/write race on startup

2025-04-09 Thread Maksim.Melnikov
Hi Michael, thanks for your answer. Test 041_checkpoint_at_promote.pl is really good example of using injection points, but anyway, I still don't see how injection points can help us here. In failed test case we started postgres, immediately open psql connection and commit prepared transaction. P

Re: Draft for basic NUMA observability

2025-04-09 Thread Tomas Vondra
On 4/9/25 01:29, Andres Freund wrote: > Hi, > > On 2025-04-09 01:10:09 +0200, Tomas Vondra wrote: >> On 4/8/25 15:06, Andres Freund wrote: >>> Hi, >>> >>> On 2025-04-08 17:44:19 +0500, Kirill Reshke wrote: On Mon, 7 Apr 2025 at 23:00, Tomas Vondra wrote: > I'll let the CI run the tests o

Re: Draft for basic NUMA observability

2025-04-09 Thread Tomas Vondra
Updated patches with proper commit messages etc. -- Tomas Vondra From e1f093d091610d70fba72b2848f25ff44899ea8e Mon Sep 17 00:00:00 2001 From: Tomas Vondra Date: Tue, 8 Apr 2025 23:31:29 +0200 Subject: [PATCH 1/2] Cleanup of pg_numa.c This moves/renames some of the functions defined in pg_numa.c

Build macOS shared modules as dylib rather than bundle

2025-04-09 Thread Peter Eisentraut
I'm splitting this out from [0] so that it can be tracked and referenced separately. [0]: https://www.postgresql.org/message-id/170e7e57-fe43-4e7e-8566-a96fcdf23075%40eisentraut.org Historically, bundles (ld -bundle) were the only file type that could be dynamically loaded at run time on macO

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-09 Thread Amit Kapila
On Wed, Apr 9, 2025 at 11:24 AM Bertrand Drouvot wrote: > > On Wed, Apr 09, 2025 at 12:03:06PM +0900, Michael Paquier wrote: > > On Tue, Apr 08, 2025 at 06:42:53AM +, Bertrand Drouvot wrote: > > > Fully agree. Will need to find another way to prevent a process to wait > > > between the > > >

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

2025-04-09 Thread Peter Smith
Hi Ajin. Some review comments for patch v16-0001. == Commit message 1. A hash cache of relation file locators is implemented to cache whether a relation is filterable or not. This ensures that we only need to retrieve ~ /hash cache/hash table/ (AFAICT you called this a hash table elsewher

Re: pg_recvlogical cannot create slots with failover=true

2025-04-09 Thread Masahiko Sawada
On Sun, Apr 6, 2025 at 7:19 PM Zhijie Hou (Fujitsu) wrote: > > On Sat, Apr 5, 2025 at 1:45 AM Masahiko Sawada wrote: > > Hi, > > > Thank you for updating the patch! Pushed with small cosmetic changes. > > Thanks for pushing the feature ! > > I noticed one typo in the doc and here is a tiny patch t

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-09 Thread Bertrand Drouvot
Hi, On Tue, Apr 08, 2025 at 02:13:20PM +0900, Michael Paquier wrote: > On Tue, Apr 08, 2025 at 02:00:35AM +, Hayato Kuroda (Fujitsu) wrote: > > Your patch looks good to me and it could pass on my env. PSA patches for > > PG16. > > Patch for PG17 is not changed, just renamed. > > @@ -1287,6 +

Re: speedup COPY TO for partitioned table.

2025-04-09 Thread jian he
hi. rebase and simplify regress tests. From f56c94ccb018928e41cc35e162174831cb016c1d Mon Sep 17 00:00:00 2001 From: jian he Date: Thu, 10 Apr 2025 10:41:40 +0800 Subject: [PATCH v10 1/1] support COPY partitioned_table TO CREATE TABLE pp (id int, val int ) PARTITION BY RANGE (id); CREATE TABLE p

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-09 Thread Michael Paquier
On Wed, Apr 09, 2025 at 12:07:31PM +0530, Amit Kapila wrote: > On Wed, Apr 9, 2025 at 11:24 AM Bertrand Drouvot > wrote: > I can't think of a good reason to have this DEBUG1 as there is no > predictability of it getting generated even with tests using an > injection point. OTOH, I don't have any o

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-09 Thread David Rowley
On Wed, 9 Apr 2025 at 17:38, Amit Langote wrote: > Still, maybe a tiny tweak to the last line could help steer readers > right without diving into storage. How about: > > Most operations on EquivalenceClasses should ignore child members, > which are stored separately from normal members. I think

Improve a few appendStringInfo calls new to v18

2025-04-09 Thread David Rowley
Looks like v18 has grown a few appendStringInfo misusages, e.g. using appendStringInfo() when no formatting is needed or just using format "%s" instead of using appendStringInfoString(). I've attached a couple of patches. The 0001 is just my method for finding these, not for commit. 0002 contains

Fw: Mention the default io_method?

2025-04-09 Thread Daniel Westermann (DWE)
Hi, forwarding from docs to maybe get some attention. At least Pavel and me agree that the default io_method should be mentioned in the docs. https://www.postgresql.org/message-id/flat/ZR0P278MB04279CB0C1D8F49DE68F168ED2AF2%40ZR0P278MB0427.CHEP278.PROD.OUTLOOK.COM Regards Daniel

Re: Add .DS_Store to .gitignore

2025-04-09 Thread David Rowley
On Thu, 10 Apr 2025 at 18:05, Dianjin Wang wrote: > Here is a patch to add the `.DS_Store` to the .gitignore to ignore the > `.DS_Store` file tracking by git when doing some development work in > the macOS terminal. It helps me and my community (Apache Cloudberry) a > lot. I hope this small patch

Add .DS_Store to .gitignore

2025-04-09 Thread Dianjin Wang
Hi, Here is a patch to add the `.DS_Store` to the .gitignore to ignore the `.DS_Store` file tracking by git when doing some development work in the macOS terminal. It helps me and my community (Apache Cloudberry) a lot. I hope this small patch can help you, too. This is my first time sending the

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Sami Imseih
> This is *not* an oversight. It is by design. See commit b07642dbcd8d. The > documentation says I don't see in the commit message why inserts in an aborted transaction must count towards n_ins_since_vacuum. > Estimated number of rows inserted since this table was last vacuumed > > Those rows w

Re: Possible api miuse bms_next_member

2025-04-09 Thread Ranier Vilela
Em qua., 9 de abr. de 2025 às 10:27, Matthias van de Meent < boekewurm+postg...@gmail.com> escreveu: > On Wed, 9 Apr 2025 at 15:01, Ranier Vilela wrote: > > > > Hi. > > > > Per Coverity. > > > > CID 1608872: (#1 of 1): Improper use of negative value (NEGATIVE_RETURNS) > > 32. negative_returns: bm

Re: Consistently use macro HeapTupleIsValid to check the validity of tuples in tablecmds.c

2025-04-09 Thread Tom Lane
Heikki Linnakangas writes: > Inconsistency is not good either though. I'm not sure it's worth the > churn, but I could get on board a patch to actually replace all > HeapTupleIsValid(tuple) calls with plain "tuple != NULL" checks. Keep > HeapTupleIsValid() just for compatibility, with a comment

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-09 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 05:37:50PM -0400, Jan Wieck wrote: > I remember an incident where large amounts of LOs ran pg_upgrade into a > transaction-ID wrap around because the restore part would create individual > single statement transactions per LO to create, then change permissions and > ownershi

Re: Draft for basic NUMA observability

2025-04-09 Thread Tomas Vondra
On 4/9/25 17:51, Andres Freund wrote: > Hi, > > On 2025-04-09 17:28:31 +0200, Tomas Vondra wrote: >> On 4/9/25 17:14, Andres Freund wrote: >>> I'd mention that the includes of postgres.h/fmgr.h is what caused missing >>> build-time dependencies and via that failures on buildfarm member dogfish. >>

Re: Add missing PGDLLIMPORT markings

2025-04-09 Thread Andres Freund
Hi, On April 9, 2025 12:58:23 PM EDT, Robert Haas wrote: >On Wed, Apr 9, 2025 at 11:28 AM Andres Freund wrote: >> FWIW, the AIO ones really don't make sense to make public - the only reason >> for those variables to exists is so they can be put into an array of >> callbacks. There's no way an e

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-09 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 09:41:06PM +0200, Hannu Krosing wrote: > On Tue, Apr 8, 2025 at 8:39 PM Nathan Bossart > wrote: >> I've also verified that the dependency information is carried over in >> upgrades to later versions (AFAICT all the supported ones). > > If I remember correctly the change t

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-09 Thread Nathan Bossart
On Tue, Apr 08, 2025 at 01:51:22PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> On Tue, Apr 08, 2025 at 01:36:58PM -0400, Tom Lane wrote: >>> Hmm ... one annoying thing for this project is that AFAICS pg_upgrade >>> does *not* preserve database OIDs, which is problematic for using >>> COPY t

Re: Consistently use macro HeapTupleIsValid to check the validity of tuples in tablecmds.c

2025-04-09 Thread Heikki Linnakangas
On 09/04/2025 14:51, Tender Wang wrote: Hi, While working on another patch, I find that tablecmds.c now has three ways to check the validity of catalog tuples. i. if (tuple != NULL) ii. if (!tuple) iii. if (HeapTupleIsValid(tuple) In tablecmds.c, most checks use macro HeapTupleIsValid. For c

Re: Feature freeze

2025-04-09 Thread Nico Williams
On Tue, Apr 08, 2025 at 11:24:39AM -0500, Nathan Bossart wrote: > I always forget if AoE is UTC+12 or UTC-12. [...] One isn't supposed to think "is the freeze on everywhere", just "is the freeze on for me" and "is the freeze on for this particular contribution (check the date header)". Nico --

Possible api miuse bms_next_member

2025-04-09 Thread Ranier Vilela
Hi. Per Coverity. CID 1608872: (#1 of 1): Improper use of negative value (NEGATIVE_RETURNS) 32. negative_returns: bms_next_member(child_joinrel->relids, -1) is passed to a parameter that cannot be negative.[show details] CID 1608871: (#1 of 1): Out-of-bounds access (OVERRUN) 32. overrun-buffer-a

Re: [PoC] Reducing planning time when tables have many partitions

2025-04-09 Thread Amit Langote
Hi David, On Wed, Apr 9, 2025 at 5:12 AM David Rowley wrote: > On Wed, 9 Apr 2025 at 02:24, Tom Lane wrote: > > > > David Rowley writes: > > > I've pushed the patch now. Thanks for all the reviews of my adjustments. > > > > Shouldn't the CF entry be marked committed? > > I've done that now. Sh

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Christoph Berg
Re: Jacob Champion > > How about ifdef-ing away the dlopen call if --with-libcurl is not specified. > > This sounds like a pretty decent, simple way to go. Christoph, does > that ring any alarm bells from your perspective? Ok for me. The opposite that I said in the other mail was just a suggestio

Re: SQLFunctionCache and generic plans

2025-04-09 Thread Tom Lane
I wrote: > I noticed that avocet and trilobite (two of our CLOBBER_CACHE_ALWAYS > animals) have started to fail on the deadlock-parallel isolation > test, with symptoms that look like they're timing out. > ... > The simplest fix is to force that test to use debug_discard_caches=0, > but I don't lov

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread David G. Johnston
On Wed, Apr 9, 2025 at 1:30 PM Sami Imseih wrote: > >> What I am saying is n_ins_since_vacuum should not account for aborted > inserts. > > > > It does and from what I can see it should. You need to explain why it > should not. More importantly, convincingly enough to change five year old > beh

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Mark Dilger
> > In other words, the reason n_ins_since_vacuum was introduced is to freeze > (committed) rows, so it should not need to track dead rows to do what it > intends > to do. > Wouldn't that result in the rather strange behavior that 1 million dead rows might trigger a vacuum due to one threshold, 1

Re: Streaming relation data out of order

2025-04-09 Thread Thomas Munro
Here is a more fleshed out version of this concept patch, now that we have lots of streams wired up to try it with. Bitmap Heap Scan seemed to be a good candidate. postgres=# create table t (a int unique, b int unique); CREATE TABLE postgres=# insert into t select generate_series(1, 10), gene

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Andres Freund
Hi, On 2025-04-07 09:41:25 -0700, Jacob Champion wrote: > On Mon, Apr 7, 2025 at 7:21 AM Andres Freund wrote: > > On 2025-04-04 17:27:46 -0700, Jacob Champion wrote: > > I think otherwise > > we run into the danger of the wrong library version being loaded in some > > cases. Imagine a program bei

Re: Assertion with aborted UPDATE in subtransaction

2025-04-09 Thread Luc Vlaming Hummel
The considerations we ran in to are is that now you will create more calls to TransactionIdIsInProgress. However, we think it would then also create a correct multixact with the right members only, and thereby avoid the assertion. The one thing we have not been able to figure out yet, is what ot

Re: Changing shared_buffers without restart

2025-04-09 Thread Dmitry Dolgov
> On Wed, Apr 09, 2025 at 11:12:18AM GMT, Ashutosh Bapat wrote: > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > In the new v4 version > > of the patch the first option is implemented. > > > > The patches don't apply cleanly using git am but patch -p1 applies >

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Christoph Berg
Re: Jacob Champion > Is it acceptable/desirable for a build, which has not been configured > --with-libcurl, to still pick up a compatible OAuth implementation > installed by the distro? If so, we can go with a "bare" dlopen(). If > that's not okay, I think we will probably need to use pkglibdir or

Re: Changing shared_buffers without restart

2025-04-09 Thread Dmitry Dolgov
> On Wed, Apr 09, 2025 at 01:20:16PM GMT, Ashutosh Bapat wrote: > ../../coderoot/pg/src/include/storage/s_lock.h:93:2: error: #error > "s_lock.h may not be included from frontend code" > > How about this? Why is that happening? The same -- as you can see it comes from compiling pg_numa.c, which as

Re: Draft for basic NUMA observability

2025-04-09 Thread Jakub Wartak
On Wed, Apr 9, 2025 at 12:48 AM Tomas Vondra wrote: > On 4/8/25 16:59, Andres Freund wrote: > > Hi, > > > > On 2025-04-08 09:35:37 -0400, Andres Freund wrote: > >> On April 8, 2025 9:21:57 AM EDT, Tomas Vondra wrote: > >>> On 4/8/25 15:06, Andres Freund wrote: > On 2025-04-08 17:44:19 +0500,

Re: [PATCH] New predefined role pg_manage_extensions

2025-04-09 Thread Laurenz Albe
On Sat, 2025-04-05 at 19:07 -0400, Tom Lane wrote: > I took another look at this patch, and I still can't persuade > myself that a good case has been made for it. There are too > many scenarios where pg_manage_extensions would be a security > problem and too few where it seems to really improve ma

Re: BUG #18815: Logical replication worker Segmentation fault

2025-04-09 Thread Amit Kapila
On Tue, Apr 8, 2025 at 12:26 PM Zhijie Hou (Fujitsu) wrote: > > > I am attaching the V2 patch which implements this idea by > postponing index info initialization until the FindConflictTuple(). I > confirmed > It can pass regression and pgindent check. > Thanks, I pushed this patch yesterday. -

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-04-09 Thread Amit Kapila
On Wed, Apr 9, 2025 at 7:48 AM Peter Smith wrote: > > I was looking at the recent push for the pg_createsubscription "--all" > option [1], because I was absent for several weeks prior. > > I found some minor non-functional issues as follows: > > == > doc/src/sgml/ref/pg_createsubscriber.sgml >

Re: Memoize ANTI and SEMI JOIN inner

2025-04-09 Thread David Rowley
On Wed, 9 Apr 2025 at 18:48, Richard Guo wrote: > > On Thu, Mar 20, 2025 at 3:02 PM David Rowley wrote: > > For making this work, I think the attached should be about the guts of > > the code changes. I didn't look at the comments. Right now I can't > > think of any reason why this can't be done,

Re: Feature Recommendations for Logical Subscriptions

2025-04-09 Thread Amit Kapila
On Tue, Apr 8, 2025 at 2:48 PM YeXiu <1518981...@qq.com> wrote: > > Business Scenario: > The BI department requires real-time data from the operational database. In > our current approach (on platform 14), we create a separate database within > our department's real-time backup instance, set up a

Re: pgsql: Make cancel request keys longer

2025-04-09 Thread Heikki Linnakangas
On 09/04/2025 13:28, Heikki Linnakangas wrote: On 09/04/2025 12:39, Peter Eisentraut wrote: On 09.04.25 10:53, Heikki Linnakangas wrote: On 08/04/2025 22:41, Heikki Linnakangas wrote: On 08/04/2025 20:06, Peter Eisentraut wrote: While I was looking at this, I suggest to make the first argument

Re: Parallel CREATE INDEX for GIN indexes

2025-04-09 Thread Kirill Reshke
On Thu, 3 Apr 2025 at 01:19, Tomas Vondra wrote: > > On 4/2/25 18:43, Andres Freund wrote: > > Hi, > > > > On 2025-03-04 20:50:43 +0100, Tomas Vondra wrote: > >> I pushed the two smaller parts today. > >> > >> Here's the remaining two parts, to keep cfbot happy. I don't expect to > >> get these in

Re: sync_standbys_defined read/write race on startup

2025-04-09 Thread Michael Paquier
On Mon, Apr 07, 2025 at 11:39:23PM +0300, Maksim.Melnikov wrote: > Hi everyone, thanks for your comments. > I've just wanted to share little cosmetic > fixes for previous patch. Note: I would suggest to use a version number for the patches, rather than a 0003 which usually implies that this is thi

Re: Changing shared_buffers without restart

2025-04-09 Thread Ashutosh Bapat
On Wed, Apr 9, 2025 at 1:15 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Wed, Apr 09, 2025 at 11:12:18AM GMT, Ashutosh Bapat wrote: > > On Mon, Apr 7, 2025 at 2:13 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > > > > In the new v4 version > > > of the patch the first option is im

  1   2   >