Persist injection points across server restarts

2025-04-29 Thread Michael Paquier
Hi all, A bug related to data visibility on standbys with a race condition mixing 2PC transactions and synchronous_standby_names with the checkpointer has been fixed a couple of days ago: https://www.postgresql.org/message-id/163fcbec-900b-4b07-beaa-d2ead8634...@postgrespro.ru One issue that we h

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-04-29 Thread Tom Lane
Amit Kapila writes: > Thanks, Dilip and Sawada-San, for the inputs, and Vignesh for the > patch. I have pushed the change. Xuneng Zhou pointed out on Discord that the test case added by 7c99dc587 has caused repeated failures in CI --- though oddly, it's not failed in the buildfarm so far as I can

Re: Persist injection points across server restarts

2025-04-29 Thread Andrey Borodin
> On 30 Apr 2025, at 10:35, Michael Paquier wrote: > > - A flush is used to write the points' data to disk. Interesting functionality. Did you consider custom resource manager to WAL-log injection points? This way we do not need to flush them explicitly, they always will be persistent. Thoug

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread Masahiko Sawada
On Tue, Apr 29, 2025 at 5:00 AM Nisha Moond wrote: > > On Tue, Apr 29, 2025 at 2:51 PM shveta malik wrote: > > > > On Mon, Apr 28, 2025 at 4:33 PM Nisha Moond > > wrote: > > > > > > Please find the v9 patch, addressing the above and all other comments as > > > well. > > > > > > > Thanks for th

Re: add --sequence-data to pg_dumpall

2025-04-29 Thread Michael Paquier
On Tue, Apr 29, 2025 at 03:55:08PM -0500, Nathan Bossart wrote: > I noticed that I forgot to add --sequence-data to pg_dumpall in commit > 9c49f0e, which added the same option to pg_dump. This option is primarily > intended for use by pg_upgrade, so we will need it if/when we teach > pg_upgrade to

Re: alphabetize long options in pg_dump[all] docs

2025-04-29 Thread Nathan Bossart
On Tue, Apr 29, 2025 at 11:45:11PM +0200, Álvaro Herrera wrote: > I think the concept here is that all short options go first in > alphabetical order, then the long options in their own alphabetical > order, and if one option has both, then the short option takes > precedence. That's what it looks

expand on_error ignore error handling scope

2025-04-29 Thread jian he
hi. Currently, (on_error ignore) only handles data type incompatibility errors. However, we can extend its functionality to also handle errors caused by extra data (additional columns) or missing data (fewer columns), I think. on_error is only applicable to non-binary formats. In non-binary forma

Re: expand on_error ignore error handling scope

2025-04-29 Thread Michael Paquier
On Wed, Apr 30, 2025 at 08:00:00AM +0800, jian he wrote: > Currently, (on_error ignore) only handles data type incompatibility errors. > However, we can extend its functionality to also handle errors caused by extra > data (additional columns) or missing data (fewer columns), I think. The original

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2025-04-29 Thread Peter Geoghegan
On Tue, Apr 1, 2025 at 7:02 PM Peter Geoghegan wrote: > Though I think it should be "" safe even when "key->sk_attno > > firstchangingattnum" "", to highlight that the rule here is > significantly more permissive than even the nearby range skip array > case, which is still safe when (key->sk_attno

Typo in multixact.c and jsonfuncs.c documentation

2025-04-29 Thread Junwang Zhao
Found a trivial typo in multixact.c, after searching the code base there is another such typo in jsonfuncs.c. - * range, that is, greater to or equal than oldestMultiXactId, and less than + * range, that is, greater than or equal to oldestMultiXactId, and less than -- Regards Junwang Zhao v1-0

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-29 Thread Michael Paquier
On Tue, Apr 29, 2025 at 02:01:54PM -0400, Tom Lane wrote: > I'm inclined to argue that it's a bug fix and therefore still in-scope > for v18. The fact that we can't back-patch such a change is all the > more reason to not let it slide another year. Not on the RMT. I have looked at the patch, and

Re: alphabetize long options in pg_dump[all] docs

2025-04-29 Thread Álvaro Herrera
On 2025-Apr-29, Nathan Bossart wrote: > I noticed some of the new pg_dump[all] long options (e.g., --with-data, > --statistics-only) are not listed in alphabetical order in the docs. > Attached is a patch to fix that. I think the concept here is that all short options go first in alphabetical ord

Re: Typo in multixact.c and jsonfuncs.c documentation

2025-04-29 Thread David Rowley
On Wed, 30 Apr 2025 at 13:25, Junwang Zhao wrote: > Found a trivial typo in multixact.c, after searching the code base > there is another such typo in jsonfuncs.c. Thanks. Pushed. David

Re: Disallow redundant indexes

2025-04-29 Thread Japin Li
On Tue, 29 Apr 2025 at 12:14, Sami Imseih wrote: >> On Mon, Apr 28, 2025 at 01:12:18PM -0500, Sami Imseih wrote: >> > If I had a dollar for every time I asked a user to check for duplicate >> > indexes :) So >> > I think it's a good idea to provide some way to detect these ( besides >> > a query i

Re: Conflict detection for update_deleted in logical replication

2025-04-29 Thread shveta malik
On Fri, Apr 25, 2025 at 4:05 PM shveta malik wrote: > > > > > Here is V30 patch set includes the following changes: > > > > Thank You for the patch, please find few concerns: > Please find few more concerns: 3) In get_candidate_xid(), we first set candidate_xid_time and later candidate_xid. And

Re: Enhancing Memory Context Statistics Reporting

2025-04-29 Thread Rahila Syed
Hi, Please find attached a patch with some comments and documentation changes. Additionaly, added a missing '\0' termination to "Remaining Totals" string. I think this became necessary after we replaced dsa_allocate0() with dsa_allocate() is the latest version. Thank you, Rahila Syed 0001-Fix-t

Re: add tab-complete for ALTER DOMAIN ADD...

2025-04-29 Thread Dagfinn Ilmari Mannsåker
jian he writes: > hi. > > per https://www.postgresql.org/docs/current/sql-alterdomain.html > > we can add tab-complete for ALTER DOMAIN ADD variants: > ALTER DOMAIN sth ADD CHECK > ALTER DOMAIN sth ADD CONSTRAINT > ALTER DOMAIN sth ADD NOT NULL Good catch. > + /* ALTER DOMAIN ADD */ > +

Re: Implement waiting for wal lsn replay: reloaded

2025-04-29 Thread Alexander Korotkov
Hi, Tomas. Thank you so much for your review! Please find the revised patchset. On Thu, Mar 13, 2025 at 4:15 PM Tomas Vondra wrote: > I did a quick look at this patch. I haven't found any correctness > issues, but I have some general review comments and questions about the > grammar / syntax. >

add tab-complete for ALTER DOMAIN ADD...

2025-04-29 Thread jian he
hi. per https://www.postgresql.org/docs/current/sql-alterdomain.html we can add tab-complete for ALTER DOMAIN ADD variants: ALTER DOMAIN sth ADD CHECK ALTER DOMAIN sth ADD CONSTRAINT ALTER DOMAIN sth ADD NOT NULL v1-0001-add-tab-complete-for-ALTER-DOMAIN-ADD.patch Description: Binary data

Re: Questions about logicalrep_worker_launch()

2025-04-29 Thread Amit Kapila
On Mon, Apr 28, 2025 at 2:37 PM Fujii Masao wrote: > > On 2025/04/26 3:03, Masahiko Sawada wrote: > > I agree with these changes. > > > > I think that while the changes for (2) should be for v19, the changes > > for (1) might be treated as a bug fix? > > Agreed. I've split the patch into two parts

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread Amit Kapila
On Tue, Apr 29, 2025 at 3:01 AM Masahiko Sawada wrote: > > Your fix looks good to me. Is it worth considering putting an > assertion to verify if new two_phase_at is equal to or greater than > confirmed_lsn (or at least it doesn't go backward), when syncing > two_phase_at? > Yeah, it makes sense.

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-04-29 Thread Nikita Malakhov
Hi, Nikhil, please consider existing discussions on using dictionaries (mentioned above by Aleksander) and extending the TOAST pointer [1], it seems you did not check them. The same question Robert asked above - it's unclear why the header wastes so much space. You mentioned metadata length - wha

libpq: Add PQapplicationname() function

2025-04-29 Thread songjinzhou
Dear hackers, here is a libpq function: PQapplicationname(), I think it may be of some use.  Looking forward to your reply, thank you. v1_0001-libpq-Add-PQapplicationname-function.patch Description: Binary data

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread Nisha Moond
On Tue, Apr 29, 2025 at 2:51 PM shveta malik wrote: > > On Mon, Apr 28, 2025 at 4:33 PM Nisha Moond wrote: > > > > Please find the v9 patch, addressing the above and all other comments as > > well. > > > > Thanks for the patch. > > 1) > > + The default is false. This option cannot be se

Re: POC: make mxidoff 64 bits

2025-04-29 Thread Ashutosh Bapat
Hi Maxim, On Wed, Apr 16, 2025 at 1:42 PM Maxim Orlov wrote: > I moved the topic to the next commitfest. > I am reviewing these patches. I notice that transam/README does not mention multixact except one place in section "Transaction Emulation during Recovery". I expected it to document how pg

Re: extension_control_path and "directory"

2025-04-29 Thread Matheus Alcantara
On Mon, Apr 28, 2025 at 5:49 PM David E. Wheeler wrote: > > > To append the /extension at each path on extension_control_path would > > require some changes on find_in_path() that > > find_extension_control_filename() calls, which I think that it would > > make the function more complicated. I've

Re: MergeAppend could consider sorting cheapest child path

2025-04-29 Thread Andrei Lepikhov
On 4/25/25 17:13, Andrei Lepikhov wrote: On 4/25/25 11:16, Alexander Pyhalov wrote: Usually, sorted cheapest_total_path will be cheaper than sorted fractional/startup path at least by startup cost (as after sorting it includes total_cost of input path). But we ignore this case when selecting c

Re: RFC: Additional Directory for Extensions

2025-04-29 Thread Matheus Alcantara
On Mon, Apr 28, 2025 at 6:15 PM David E. Wheeler wrote: > > On Apr 25, 2025, at 15:23, David E. Wheeler wrote: > > > Thank you for the review. Here’s v3*. > > V4 removes “/extension” from the end of the `extension_control_path` value. > It looks good to me. Just some a minor point: +Then th

Re: RFC: Additional Directory for Extensions

2025-04-29 Thread David E. Wheeler
On Apr 29, 2025, at 09:56, Matheus Alcantara wrote: > Typo on "io"? Maybe "into" or "in”? Bah, yes, it’s “to”. Updated in v5 (and also a PR[1] for those who prefer that UX). Best, David [1]: https://github.com/theory/postgres/pull/10 v5-0001-Flesh-out-docs-for-the-prefix-make-variable.pa

Re: Fix slot synchronization with two_phase decoding enabled

2025-04-29 Thread shveta malik
On Mon, Apr 28, 2025 at 4:33 PM Nisha Moond wrote: > > Please find the v9 patch, addressing the above and all other comments as well. > Thanks for the patch. 1) + The default is false. This option cannot be set together with + two_phase when creating the slot. However, once +

Re: meson vs. llvm bitcode files

2025-04-29 Thread Nazir Bilal Yavuz
Hi, On Thu, 13 Mar 2025 at 13:11, Nazir Bilal Yavuz wrote: > One thing I noticed is that gen_srcfiles['srcfiles'] seems wrong. > gen_sources is a better name compared to gen_srcfiles. So, I changed > it to gen_sources in v4. Rebase is needed due to b1720fe63f, v5 is attached. -- Regards, Nazir

Re: type cache cleanup improvements

2025-04-29 Thread Alexander Korotkov
Hi, Noah! On Tue, Apr 29, 2025 at 3:56 AM Noah Misch wrote: > On Mon, Apr 21, 2025 at 04:54:08AM +0300, Alexander Korotkov wrote: > > On Sat, Apr 12, 2025 at 12:43 AM Alexander Korotkov > > wrote: > > > On Fri, Apr 11, 2025 at 11:32 PM Noah Misch wrote: > > > > On Tue, Oct 22, 2024 at 08:33:24

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-04-29 Thread Robert Haas
On Mon, Apr 28, 2025 at 5:32 PM Nikhil Kumar Veldanda wrote: > Thanks for raising that question. The idea behind including a 24-bit > length field alongside the 1-byte algorithm ID is to ensure that each > compressed datum self-describes its metadata size. This allows any > compression algorithm t

Re: Disallow redundant indexes

2025-04-29 Thread Euler Taveira
On Mon, Apr 28, 2025, at 7:17 PM, Nathan Bossart wrote: > On Mon, Apr 28, 2025 at 01:12:18PM -0500, Sami Imseih wrote: > > If I had a dollar for every time I asked a user to check for duplicate > > indexes :) So > > I think it's a good idea to provide some way to detect these ( besides > > a query

Re: libpq: Add PQapplicationname() function

2025-04-29 Thread David G. Johnston
On Tuesday, April 29, 2025, songjinzhou wrote: > Dear hackers, here is a libpq function: PQapplicationname(), I think it > may be of some use. > You really should provide at least one reasoned use case for why this should exist. > > Looking forward to your reply, thank you. > > This isn’t even

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-29 Thread Nathan Bossart
On Mon, Apr 28, 2025 at 06:19:19PM -0400, Tom Lane wrote: > LGTM other than that nit. Thanks. I'll stash this away for July unless someone wants to argue that it's fair game for v18. IMHO this isn't nearly urgent enough for that, and the bugs will continue to exist on older major versions regard

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Nathan Bossart
On Tue, Apr 29, 2025 at 01:31:55PM -0400, Tom Lane wrote: > I wrote: >> Andres seemed lukewarm about reverting 38da05346 or 6d0154196, so >> I left it be for the moment. But I still feel the argument is good >> that "these will do little except confuse future hackers". Barring >> objection, I'll

Re: Disallow redundant indexes

2025-04-29 Thread Sami Imseih
> On Mon, Apr 28, 2025 at 01:12:18PM -0500, Sami Imseih wrote: > > If I had a dollar for every time I asked a user to check for duplicate > > indexes :) So > > I think it's a good idea to provide some way to detect these ( besides > > a query in the > > Wiki ), but I don't think we should prevent i

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Nathan Bossart
On Tue, Apr 29, 2025 at 08:36:48AM +0200, Peter Eisentraut wrote: > On 28.04.25 16:41, Nathan Bossart wrote: >> On Mon, Apr 28, 2025 at 09:14:54AM -0400, Tom Lane wrote: >> > My own proposal given the way it works now is to just print >> > max_connections and not mention autovacuum_worker_slots at

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Tom Lane
Peter Eisentraut writes: > On 28.04.25 16:41, Nathan Bossart wrote: >> However, weren't we considering reverting some of this stuff [0]? I see >> that sawshark is now choosing max_connections = 40 and >> autovacuum_worker_slots = 6, and since there are no other apparent related >> buildfarm failu

Re: Slot's restart_lsn may point to removed WAL segment after hard restart unexpectedly

2025-04-29 Thread Masahiko Sawada
On Mon, Apr 28, 2025 at 6:39 PM Alexander Korotkov wrote: > > On Tue, Apr 29, 2025 at 4:03 AM Masahiko Sawada wrote: > > > > On Mon, Apr 28, 2025 at 8:17 AM Alexander Korotkov > > wrote: > > > > > > > I have a question - is there any interest to backport the solution into > > > > existing major

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Nathan Bossart
On Tue, Apr 29, 2025 at 01:19:18PM -0400, Tom Lane wrote: > Peter Eisentraut writes: >> On 28.04.25 16:41, Nathan Bossart wrote: >>> However, weren't we considering reverting some of this stuff [0]? I see >>> that sawshark is now choosing max_connections = 40 and >>> autovacuum_worker_slots = 6,

Re: MergeAppend could consider sorting cheapest child path

2025-04-29 Thread Alexander Pyhalov
Andrei Lepikhov писал(а) 2025-04-29 16:52: On 4/25/25 17:13, Andrei Lepikhov wrote: On 4/25/25 11:16, Alexander Pyhalov wrote: Usually, sorted cheapest_total_path will be cheaper than sorted fractional/startup path at least by startup cost (as after sorting it includes total_cost of input path

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Tom Lane
I wrote: > Andres seemed lukewarm about reverting 38da05346 or 6d0154196, so > I left it be for the moment. But I still feel the argument is good > that "these will do little except confuse future hackers". Barring > objection, I'll go revert them. Actually ... on looking again at 6d0154196 ("Lo

Re: extension_control_path and "directory"

2025-04-29 Thread David E. Wheeler
On Apr 29, 2025, at 09:49, Matheus Alcantara wrote: > Yes, it is a single path from the search path, in your case it will be > "/Users/david/Downloads/share/postgresql" and "$system". We split these > paths based on the system path separator and get the next "piece" here: > >char

Re: extension_control_path and "directory"

2025-04-29 Thread Matheus Alcantara
On Tue, Apr 29, 2025 at 11:08 AM David E. Wheeler wrote: > Right. My point is a minor one, but I thin you can use an if/ else there: > > ```c > if (strcmp(piece, "$system") == 0) { > /* Substitute the path macro if needed */ > mangled = substitute_path_macro(piece, "$system", syste

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-29 Thread Tom Lane
Nathan Bossart writes: > Thanks. I'll stash this away for July unless someone wants to argue that > it's fair game for v18. IMHO this isn't nearly urgent enough for that, and > the bugs will continue to exist on older major versions regardless. I'm inclined to argue that it's a bug fix and ther

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-29 Thread Nathan Bossart
On Tue, Apr 29, 2025 at 02:01:54PM -0400, Tom Lane wrote: > Nathan Bossart writes: >> Thanks. I'll stash this away for July unless someone wants to argue that >> it's fair game for v18. IMHO this isn't nearly urgent enough for that, and >> the bugs will continue to exist on older major versions

Re: What's our minimum supported Python version?

2025-04-29 Thread Peter Eisentraut
On 29.04.25 17:16, Tom Lane wrote: Peter Eisentraut writes: On 24.04.25 18:20, Jacob Champion wrote: Seems like no one is pushing hard for an earlier version, yet, so here's a patch with your suggested wording from upthread. I'm not sure if this meets Peter's request for precision. (Though I'm

Re: What's our minimum supported Python version?

2025-04-29 Thread Jacob Champion
On Tue, Apr 29, 2025 at 8:46 AM Peter Eisentraut wrote: > > On the reading that "supported" means "we'll try to fix a problem > > rather than telling you to use a newer Python", I suspect that the > > correct thing to say is 3.6.8 not 3.6. There may be no difference > > in practice; but if push c

Re: extension_control_path and "directory"

2025-04-29 Thread David E. Wheeler
On Apr 29, 2025, at 11:06, Matheus Alcantara wrote: > The substitute_path_macro() already handles the if/else on "piece" but I > think that this if/else version looks nicer. Fixed. > > I've also included some documentation changes for this v5 version to > remove the "extension" from the examples

Re: libpq: Add PQapplicationname() function

2025-04-29 Thread Japin Li
On Tue, 29 Apr 2025 at 19:22, "songjinzhou" wrote: > Dear hackers, here is a libpq function: PQapplicationname(), I think it may > be of some use. > > Looking forward to your reply, thank you. > I believe the use cases for this are narrow. Generally, clients set the application_name and then

Re: What's our minimum supported Python version?

2025-04-29 Thread Tom Lane
Peter Eisentraut writes: > On 24.04.25 18:20, Jacob Champion wrote: >> Seems like no one is pushing hard for an earlier version, yet, so >> here's a patch with your suggested wording from upthread. I'm not sure >> if this meets Peter's request for precision. (Though I'm not really >> excited about

Re: dispchar for oauth_client_secret

2025-04-29 Thread Jacob Champion
On Mon, Apr 28, 2025 at 6:49 PM Noah Misch wrote: > > Subject: [PATCH 1/2] oauth: Disallow OAuth connections via > > postgres_fdw/dblink > > Subject: [PATCH 2/2] oauth: Classify oauth_client_secret as a password > > Both look good. Committed. Thanks for the review! --Jacob

Re: What's our minimum supported Python version?

2025-04-29 Thread Jacob Champion
On Tue, Apr 29, 2025 at 8:57 AM Jacob Champion wrote: > Cool, I will plan to push this Sometime Soon, then. This is now committed. Thanks everybody! --Jacob

Re: Built-in Raft replication

2025-04-29 Thread Jim Nasby
I've always assumed there'd have to be at least one global stream, if for no other purpose than to be the source of truth about transaction commit ordering (though, I was thinking of supporting multiple streams for one database). Presumably the same could be used for shared objects. Or perhaps shar

add --sequence-data to pg_dumpall

2025-04-29 Thread Nathan Bossart
I noticed that I forgot to add --sequence-data to pg_dumpall in commit 9c49f0e, which added the same option to pg_dump. This option is primarily intended for use by pg_upgrade, so we will need it if/when we teach pg_upgrade to use pg_dumpall to dump the databases. (Right now pg_upgrade directly c

alphabetize long options in pg_dump[all] docs

2025-04-29 Thread Nathan Bossart
I noticed some of the new pg_dump[all] long options (e.g., --with-data, --statistics-only) are not listed in alphabetical order in the docs. Attached is a patch to fix that. -- nathan >From 47b974f8ecbd44c26c4d9d8e1a8354e28f43549d Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 29 Apr 2

Re: New committer: Jacob Champion

2025-04-29 Thread Fabrízio de Royes Mello
On Fri, Apr 11, 2025 at 5:26 PM Jonathan S. Katz wrote: > The Core Team would like to extend our congratulations to Jacob > Champion, who has accepted an invitation to become our newest PostgreSQL > committer. > > This is awesome news... congrats Jacob!!! Regards, -- Fabrízio de Royes Mello

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Tom Lane
Nathan Bossart writes: > On Tue, Apr 29, 2025 at 01:31:55PM -0400, Tom Lane wrote: >> So now I'm inclined to leave that one alone. I'd still revert >> 38da05346, which means the comment added by 6d0154196 needs some minor >> adjustments. But I think we can stick with the "(max_connections / >> 6

Re: allow changing autovacuum_max_workers without restarting

2025-04-29 Thread Nathan Bossart
On Tue, Apr 29, 2025 at 05:31:01PM -0400, Tom Lane wrote: > Pushed. I realized that the text about SEMMNI/SEMMNS in runtime.sgml > needed some work too, since it still said that increasing them was > optional on NetBSD/OpenBSD. Thanks. Your updates look good to me. -- nathan