Re: plperl version on the meson setup summary screen

2025-03-13 Thread vignesh C
On Tue, 4 Feb 2025 at 12:07, Zharkov Roman wrote: > > Hello, > > Here is a new patch version. > I tried to use perl 'version' instead of 'api_versionstring' to sync > with configure script. One suggestion, there are many other external libraries for which we don't display the version, can we incl

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-13 Thread David G. Johnston
On Tuesday, March 11, 2025, Peter Smith wrote: > > Choice 3. Implement some option that has an argument saying what to delete > > Implement an option that takes some argument saying what objects to remove. > > Here, the current patch would be something like: > "--remove-target-objects=publication

Re: SQLFunctionCache and generic plans

2025-03-13 Thread Alexander Pyhalov
Tom Lane писал(а) 2025-03-13 21:29: Pavel Stehule writes: Maybe interesting change is the change of error message context QUERY: SELECT public.dep_req2() || ' req3b'. -CONTEXT: SQL function "dep_req3b" during startup +CONTEXT: SQL function "dep_req3b" statement 1 I'm not hugely excite

Re: Reducing the log spam

2025-03-13 Thread Laurenz Albe
On Tue, 2025-03-11 at 10:46 +0100, Laurenz Albe wrote: > Attached is the fifth version of the patch. ... and here, for good measure, a pgindented version Apart from that, no changes. Yours, Laurenz Albe From d644d3fe1b2c5ab2cbf7f10e9c653b7baa9efe01 Mon Sep 17 00:00:00 2001 From: Laurenz Albe Dat

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-13 Thread Amit Kapila
On Fri, Mar 14, 2025 at 3:06 AM Euler Taveira wrote: > > On Wed, Mar 12, 2025, at 3:41 AM, Amit Kapila wrote: > > This could be another option, but I feel an option with a tool is more > meaningful than allowing users to do afterward steps. > > > I wasn't paying much attention to this discussion.

RE: Skip collecting decoded changes of already-aborted transactions

2025-03-13 Thread Hayato Kuroda (Fujitsu)
Dear hackers, I hope I'm in the correct thread. In abfb296, rbtxn_skip_prepared() was removed from SnapBuildDistributeNewCatalogSnapshot(). ISTM it was an only caller of the function. Is it an intentional for external projects? Or it can be removed like attached? Best regards, Hayato Kuroda FU

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Michael Paquier
> On Mar 14, 2025, at 9:27, David Rowley wrote: > > Err, what about non-node types? Those'll go to AppendJumble(). > > I think basically everyone here apart from you is having a hard time > understanding what additional benefits your counter solution brings > over just ensuring we always Appen

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread David Rowley
On Fri, 14 Mar 2025 at 15:46, Michael Paquier wrote: > > > On Mar 14, 2025, at 9:27, David Rowley wrote: > > I think basically everyone here apart from you is having a hard time > > understanding what additional benefits your counter solution brings > > over just ensuring we always AppendJumble w

Re: ecdh support causes unnecessary roundtrips

2025-03-13 Thread Jacob Champion
On Tue, Mar 4, 2025 at 4:05 PM Daniel Gustafsson wrote: > > On 4 Mar 2025, at 20:19, Daniel Gustafsson wrote: > > Thanks for the reminder, this is sitting on my must-have TODO for 18 and I > > agree that we should add x25519 to the default set. > > And to add some code for that proposal, the atta

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-13 Thread Andres Freund
On 2025-03-13 10:29:49 -0700, Jacob Champion wrote: > On Thu, Mar 13, 2025 at 9:56 AM Andres Freund wrote: > > I am wondering if PAM is so fundamentally incompatible with handling > > interrupts / a non-blocking interface that we have little choice but to > > eventually remove it... > > Given the

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread David Rowley
On Fri, 14 Mar 2025 at 14:51, Michael Paquier wrote: > > > > On Mar 14, 2025, at 8:15, Sami Imseih wrote: > >> FWIW, another idea I have on top of my mind is the addition of a > >> counter in JumbleState that we increment each time we enter > >> _jumbleNode(), then simply call JUMBLE_FIELD_SINGLE

Re: Available disk space per tablespace

2025-03-13 Thread Quan Zongliang
On 2025/3/14 02:10, Christoph Berg wrote: Hi, I'm picking up a 5 year old patch again: https://www.postgresql.org/message-id/flat/20191108132419.GG8017%40msg.df7cb.de Users will be interested in knowing how much extra data they can load into a database, but PG currently does not expose that n

Re: per backend WAL statistics

2025-03-13 Thread Nazir Bilal Yavuz
Hi, On Mon, 10 Mar 2025 at 17:43, Bertrand Drouvot wrote: > On Mon, Mar 10, 2025 at 03:08:49PM +0300, Nazir Bilal Yavuz wrote: > > > I think that every time we flush IO or WAL stats, we want(?) to flush > > backend stats as well, > > Yeah, I think that's happening anyway. > > > so would it make s

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Michael Paquier
> On Mar 14, 2025, at 8:15, Sami Imseih wrote: >> FWIW, another idea I have on top of my mind is the addition of a >> counter in JumbleState that we increment each time we enter >> _jumbleNode(), then simply call JUMBLE_FIELD_SINGLE() after the >> incrementation. And we can rely on this counter

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-03-13 Thread Daniel Gustafsson
> On 13 Mar 2025, at 19:31, Tom Lane wrote: > > Jacob Champion writes: >> Adding the PG prefix to the envvar name addresses my collision >> concern, but I think Tom's comment upthread [1] was saying that we >> should not provide any envvar at all: > >>> I think it might be safer if we only ac

dsm_registry: Add detach and destroy features

2025-03-13 Thread Sungwoo Chang
Hi, I had use cases for manually detaching and destroying dsm segment while developing my extension (emulating Oracle's DBMS_PIPE package). In this extension user can call user defined procedures that destroy the resources in the shared memory. There were mentions about adding detach features in t

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread David Rowley
On Tue, 11 Mar 2025 at 20:48, David Rowley wrote: > Since then, I see that Ivan > has already submitted a patch that accounts for NULL nodes and adds a > byte to the jumble buffer to account for NULLs. This seems quite clean > and simple. However, Sami seems to have concerns about the overhead of

Re: Separate GUC for replication origins

2025-03-13 Thread Euler Taveira
On Thu, Mar 13, 2025, at 11:10 AM, vignesh C wrote: > Few comments: Thanks for taking a look. > 1) After selecting max_active_replication_origins setting in the > SELECT query having order by, the first record returned will be the > one with max_active_replication_origins, rather than the second

Re: Add contrib/pg_logicalsnapinspect

2025-03-13 Thread Euler Taveira
On Tue, Mar 11, 2025, at 7:34 PM, Masahiko Sawada wrote: > Pushed. pgindent is saying this commit included some extra tabs. git diff diff --git a/contrib/pg_logicalinspect/pg_logicalinspect.c b/contrib/pg_logicalinspect/pg_logicalinspect.c index ff6c682679f..5a44718bea8 100644 --- a/contrib/pg_l

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Sami Imseih
> FWIW, another idea I have on top of my mind is the addition of a > counter in JumbleState that we increment each time we enter > _jumbleNode(), then simply call JUMBLE_FIELD_SINGLE() after the > incrementation. And we can rely on this counter to be unique each > time we jumble a node.. With thi

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Michael Paquier
On Thu, Mar 13, 2025 at 07:17:20PM -0500, Sami Imseih wrote: >> Then we have the same problem with another Node using this Foo node >> two times in a row, depending on how it's used by the query parsing >> and transform: > > hmm, it's hard to imagine such a case being real-world and useful for > q

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Michael Paquier
On Wed, Mar 12, 2025 at 03:28:27PM +1300, David Rowley wrote: > If we account for the NULLs, those two cases become: " bytes>" and > " bytes>", which is going to be highly unlikely to hash to the same > value due to the buffer not being the same. > > Can you explain where my understanding is wrong

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-13 Thread Euler Taveira
On Wed, Mar 12, 2025, at 3:41 AM, Amit Kapila wrote: > This could be another option, but I feel an option with a tool is more > meaningful than allowing users to do afterward steps. I wasn't paying much attention to this discussion. The thread title refers to a general option to clean publisher

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Sami Imseih
> Then we have the same problem with another Node using this Foo node > two times in a row, depending on how it's used by the query parsing > and transform: hmm, it's hard to imagine such a case being real-world and useful for query jumbling purposes. Also, I think if we introduce something like J

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-03-13 Thread Melanie Plageman
On Thu, Mar 13, 2025 at 5:46 AM Jakub Wartak wrote: > > Cool, anything > 1 is just better. Just quick question, so now we have: > > #define DEFAULT_EFFECTIVE_IO_CONCURRENCY 16 > #define DEFAULT_MAINTENANCE_IO_CONCURRENCY 10 > > Shouldn't maintenance be now also at the same value (16) instead of >

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-13 Thread Thomas Munro
On Fri, Mar 14, 2025 at 12:23 PM Andres Freund wrote: > On 2025-03-14 11:31:47 +1300, Thomas Munro wrote: > > 2. Somehow tell elog.c not to call CHECK_FOR_INTERRUPTS() instead. > > Also yuck, but at least elog.c is already the right place to clean > > state up on non-local exit... > > How would t

Re: Changing the state of data checksums in a running cluster

2025-03-13 Thread Tomas Vondra
On 3/13/25 17:26, Tomas Vondra wrote: > On 3/13/25 13:32, Daniel Gustafsson wrote: >>> On 13 Mar 2025, at 12:03, Tomas Vondra wrote: >>> >>> ... >>> >>> This also reminds me I had a question about the barrier - can't it >>> happen a process gets to process multiple barriers at the same time? I >>>

Re: ecdh support causes unnecessary roundtrips

2025-03-13 Thread Jacob Champion
On Thu, Mar 13, 2025 at 2:41 PM Daniel Gustafsson wrote: > OpenSSL 3.4 also doesn't like it and AFAICT neither does the upcoming 3.5 Hm. FWIW, I have no issues locally with 3.4 or 3.5-alpha. Only with LibreSSL. --Jacob

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-13 Thread Andres Freund
Hi, On 2025-03-14 11:31:47 +1300, Thomas Munro wrote: > 2. Somehow tell elog.c not to call CHECK_FOR_INTERRUPTS() instead. > Also yuck, but at least elog.c is already the right place to clean > state up on non-local exit... How would that differ from HOLD_INTERRUPTS? > 3. Considering errfinis

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-03-13 Thread Abhishek Chanda
Thanks, Daniel. Should there be the ifdef guard in here as well? + {"sslkeylogfile", NULL, NULL, NULL, + "SSL-Key-Log-File", "", 0, /* sizeof("") = 0 */ + offsetof(struct pg_conn, sslkeylogfile)}, + A small nit, this line should say NULL + /* line is guaranteed by OpenSSL to be NUL terminated *

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-13 Thread Thomas Munro
On Fri, Mar 14, 2025 at 5:03 AM Andres Freund wrote: > If there is any LOG/DEBUG elog inside functions like mdreadv(), mdwritev(), > mdextend(), be it directly or indirectly, the functions become unsafe. The > problem is that at the end of errfinish() there is a CFI: > > /* > * Ch

Re: making EXPLAIN extensible

2025-03-13 Thread Sami Imseih
> The validation point is an interesting one. I agree that we don't > want the behavior to depend on the order in which options are > written. Here is what I applied on top of v6-0001 to correct this issue. Attaching it as a text file only as Robert may have a different opinion on how to fix this

Re: ecdh support causes unnecessary roundtrips

2025-03-13 Thread Daniel Gustafsson
> On 13 Mar 2025, at 22:39, Jacob Champion > wrote: > > On Tue, Mar 4, 2025 at 4:05 PM Daniel Gustafsson wrote: >>> On 4 Mar 2025, at 20:19, Daniel Gustafsson wrote: >>> Thanks for the reminder, this is sitting on my must-have TODO for 18 and I >>> agree that we should add x25519 to the defaul

Re: dblink: Add SCRAM pass-through authentication

2025-03-13 Thread Matheus Alcantara
On Thu, Mar 13, 2025 at 4:54 PM Jacob Champion wrote: > > On Thu, Mar 13, 2025 at 6:59 AM Matheus Alcantara > wrote: > > Fixed by wrapping on PG_CATCH/pfree/PG_RE_THROW. I didn't managed to > > create a test that use this code path, so let me know if I'm still > > missing something. > > Thanks! L

Re: Get rid of WALBufMappingLock

2025-03-13 Thread Alexander Korotkov
On Fri, Mar 7, 2025 at 5:08 PM Alexander Korotkov wrote: > On Sun, Mar 2, 2025 at 1:58 PM Alexander Korotkov > wrote: > > > > On Fri, Feb 28, 2025 at 3:13 PM Álvaro Herrera > > wrote: > > > On 2025-Feb-28, Michael Paquier wrote: > > > > > > > Saying that, I have also done similar tests with yo

Re: Proposal: manipulating pg_control file from Perl

2025-03-13 Thread Tom Lane
Aleksander Alekseev writes: > Previously it was pointed out [1] that manipulating the pg_control > file from Cluster.pm may be beneficial under certain conditions. > As an example, one can downgrade the catalog version in pg_control, > place input files for pg_upgrade, execute pg_upgrade and chec

Proposal: manipulating pg_control file from Perl

2025-03-13 Thread Aleksander Alekseev
Hi, Previously it was pointed out [1] that manipulating the pg_control file from Cluster.pm may be beneficial under certain conditions. As an example, one can downgrade the catalog version in pg_control, place input files for pg_upgrade, execute pg_upgrade and check the results. This would allow

Re: dblink: Add SCRAM pass-through authentication

2025-03-13 Thread Jacob Champion
On Thu, Mar 13, 2025 at 6:59 AM Matheus Alcantara wrote: > Fixed by wrapping on PG_CATCH/pfree/PG_RE_THROW. I didn't managed to > create a test that use this code path, so let me know if I'm still > missing something. Thanks! Looks like the regression suite has one test that takes that path (foun

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-03-13 Thread Jacob Champion
On Wed, Mar 5, 2025 at 9:21 AM Daniel Gustafsson wrote: > I managed to misunderstand skip blocks in TAP tests in the 0002, so the > attached version fixes that. It has been failing on Debian in CI which I have > yet to look into. Drive-by comment: > +{"sslkeylogfile", "PGSSLKEYLOGFILE", > +

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-03-13 Thread Tom Lane
Jacob Champion writes: > Adding the PG prefix to the envvar name addresses my collision > concern, but I think Tom's comment upthread [1] was saying that we > should not provide any envvar at all: >> I think it might be safer if we only accepted it as a connection >> parameter and not via an envi

Re: SQLFunctionCache and generic plans

2025-03-13 Thread Tom Lane
Pavel Stehule writes: > Maybe interesting change is the change of error message context > QUERY: SELECT public.dep_req2() || ' req3b'. > -CONTEXT: SQL function "dep_req3b" during startup > +CONTEXT: SQL function "dep_req3b" statement 1 I'm not hugely excited about that given that it's just

Available disk space per tablespace

2025-03-13 Thread Christoph Berg
Hi, I'm picking up a 5 year old patch again: https://www.postgresql.org/message-id/flat/20191108132419.GG8017%40msg.df7cb.de Users will be interested in knowing how much extra data they can load into a database, but PG currently does not expose that number. This patch introduces a new function pg

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-13 Thread Jacob Champion
On Thu, Mar 13, 2025 at 9:56 AM Andres Freund wrote: > I am wondering if PAM is so fundamentally incompatible with handling > interrupts / a non-blocking interface that we have little choice but to > eventually remove it... Given the choice between a usually-working PAM module with known architec

DROP TABLESPACE waits for checkpoint with lwlock held

2025-03-13 Thread Andres Freund
Hi, For a while I thought there was a hard deadlock involving DROP TABLESPACE, due to doing RequestCheckpoint() while holding an lwlock. That turned out to not be the case, the fd.c LRU was corrupted, causing checkpointer to spin endlessly in _dump_lru(), due to [1]. After putting HOLD/RESUME_INTE

Re: SQLFunctionCache and generic plans

2025-03-13 Thread Pavel Stehule
Hi I am checking last patches Maybe interesting change is the change of error message context QUERY: SELECT public.dep_req2() || ' req3b'. -CONTEXT: SQL function "dep_req3b" during startup +CONTEXT: SQL function "dep_req3b" statement 1 almost all SQL functions have just one statement, so s

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-13 Thread Andres Freund
Hi, On 2025-03-13 09:23:10 -0700, Jacob Champion wrote: > On Wed, Mar 12, 2025 at 3:16 PM Jacob Champion > wrote: > > I missed PAM_CONV, sorry. I'm worried about the sendAuthRequest() > > being done there; it doesn't seem safe to potentially ereport(ERROR) > > and longjmp through a PAM call stack

Advanced Patch Feedback Session / pgconf.dev 2025

2025-03-13 Thread Tomas Vondra
Hi all, pgconf.dev 2025 will host "Advanced Patch Feedback Session", with the same format as in 2024 [1]: Participants will work in small groups with a Postgres committer to analyze a past contribution or attempted contribution, discussing what they could have done better both in the patch

like pg_shmem_allocations, but fine-grained for DSM registry ?

2025-03-13 Thread Florents Tselai
I’ve been working with the DSM registry API. I was wondering if it is possible (it doesn’t look like it) or if it has been discussed: can we expose a view like pg_shmem_allocations, but fine-grained for every named segment (i.e. created by GetNamedDSMSegment )? Currently, there is a "DSM Registr

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-13 Thread Sami Imseih
> If we add something for NULLs, we'll always add distinctClause before > the sortClause. If the distinctClause is NULL we won't end up with the > same jumble bytes as if the sortClause were NULL, as the order the > NULL byte(s) are added to the buffer changes. That is how I understand it as well.

Re: [PATCH] Add reverse(bytea)

2025-03-13 Thread Nathan Bossart
Committed. -- nathan

RE: Improve CRC32C performance on SSE4.2

2025-03-13 Thread Devulapalli, Raghuveer
> > I've gone ahead and added the generated AVX-512 algorithm in > > v14-0005 Here is my benchmark numbers (master v/s v14) on TGL (time in ms): Buf-size-bytes Master v14 64 9.547 6.095 80 10.999 6.248 96 12.443 7.756 112 14.129 9.62 128 15.295 6.534 144 16.7

Re: Changing the state of data checksums in a running cluster

2025-03-13 Thread Tomas Vondra
On 3/13/25 13:32, Daniel Gustafsson wrote: >> On 13 Mar 2025, at 12:03, Tomas Vondra wrote: >> On 3/13/25 10:54, Daniel Gustafsson wrote: On 12 Mar 2025, at 14:16, Tomas Vondra wrote: > I believe the approach is correct, but the number of possible states (e.g. after a crash/restar

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-13 Thread Jacob Champion
On Wed, Mar 12, 2025 at 3:16 PM Jacob Champion wrote: > I missed PAM_CONV, sorry. I'm worried about the sendAuthRequest() > being done there; it doesn't seem safe to potentially ereport(ERROR) > and longjmp through a PAM call stack? But I'll switch those over to > something safe or else drop that

md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-13 Thread Andres Freund
Hi, If there is any LOG/DEBUG elog inside functions like mdreadv(), mdwritev(), mdextend(), be it directly or indirectly, the functions become unsafe. The problem is that at the end of errfinish() there is a CFI: /* * Check for cancel/die interrupt first --- this is so that the u

Re: A few patches to clarify snapshot management

2025-03-13 Thread Heikki Linnakangas
On 06/01/2025 23:30, Heikki Linnakangas wrote: On 20/12/2024 19:31, Heikki Linnakangas wrote: /*  * Struct representing all kind of possible snapshots.  *  * There are several different kinds of snapshots:  * * Normal MVCC snapshots  * * MVCC snapshots taken during recovery (in Hot-Standby mode)

Re: protocol support for labels

2025-03-13 Thread Nico Williams
On Wed, Mar 12, 2025 at 11:39:57PM -0700, Jeremy Schneider wrote: > Isn't multiple queries in one packet only possible with the simple > protocol, but not possible with the extended protocol? So this would be > entirely incompatible with prepared/parameterized statements? It's TCP. Packet and seg

Re: Implement waiting for wal lsn replay: reloaded

2025-03-13 Thread Tomas Vondra
Hi, I did a quick look at this patch. I haven't found any correctness issues, but I have some general review comments and questions about the grammar / syntax. 1) The sgml docs don't really show the syntax very nicely, it only shows this at the beginning of wait_for.sgml: WAIT FOR ( parameter

Re: Some read stream improvements

2025-03-13 Thread Thomas Munro
On Wed, Mar 12, 2025 at 8:29 AM Andres Freund wrote: > On 2025-03-12 07:35:46 +1300, Thomas Munro wrote: > > On Thu, Feb 27, 2025 at 11:20 PM Andres Freund wrote: > > > On 2025-02-27 11:19:55 +1300, Thomas Munro wrote: > > I wonder if we should use temp_buffers - 100? Then leave the minimum GUC

Re: Add support for EXTRA_REGRESS_OPTS for meson

2025-03-13 Thread vignesh C
On Thu, 27 Feb 2025 at 18:51, Andres Freund wrote: > > Hi, > > On 2025-02-27 13:53:17 +0100, Andreas Karlsson wrote: > > We use EXTRA_REGRESS_OPTS to make sure the whole test suite passes with our > > extension loaded and since I prefer develop in meson over using autotools > > and make the lack o

Re: plperl version on the meson setup summary screen

2025-03-13 Thread vignesh C
On Tue, 4 Feb 2025 at 12:07, Zharkov Roman wrote: > > Hello, > > Here is a new patch version. > I tried to use perl 'version' instead of 'api_versionstring' to sync > with configure script. Thanks for the patch, this looks good to me. Regards, Vignesh

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-13 Thread Bertrand Drouvot
Hi, On Tue, Mar 11, 2025 at 06:10:32AM +, Bertrand Drouvot wrote: > WalSndWaitForWal() is being used only for logical walsender. So we'd need to > find another location for the physical walsender case. One option is to keep > the > WalSndLoop() location and control the reports frequency. I

Re: dblink: Add SCRAM pass-through authentication

2025-03-13 Thread Matheus Alcantara
Hi, thanks for all the comments! Attached v5 with some fixes (I'll answer comments for different emails on this since most of them is fixed on this new v5 version) > I think this fix may break the other usage in dblink_get_conn(), where > rconn comes from a hash entry. Maybe dblink_connect() shou

Re: Draft for basic NUMA observability

2025-03-13 Thread Bertrand Drouvot
Hi, On Wed, Mar 12, 2025 at 04:41:15PM +0100, Jakub Wartak wrote: > On Mon, Mar 10, 2025 at 11:14 AM Bertrand Drouvot > wrote: > > > Thanks for the new version! > > v10 is attached with most fixes after review and one new thing > introduced: pg_numa_available() for run-time decision inside test

Re: Separate GUC for replication origins

2025-03-13 Thread vignesh C
On Thu, 13 Mar 2025 at 05:59, Euler Taveira wrote: > > On Wed, Mar 12, 2025, at 8:47 AM, vignesh C wrote: > > I reviewed the discussion on this thread and believe we now have an > agreement on the design and GUC names. However, the patch still needs > updates and extensive testing, especially cons

Re: Allow default \watch interval in psql to be configured

2025-03-13 Thread Greg Sabino Mullane
Patch looks good. One minor issue: greg=# \set WATCH_INTERVAL -1 invalid value "-1" for "WATCH_INTERVAL": must be greater than 0.00 greg=# \set WATCH_INTERVAL 0.00 greg=# We should disallow 0 as the error message implies I've long wanted to be able to set the default interval for \watch in psql

Re: Enhancing Memory Context Statistics Reporting

2025-03-13 Thread Rahila Syed
Hi, Please find attached updated and rebased patches. It has the following changes 1. To prevent memory leaks, ensure that the latest statistics published by a process are freed before it exits. This can be achieved by calling dsa_free in the before_shmem_exit callback. 2. Add a level column to m

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-13 Thread Dilip Kumar
On Thu, Mar 13, 2025 at 3:20 PM Amit Kapila wrote: > > On Thu, Mar 13, 2025 at 11:43 AM Dilip Kumar wrote: > > > > Looks fine, shall we add the missing publication point as well > > something like below > > > > /* > > * In operations like 'ALTER SUBSCRIPTION ... ADD/SET PUBLICATION' and > > * 'CR

Re: Update Unicode data to Unicode 16.0.0

2025-03-13 Thread Peter Eisentraut
On 17.02.25 20:46, Jeff Davis wrote: Note that the Unicode update has a few test diffs for NORMALIZE(), please check to see if the tests themselves need an update. I think these test result changes are incorrect. AFAICT, nothing has changed in the upstream data that would explain such a chang

Re: Missing constraint when duplicated unique index ?

2025-03-13 Thread Marcos Pegoraro
Em qui., 13 de mar. de 2025 às 09:17, Álvaro Herrera < alvhe...@alvh.no-ip.org> escreveu: > I tried this example all the way back to pg 9.5, and they all end up > with a single constraint and a single index -- namely, the test_pk > constraint and corresponding index. There's no second index and n

Re: Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-13 Thread Bertrand Drouvot
Hi, On Thu, Mar 13, 2025 at 07:35:24PM +0800, Xuneng Zhou wrote: > Hi, > Given that PGSTAT_MIN_INTERVAL is the go-to setting for stats flushes > * Unless called with 'force', pending stats updates are flushed happen once > * per PGSTAT_MIN_INTERVAL (1000ms). When not forced, stats flushes do not

Re: Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-13 Thread Bertrand Drouvot
Hi, On Thu, Mar 13, 2025 at 07:33:24PM +0800, Xuneng Zhou wrote: > Regarding patch 0001, the optimization in pgstat_backend_have_pending_cb > looks good: Thanks for looking at it! > bool > pgstat_backend_have_pending_cb(void) > { > - return (!pg_memory_is_all_zeros(&PendingBackendStats, > - size

Re: Orphaned users in PG16 and above can only be managed by Superusers

2025-03-13 Thread Ashutosh Sharma
Hi, On Thu, Mar 13, 2025 at 11:14 AM Ashutosh Sharma wrote: > > Hi, > > On Wed, Mar 12, 2025 at 9:06 PM Nathan Bossart > wrote: > > > > There are also other ways besides DROP ROLE that roles can end up without > > any admins: > > > > postgres=# create role a createrole; > > CREA

Re: pg_attribute_noreturn(), MSVC, C11

2025-03-13 Thread Peter Eisentraut
I committed the first two patches (squashed together) (about pg_noreturn). I had to make one change: I put back the GCC fallback that I had removed between v1 and v2. This is needed for GCC versions before C11 became the default (gcc 5) and also for situations like buildfarm member mylodon th

Re: Allow default \watch interval in psql to be configured

2025-03-13 Thread Laurenz Albe
On Fri, 2025-02-14 at 16:15 +0100, Daniel Gustafsson wrote: > > On 2 Feb 2025, at 19:25, Daniel Gustafsson wrote: > > Attached is rebase with some small levels of polish, unless objected to I > would > like to go ahead with this version. I gave the patch a try, and it works as expected. The cod

Re: Elimination of the repetitive code at the SLRU bootstrap functions.

2025-03-13 Thread Álvaro Herrera
Hello I think this is going too far. The new function BootStrapSlruPage() now is being used for things other than bootstrapping, and that doesn't seem appropriate to me. I think we should leave the things that aren't bootstrap out of this patch. For instance, ExtendCLOG was more understandable

Re: Random pg_upgrade 004_subscription test failure on drongo

2025-03-13 Thread Heikki Linnakangas
On 13/03/2025 11:04, vignesh C wrote: ## Analysis I think it was caused due to the STATUS_DELETE_PENDING failure, not related with recent updates for pg_upgrade. The file "base/1/2683" is an index file for pg_largeobject_loid_pn_index, and the output meant that file creation failed. Below is a b

Re: Test to dump and restore objects left behind by regression

2025-03-13 Thread Ashutosh Bapat
Here are patches missing in the previous email. On Thu, Mar 13, 2025 at 6:09 PM Ashutosh Bapat wrote: > > On Thu, Mar 13, 2025 at 2:12 PM Alvaro Herrera > wrote: > > > > Hello > > > > On 2025-Mar-13, Ashutosh Bapat wrote: > > > > > 1. can you please run the test again and share the dump outputs

Re: Test to dump and restore objects left behind by regression

2025-03-13 Thread Ashutosh Bapat
On Thu, Mar 13, 2025 at 2:12 PM Alvaro Herrera wrote: > > Hello > > On 2025-Mar-13, Ashutosh Bapat wrote: > > > 1. can you please run the test again and share the dump outputs. They > > will be located in a temporary directory with names > > src_dump.sql_adjusted and dest_dump..sql_adjusted. > > A

Re: AIO v2.5

2025-03-13 Thread Antonin Houska
Andres Freund wrote: > Attached is v2.7, with the following changes: Attached are a few proposals for minor comment fixes. Besides that, it occurred to me when I was trying to get familiar with the patch set (respectable work, btw) that an additional Assert() statement could make sense: diff -

Re: Changing the state of data checksums in a running cluster

2025-03-13 Thread Daniel Gustafsson
> On 13 Mar 2025, at 12:03, Tomas Vondra wrote: > On 3/13/25 10:54, Daniel Gustafsson wrote: >>> On 12 Mar 2025, at 14:16, Tomas Vondra wrote: >>> I believe the approach is correct, but the number of possible states >>> (e.g. after a crash/restart) seems a bit complex. I wonder if there's a >>>

Re: support virtual generated column not null constraint

2025-03-13 Thread jian he
hi. new patch attached. 0001 for virtual generated columns not null. minor change to fix the compiler warning. 0002-0004 is for domain over virtual generated columns. 0002: we need to compute the generation expression for the domain with constraints, thus rename ExecComputeStoredGenerated to Exe

Re: Minor rework of ALTER TABLE SET RelOptions code

2025-03-13 Thread Timur Magomedov
On Fri, 2025-03-07 at 20:02 +0300, Nikolay Shaplov wrote: > While working with my New Options Engine patch > https://commitfest.postgresql.org/patch/4688/ > I found out that I can detach a small portion of it as a separate > patch. > It has own value, even if big patch is never committed, and it w

Re: Missing constraint when duplicated unique index ?

2025-03-13 Thread Álvaro Herrera
On 2025-Mar-12, Marcos Pegoraro wrote: > CREATE TABLE table_test ( > foo text NOT NULL, > CONSTRAINT test_pk PRIMARY KEY (foo), > CONSTRAINT test_uq UNIQUE (foo) > ); > building index "pg_toast_29364884_index" on table "pg_toast_29364884" > serially > building index "test_pk" on table

Re: Changing the state of data checksums in a running cluster

2025-03-13 Thread Tomas Vondra
On 3/13/25 10:54, Daniel Gustafsson wrote: >> On 12 Mar 2025, at 14:16, Tomas Vondra wrote: > >> I continued investigating this and experimenting with alternative >> approaches, and I think the way the patch relies on ControlFile is not >> quite right. That is, it always sets data_checksum_versio

Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-13 Thread Xuneng Zhou
Sorry, forgot to cc this one too. -- Forwarded message - 发件人: Xuneng Zhou Date: 2025年3月13日周四 19:29 Subject: Re: [BUG]: the walsender does not update its IO statistics until it exits To: Bertrand Drouvot Hi, Given that PGSTAT_MIN_INTERVAL is the go-to setting for stats flushes

Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-03-13 Thread Xuneng Zhou
Forgot to cc... -- Forwarded message - 发件人: Xuneng Zhou Date: 2025年3月13日周四 19:15 Subject: Re: [BUG]: the walsender does not update its IO statistics until it exits To: Bertrand Drouvot Hi, Thanks for working on this! I'm glad to see that the patch ( https://www.postgresql.org/

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2025-03-13 Thread Oliver Ford
On Sun, 9 Mar 2025, 20:07 Oliver Ford, wrote: > On Sun, Mar 9, 2025 at 6:40 AM Tatsuo Ishii wrote: > >> > Attached version removes the non-nulls array. That seems to speed >> > everything up. Running the above query with 1 million rows averages >> 450ms, >> > similar when using lead/lag. >> >>

Re: Selectively invalidate caches in pgoutput module

2025-03-13 Thread Amit Kapila
On Wed, Mar 12, 2025 at 5:46 PM Hayato Kuroda (Fujitsu) wrote: > > > Thanks, the patch looks mostly good to me. I have made few cosmetic > > changes in the attached and combined both the patches. > > Thanks, it looks good to me. > > > The existing > > Alter Publication ... Rename tests don't test

Re: Conflict detection for multiple_unique_conflicts in logical replication

2025-03-13 Thread Nisha Moond
Hi, For v3, CFbot failed for the patch test case due to unstable test steps. Truncating the subscriber table resolved the apply worker error but sometimes caused out-of-order inserts, leading to unexpected failures instead of a conflict error. Fixed this and updated the LOG regex check to avoid fa

Re: Draft for basic NUMA observability

2025-03-13 Thread Jakub Wartak
On Wed, Mar 12, 2025 at 4:41 PM Jakub Wartak wrote: > > On Mon, Mar 10, 2025 at 11:14 AM Bertrand Drouvot > wrote: > > > Thanks for the new version! > > v10 is attached with most fixes after review and one new thing > introduced: pg_numa_available() for run-time decision inside tests > which was

Re: meson vs. llvm bitcode files

2025-03-13 Thread Nazir Bilal Yavuz
Hi, On Wed, 12 Mar 2025 at 16:39, Diego Fronza wrote: > > Hi, > > The v7 patch looks good to me, handling the bitcode modules in a uniform way > and also avoiding the hacky code and warnings, much better now. > > A small note about the bitcode emission for generated sources in contrib, > using

Re: Parallel safety docs for CTEs

2025-03-13 Thread Kirill Reshke
On Wed, 12 Mar 2025 at 22:11, James Coleman wrote: > > On Tue, Nov 19, 2024 at 2:16 PM James Coleman wrote: > > > > Hello, > > > > A colleague noticed today that the docs still say that "Scans of > > common table expressions (CTEs)" are "always parallel restricted". > > > > While I think that str

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-03-13 Thread Amit Kapila
On Thu, Mar 13, 2025 at 11:43 AM Dilip Kumar wrote: > > Looks fine, shall we add the missing publication point as well > something like below > > /* > * In operations like 'ALTER SUBSCRIPTION ... ADD/SET PUBLICATION' and > * 'CREATE SUBSCRIPTION', if the specified publication does not exist or > *

Re: Changing the state of data checksums in a running cluster

2025-03-13 Thread Daniel Gustafsson
> On 12 Mar 2025, at 14:16, Tomas Vondra wrote: > I continued investigating this and experimenting with alternative > approaches, and I think the way the patch relies on ControlFile is not > quite right. That is, it always sets data_checksum_version to the last > ("current") value, but that's not

Re: PoC: Simplify recovery after dropping a table by LOGGING the restore LSN

2025-03-13 Thread Kirill Reshke
On Fri, 8 Nov 2024 at 08:47, Kirk Wolak wrote: > > Hackers, > The concept was driven by an all too common support request. A user > accidentally dropped the wrong table. (this could also be applied to > dropping a database, etc). > > If we had the LSN before the drop, this would be easier.

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-03-13 Thread Jakub Wartak
On Wed, Mar 12, 2025 at 9:02 PM Melanie Plageman wrote: > > Thanks for taking a look. I've pushed the patch to increase the > default effective_io_concurrency. Cool, anything > 1 is just better. Just quick question, so now we have: #define DEFAULT_EFFECTIVE_IO_CONCURRENCY 16 #define DEFAULT_MAIN

Random pg_upgrade 004_subscription test failure on drongo

2025-03-13 Thread vignesh C
Hi, In one of the recent buildfarm runs, the test pg_upgrade/004_subscription test fails at [1]. It has failed with the following: Restoring database schemas in the new cluster *failure* Consult the last few lines of "C:/prog/bf/root/HEAD/pgsql.build/testrun/pg_upgrade/004_subscription/data/t_004

Re: relfilenode statistics

2025-03-13 Thread Kirill Reshke
On Fri, 3 Jan 2025 at 21:18, Bertrand Drouvot wrote: > As mentioned by Andres in [1], relying on the relation OID would not work to > "recover" the stats because we don't have access to the relation oid during > crash > recovery. So, I'm going to resume working on the "initial" idea (i.e having

Re: [Patch] remove duplicated smgrclose

2025-03-13 Thread Steven Niu
在 2025/3/12 6:31, Masahiko Sawada 写道: On Mon, Mar 10, 2025 at 3:08 AM Steven Niu wrote: Hi, Masahiko Thanks for your comments! I understand your concern as you stated. However, my initial patch was split into two parts as Kirill suggested. This thread is about the first part. Another p

Re: Test to dump and restore objects left behind by regression

2025-03-13 Thread Alvaro Herrera
Hello On 2025-Mar-13, Ashutosh Bapat wrote: > 1. can you please run the test again and share the dump outputs. They > will be located in a temporary directory with names > src_dump.sql_adjusted and dest_dump..sql_adjusted. Ah, I see the problem :-) The first initdb does this: # Running

  1   2   >