Re: Conflict detection for update_deleted in logical replication

On Wed, Mar 26, 2025 at 4:17 PM Zhijie Hou (Fujitsu) wrote: > > Here's a rebased version of the patch series. > Thanks for the patches. While testing the GUC "max_conflict_retention_duration", I noticed a behavior that seems to bypass its intended purpose. On Pub, if a txn is stuck in the COMMI

Re: jsonapi: scary new warnings with LTO enabled

Daniel Gustafsson writes: >> On 16 Apr 2025, at 23:42, Tom Lane wrote: >> I'm not sure >> how other than giving up on stack allocation of JsonLexContexts, >> though, especially if we consider the jsonapi API frozen. But seeing >> that there are only three such call sites and none of them seem in

Re: use correct variable in error message in _allocAH function (pg_backup_archiver.c)

On 2025/04/16 13:30, Fujii Masao wrote: On 2025/04/15 23:37, Tom Lane wrote: Fujii Masao writes: Thanks for the report and patch! It looks good to me. Agreed. Since this issue exists in the back branches, the patch needs be back-patched to all supported versions. I doubt it's worth

Re: Make prep_status() message translatable

On 2025/04/07 16:26, Fujii Masao wrote: On 2025/04/07 15:55, Kyotaro Horiguchi wrote: Hello, The recent commit 173c97812ff made the following change: -    prep_status("Adding \".old\" suffix to old global/pg_control"); +    prep_status("Adding \".old\" suffix to old " XLOG_CONTROL_FILE);

Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding

On Thu, Apr 3, 2025 at 11:01 PM Shlok Kyal wrote: > > Hi Ajin, > > I have reviewed patch v16-0001, here are my comments: > > 1. There are some places where comments are more than 80 columns > window. I think it should be <=80 as per [1]. Fixed. > > 2. I think, 'rb->unfiltered_changes_count' shou

Re: Skipping schema changes in publication

On Wed, Apr 16, 2025 at 8:22 AM Zhijie Hou (Fujitsu) wrote: > > On Thu, Apr 10, 2025 at 7:25 PM Amit Kapila wrote: > > > > On Tue, Jan 9, 2024 at 12:02 PM vignesh C wrote: > > > > > > As I did not see much interest from others, I'm withdrawing this patch > > > for now. But if there is any interes

Decouple C++ support in Meson's PGXS from LLVM enablement

Howdy folks, While playing around with pg_duckdb[0], a Postgres extension written in C++ which uses PGXS, I came across a strange build error: std=c++17 -Wno-sign-compare... /bin/sh: line 1: -Wno-sign-compare: command not found I was very confused by the error, but reading the command line, it

Re: Fixup some appendPQExpBuffer() calls

On Mon, 14 Apr 2025 at 19:39, David Rowley wrote: > 1) I should commit the attached 0002 patch now, or; > 2) Should commit only the new-to-v18 ones now, or; > 3) do nothing. > > I think #3 isn't a good option as we (or I) have made efforts in the > past to keep these in check. Also, for me, #2 isn

Re: New committer: Jacob Champion

Congratulations Jacob! Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com On Sat, 12 Apr 2025 at 1:56 AM, Jonathan S. Katz wrote: > The Core Team would like to extend our congratulations to Jacob > Champion, who has accepted an invitation to become our newest PostgreSQL > committer

Re: jsonapi: scary new warnings with LTO enabled

Daniel Gustafsson writes: > On 17 Apr 2025, at 00:12, Tom Lane wrote: >> The only alternative I can see that might stop the warning is if we >> can find a way to make it clearer to the optimizer that the FREE() >> isn't reached. But I'm not sure about a trustworthy way to make that >> happen. >

Re: jsonapi: scary new warnings with LTO enabled

> On 16 Apr 2025, at 23:42, Tom Lane wrote: > It seems fairly dangerous to ignore -Wfree-nonheap-object warnings. > I feel like we ought to move to prevent these somehow. Absolutely agree. > I'm not sure > how other than giving up on stack allocation of JsonLexContexts, > though, especially if

Re: Built-in Raft replication

On Wed, Apr 16, 2025 at 6:27 AM Tom Lane wrote: > > Andrey Borodin writes: > > I think it's what Konstantin is proposing. To have our own Raft > > implementation, without dependencies. > > Hmm, OK. I thought that the proposal involved relying on some existing > code, but re-reading the thread t

Re: Align memory context level numbering in pg_log_backend_memory_contexts()

On Thu, 17 Apr 2025 at 02:20, torikoshia wrote: > Regarding the implementation: > In the initial patch attached, I naïvely incremented the level just > before emitting the log line. > However, it might be cleaner to simply initialize the level variable to > 1 from the start. This could help avoid

Re: pipelining in psql, commit 41625ab

On Wed, Apr 16, 2025 at 09:46:42AM -0700, Michael Paquier wrote: > On Tue, Apr 15, 2025 at 02:34:50PM -0700, Noah Misch wrote: > > On Fri, Feb 21, 2025 at 11:33:41AM +0900, Michael Paquier wrote: > > commit 41625ab wrote: > >> --- a/src/bin/psql/help.c > >> +++ b/src/bin/psql/help.c > >> @@ -167,15

Re: jsonapi: scary new warnings with LTO enabled

On Wed, Apr 16, 2025 at 4:04 PM Tom Lane wrote: > Looking through all of the callers of freeJsonLexContext, quite > a lot of them use local JsonLexContext structs, and probably some > of them are more performance-critical than these. So that raises > the question of why are we seeing warnings for

Re: pg_dump: Fix dangling pointer in EndCompressorZstd()

Michael Paquier writes: > On Wed, Apr 16, 2025 at 04:19:02PM +0200, Daniel Gustafsson wrote: >> Agreed, while it's perfectly safe today the end method should not make >> assumptions about the use of the private_data pointer upon return and should >> leave it set to NULL. > Indeed. I was just loo

Re: pg_dump: Fix dangling pointer in EndCompressorZstd()

On Wed, Apr 16, 2025 at 04:19:02PM +0200, Daniel Gustafsson wrote: > Agreed, while it's perfectly safe today the end method should not make > assumptions about the use of the private_data pointer upon return and should > leave it set to NULL. Indeed. I was just looking at applying what Alexander

Re: Rename injection point names in test_aio

On Mon, Apr 14, 2025 at 11:14:59AM +, Hayato Kuroda (Fujitsu) wrote: >> Anyway, would you like to propose a patch for the documentation? > > Sure, I did. Please see [1]. > > [1]: > https://www.postgresql.org/message-id/OSCPR01MB14966E14C1378DEE51FB7B7C5F5B32%40OSCPR01MB14966.jpnprd01.prod.ou

Re: Support for runtime parameters in injection points, for AIO tests

On Mon, Apr 14, 2025 at 04:05:40AM -0400, Andres Freund wrote: > On 2025-04-14 16:46:22 +0900, Michael Paquier wrote: >> We are still in early April, and I'd like to propose a cleanup of the >> AIO code on HEAD, even if we are post-feature freeze, to not release >> this new code in this state, impl

Re: jsonapi: scary new warnings with LTO enabled

> On 17 Apr 2025, at 00:12, Tom Lane wrote: > > Daniel Gustafsson writes: >>> On 16 Apr 2025, at 23:42, Tom Lane wrote: >>> I'm not sure >>> how other than giving up on stack allocation of JsonLexContexts, >>> though, especially if we consider the jsonapi API frozen. But seeing >>> that there

Re: jsonapi: scary new warnings with LTO enabled

Em qua., 16 de abr. de 2025 às 18:42, Tom Lane escreveu: > I noticed some new warnings from buildfarm member chafer, > which I'm able to reproduce locally on a Fedora 41 box > by building with "meson setup build -Db_lto=true": > > ninja: Entering directory `build' > [1515/2472] Linking target src

jsonapi: scary new warnings with LTO enabled

I noticed some new warnings from buildfarm member chafer, which I'm able to reproduce locally on a Fedora 41 box by building with "meson setup build -Db_lto=true": ninja: Entering directory `build' [1515/2472] Linking target src/interfaces/libpq/libpq.so.5.18 In function 'freeJsonLexContext',

Prototype: Implement dead tuples xid histograms

Hi hackers, in a recent hacking workshop organized by Robert Haas, we discussed [1]. Among the autovacuum issues exposed, the useless vacuum case caught my attention. I've started to study the respective code and I came up with a prototype to improve the statistics system regarding dead tuples. T

Re: Built-in Raft replication

* Greg Sabino Mullane [25/04/16 22:33]: > > Users find it a waste of resources to deploy 3 big PostgreSQL instances > > just for HA where 2 suffice even if they deploy 3 lightweight DCS > > instances. Having only some of the nodes act as DCS and others purely > > PostgreSQL nodes will reduce waste

Re: not null constraints, again

Alvaro Herrera writes: > Here's another version where I do skip searching for children twice, and > rewrote the comments. v2 LGTM, with two small nits: 1. Grammar feels shaky here: + * normal case where we're asked to recurse, this routine ensures that the + * not-null constraints either exist

Re: Built-in Raft replication

On Wed, Apr 16, 2025 at 2:18 AM Ashutosh Bapat wrote: > Users find it a waste of resources to deploy 3 big PostgreSQL instances > just for HA where 2 suffice even if they deploy 3 lightweight DCS > instances. Having only some of the nodes act as DCS and others purely > PostgreSQL nodes will reduc

Re: Performance issues with v18 SQL-language-function changes

Bruce Momjian writes: > On Wed, Apr 16, 2025 at 01:43:46PM -0400, Tom Lane wrote: >> I'm confused? 0dca5d68d didn't have anything to do with >> syntax changes, just with when planning happens. > I was referencing the contrib initialization functions you converted to > use SQL-standard function b

Re: Performance issues with v18 SQL-language-function changes

On Wed, Apr 16, 2025 at 01:43:46PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, Apr 14, 2025 at 10:38:29AM -0400, Robert Haas wrote: > >> I agree that we should do something about this. I haven't reviewed > >> your patches but the approach sounds broadly reasonable. > > > Yep, we we

Re: A modest proposal: make parser/rewriter/planner inputs read-only

On 4/16/25 18:22, Tom Lane wrote: Maybe we would like to have some enforced contract about what subquery_planner can and can't touch in the outer planner level's data, but I'm not feeling a great need for that right now. I wouldn't say I'm entirely convinced that side effects are absent, but mul

Re: recoveryCheck test failure on flaviventris

Dear Kuroda-san, 16.04.2025 11:12, Hayato Kuroda (Fujitsu) wrote: Dear Richard, Alexander, FYI - I felt there is a same type of failure [1] for REL_17_STABLE, added in the wikipage. Feel free to revert it if I did something wrong. [1]:https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl

Re: Performance issues with v18 SQL-language-function changes

Hi st 16. 4. 2025 v 19:43 odesílatel Tom Lane napsal: > Bruce Momjian writes: > > On Mon, Apr 14, 2025 at 10:38:29AM -0400, Robert Haas wrote: > >> I agree that we should do something about this. I haven't reviewed > >> your patches but the approach sounds broadly reasonable. > > > Yep, we went

Re: pg_dump --if-exists --clean when drop index that is partition of a partitioned index

jian he writes: > On Wed, Apr 16, 2025 at 6:51 AM Tom Lane wrote: >> Or we could do what Jian suggested and just not emit any dropStmt >> for child indexes. I continue to fear that that will have >> undesirable side-effects, but I have to admit that I'm not sure >> what. The fact that the backe

Re: pgsql: Non text modes for pg_dumpall, correspondingly change pg_restore

On 2025-Apr-16, Andrew Dunstan wrote: > Pushed your patch plus cleanup of the sscanf stuff. Thank you! -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "El sabio habla porque tiene algo que decir; el tonto, porque tiene que decir algo" (Platon).

Re: Performance issues with v18 SQL-language-function changes

Bruce Momjian writes: > On Mon, Apr 14, 2025 at 10:38:29AM -0400, Robert Haas wrote: >> I agree that we should do something about this. I haven't reviewed >> your patches but the approach sounds broadly reasonable. > Yep, we went down the road in PG 18 to convert syntax, and now we have > to fix

Re: Performance issues with v18 SQL-language-function changes

On Mon, Apr 14, 2025 at 10:38:29AM -0400, Robert Haas wrote: > On Sun, Apr 13, 2025 at 3:23 PM Tom Lane wrote: > > create function fx(p_summa bigint) returns text immutable strict > > return ltrim(to_char(p_summa, '999 999 999 999 999 999 999 999')); > > > > explain analyze select fx(i) from gener

Re: pipelining in psql, commit 41625ab

On Tue, Apr 15, 2025 at 02:34:50PM -0700, Noah Misch wrote: > On Fri, Feb 21, 2025 at 11:33:41AM +0900, Michael Paquier wrote: > commit 41625ab wrote: >> * \syncpipeline queues a synchronisation request, without flushing the >> commands to the server, equivalent of PQsendPipelineSync(). >>

Re: pgsql: Non text modes for pg_dumpall, correspondingly change pg_restore

On 2025-04-14 Mo 8:20 AM, Álvaro Herrera wrote: On 2025-Apr-04, Andrew Dunstan wrote: Non text modes for pg_dumpall, correspondingly change pg_restore I think the new oid_string_list stuff in this commit is unnecessary, and we can remove a bunch of lines by simplifying that to using SimplePt

Re: A modest proposal: make parser/rewriter/planner inputs read-only

Andrei Lepikhov writes: > I just want to understand how your idea will work. The query_planner > does the job for subqueries separately. If a query is transformed in > some way (let's say, an unnecessary join is deleted), we need to change > references in the parse tree of another subquery, or

Re: Add estimated hit ratio to Memoize in EXPLAIN to explain cost adjustment

On Tue, Apr 15, 2025 at 1:50 PM David Rowley wrote: > On Wed, 16 Apr 2025 at 04:25, Robert Haas wrote: > > On Mon, Apr 14, 2025 at 8:23 PM David Rowley wrote: > > > "Estimates: capacity=N distinct keys=N lookups=N hit ratio=N.N%" > > > > Is lookups=N here the estimated number of lookups i.e. wha

Re: Add Pipelining support in psql

On Wed, Apr 16, 2025 at 09:31:59PM +0700, a.kozhemyakin wrote: > After commit 2cce0fe on master > > ERROR:  unexpected message type 0x50 during COPY from stdin > CONTEXT:  COPY psql_pipeline, line 1 > Pipeline aborted, command did not run > psql: common.c:1510: discardAbortedPipelineResults: Asser

Re: transforms [was Re: FmgrInfo allocation patterns (and PL handling as staged programming)]

On 04/15/25 23:39, Tom Lane wrote: > My own beef with the whole setup is that you can't specify *which* > arguments or results you want transformed. I don't believe that > this should have been keyed off data types to begin with. But > that design was the SQL committee's choice so we're stuck ...

Re: Built-in Raft replication

16.04.2025 07:58, Andrey Borodin пишет: > Yes, shared DCS are common these days. AFAIK, we use one Zookeeper instance > per hundred Postgres clusters to coordinate pg_consuls. > > Actually, scalability is opposite to topic of this thread. Let me explain. > Currently, Postgres automatic failover t

Re: Add Pipelining support in psql

Hello, After commit 2cce0fe on master When executing query: psql postgres

Re: Built-in Raft replication

16.04.2025 08:24, Andrey Borodin пишет: > 2. After failover, old Primary node must rejoin cluster by running pg_rewind > and following timeline switch. It is really do-able: BiHA already does it. And BiHA runs as a child process of postmaster, ie both postmaster and BiHA doesn't restart when Post

Re: Typos in the comment for the estimate_multivariate_ndistinct()

> On 14 Apr 2025, at 15:34, Tender Wang wrote: > > Hi, > > While reading the estimate_multivariate_ndistinct(), > I think "If a match it found, *varinfos is > * updated to remove the list of matched varinfos" > should be "If a match is found, *varinfos is > * updated to remove the list of mat

Re: Align memory context level numbering in pg_log_backend_memory_contexts()

On 2025-04-16 06:18, Daniel Gustafsson wrote: On 15 Apr 2025, at 23:03, David Rowley wrote: My vote is to make the levels 1-based in all locations where we output the context information. I agree with this, pg_get_process_memory_contexts() also use 1-based levels fwiw. +1. I believe the

Re: pg_dump: Fix dangling pointer in EndCompressorZstd()

> On 16 Apr 2025, at 13:48, Ashutosh Bapat wrote: > > On Wed, Apr 16, 2025 at 1:57 PM Alexander Kuznetsov > wrote: >> >> Hello everyone, >> >> We've found that EndCompressorZstd() doesn't set cs->private_data to NULL >> after pg_free(), >> unlike other EndCompressor implementations. >> While

Re: Typos in the comment for the estimate_multivariate_ndistinct()

Tender Wang 于2025年4月14日周一 21:34写道： > Hi, > > While reading the estimate_multivariate_ndistinct(), > I think "If a match it found, *varinfos is > * updated to remove the list of matched varinfos" > should be "If a match is found, *varinfos is > * updated to remove the list of matched varinfos"

Re: Built-in Raft replication

* Alastair Turner [25/04/16 15:58]: > > > If you use build-in failover you have to resort to 3 big Postgres > > machines because you need 2/3 majority. Of course, you can install > > MySQL-stype arbiter - host that had no real PGDATA, only participates in > > voting. But this is a solution to pro

Re: NUMA shared memory interleaving

On Wed, Apr 16, 2025 at 5:14 AM Jakub Wartak wrote: > Normal pgbench workloads tend to be not affected, as each backend > tends to touch just a small partition of shm (thanks to BAS > strategies). Some remaining questions are: > 1. How to name this GUC (numa or numa_shm_interleave) ? I prefer the

Re: NUMA shared memory interleaving

On Thu, Apr 17, 2025 at 1:58 AM Thomas Munro wrote: > I have no answers but I have speculated for years about a very > specific case (without any idea where to begin due to lack of ... I > guess all this sort of stuff): in ExecParallelHashJoinNewBatch(), > workers split up and try to work on diffe

Re: NUMA shared memory interleaving

On Wed, Apr 16, 2025 at 9:14 PM Jakub Wartak wrote: > 2. Should we also interleave DSA/DSM for Parallel Query? (I'm not an > expert on DSA/DSM at all) I have no answers but I have speculated for years about a very specific case (without any idea where to begin due to lack of ... I guess all this

Re: Built-in Raft replication

On Wed, 16 Apr 2025 at 07:18, Ashutosh Bapat wrote: > On Wed, Apr 16, 2025 at 10:29 AM Andrey Borodin > wrote: > > > > If you use build-in failover you have to resort to 3 big Postgres > machines because you need 2/3 majority. Of course, you can install > MySQL-stype arbiter - host that had no r

Re: not null constraints, again

On 2025-Apr-16, Tender Wang wrote: > if (conForm->contype != CONSTRAINT_NOTNULL) > elog(ERROR, "constraint %u is not a not-null constraint", conForm->oid); > > I feel that using conForm->conname is more friendly than oid for users. Yeah, this doesn't really matter because this function would

Re: not null constraints, again

Alvaro Herrera 于2025年4月16日周三 19:24写道： > Here's another version where I do skip searching for children twice, and > rewrote the comments. > > I also noticed that in child tables we were only looking for > pg_attribute.attnotnull, and not whether the constraints had been > validated or made inherit

Re: pg_dump: Fix dangling pointer in EndCompressorZstd()

On Wed, Apr 16, 2025 at 1:57 PM Alexander Kuznetsov wrote: > > Hello everyone, > > We've found that EndCompressorZstd() doesn't set cs->private_data to NULL > after pg_free(), > unlike other EndCompressor implementations. > While this doesn't currently cause issues (as the pointer soon gets > re

Re: not null constraints, again

Here's another version where I do skip searching for children twice, and rewrote the comments. I also noticed that in child tables we were only looking for pg_attribute.attnotnull, and not whether the constraints had been validated or made inheritable. This seemed a wasted opportunity, so I refac

POC: Parallel processing of indexes in autovacuum

Hi! The VACUUM command can be executed with the parallel option. As documentation states, it will perform index vacuum and index cleanup phases of VACUUM in parallel using *integer* background workers. But such an interesting feature is not used for an autovacuum. After a quick look at the source

Re: Built-in Raft replication

* Andrey Borodin [25/04/16 11:06]: > > You can run bash from extension, what's the point? > > You cannot run bash that will stop backend running bash. You're right there is a chicken and egg problem when you add Raft to an existing project, and rebootstrap becomes a trick, but it's a plumbing

Re: Built-in Raft replication

* Andrey Borodin [25/04/16 11:06]: > > Andrey Borodin writes: > >> I think it's what Konstantin is proposing. To have our own Raft > >> implementation, without dependencies. > > > > Hmm, OK. I thought that the proposal involved relying on some existing > > code, but re-reading the thread that

Re: Built-in Raft replication

* Ashutosh Bapat [25/04/16 11:06]: > > My view is what Konstantin wants is automatic replication topology > > management. For some reason this technology is called HA, DCS, Raft, Paxos > > and many other scary words. But basically it manages primary_conn_info of > > some nodes to provide some f

Re: Fundamental scheduling bug in parallel restore of partitioned tables

On Wed, Apr 16, 2025 at 12:19 AM Tom Lane wrote: > > Ashutosh Bapat writes: > > On Mon, Apr 14, 2025 at 11:14 PM Tom Lane wrote: > >> This is disastrous for assorted reasons. The ALTER ADD CONSTRAINT > >> command might fail outright if we've loaded data for the referencing > >> table but not th

Re: Built-in Raft replication

* Tom Lane [25/04/16 11:05]: > Nikolay Samokhvalov writes: > > This is exactly what I wanted to write as well. The idea is great. At the > > same time, I think, consensus on many decisions will be extremely hard to > > reach, so this project has a high risk of being very long. Unless it's an > >

NUMA shared memory interleaving

Thanks to having pg_numa.c, we can now simply address problem#2 of NUMA imbalance from [1] pages 11-14, by interleaving shm memory in PG19 - patch attached. We do not need to call numa_set_localalloc() as we only interleave shm segments, while local allocations stay the same (well, "local" means re

pg_dump: Fix dangling pointer in EndCompressorZstd()

Hello everyone, We've found that EndCompressorZstd() doesn't set cs->private_data to NULL after  pg_free(), unlike other EndCompressor implementations. While this doesn't currently cause issues (as the pointer soon gets reassigned), we recommend fixing this to maintain consistency with other impl

RE: recoveryCheck test failure on flaviventris

Dear Richard, Alexander, FYI - I felt there is a same type of failure [1] for REL_17_STABLE, added in the wikipage. Feel free to revert it if I did something wrong. [1]: https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=prion&dt=2025-04-15%2016%3A16%3A06&stg=recovery-check Best reg

Re: POC: make mxidoff 64 bits

I moved the topic to the next commitfest. -- Best regards, Maxim Orlov.

Re: Built-in Raft replication

Hi, On Wed, Apr 16, 2025 at 10:24:48AM +0500, Andrey Borodin wrote: > I think I can provide some reasons why it cannot be neither extension, > nor any part running within postmaster reign. > > 1. When joining cluster, there’s not PGDATA to run postmaster on top > of it. > > 2. After failover, ol

Re: Add partial :-variable expansion to psql \copy

Hello, Here is a rebased and updated version of adding \gi: - improved documentation - some refactoring to share one function - signal trap disabled on \gi as well The thread subject is now a misnomer, I'm not sure whether I should update it or start a new thread. On 05/04/2025 22:14, Fabien

Re: A modest proposal: make parser/rewriter/planner inputs read-only

On 4/15/25 19:34, Tom Lane wrote: Andrei Lepikhov writes: But what is the way you are proposing here? Do you mean that one more entity will be explicitly introduced: a transformed parse tree? No, I wasn't thinking of adding new concepts, just saying that the inputs to certain operations need

Re: recoveryCheck test failure on flaviventris

On Wed, Apr 16, 2025 at 1:00 PM Alexander Lakhin wrote: > 16.04.2025 06:05, Richard Guo wrote: > On Wed, Apr 16, 2025 at 11:54 AM Richard Guo wrote: > I noticed this recoveryCheck test failure on flaviventris after > pushing commit 3b35f9a4c (Fix an incorrect check in get_memoize_path). > ... > I