Re: Horribly slow pg_upgrade performance with many Large Objects

2025-07-14 Thread Tom Lane
Nathan Bossart writes: > Here is what I have staged for commit, which (barring feedback or > objections) I am planning to do towards the end of the week. Is it intentional that this does +#include "catalog/pg_largeobject_metadata.h" +#include "catalog/pg_shdepend.h" rather than including the co

Re: Improve LWLock tranche name visibility across backends

2025-07-14 Thread Nathan Bossart
On Mon, Jul 14, 2025 at 02:34:00PM -0500, Sami Imseih wrote: >> Why do we need three different places for the lock names? Is there a >> reason we can't put it all in shared memory? > > The real reason I felt it was better to keep three separate locations is that > it allows for a clear separation

RE: Changing shared_buffers without restart

2025-07-14 Thread Jack Ng
If I understanding correctly, putting a new buffer in the freelist before updating NBuffers could break existing logic that calls BufferIsValid(bufnum) and asserts bufnum <= NBuffers? (since a backend can grab the new buffer and checks its validity before the coordinator can add it to the freeli

Re: Changing shared_buffers without restart

2025-07-14 Thread Andres Freund
Hi, On July 14, 2025 10:39:33 AM EDT, Dmitry Dolgov <9erthali...@gmail.com> wrote: >> On Mon, Jul 14, 2025 at 10:23:23AM -0400, Andres Freund wrote: >> > Those steps are separated in time, and I'm currently trying to understand >> > what are the consequences of performing them in different order

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 10:22:17AM -0400, Burd, Greg wrote: > I'd initialize them, expand BufferDescriptors, and adjust NBuffers. The > clock-sweep algorithm will eventually find them and make use of them. The > buf->freeNext should be FREENEXT_NOT_IN_LIST so that StrategyFreeBuffer() will > do

Re: Improve LWLock tranche name visibility across backends

2025-07-14 Thread Tom Lane
Nathan Bossart writes: > Ah, I missed the problem with postmaster. Could we have the first backend > that needs to access the table be responsible for creating it and > populating it with the built-in/requested-at-startup entries? Also, is > there any chance that postmaster might need to access

Re: POC: Parallel processing of indexes in autovacuum

2025-07-14 Thread Masahiko Sawada
On Sun, Jul 6, 2025 at 5:00 PM Daniil Davydov <3daniss...@gmail.com> wrote: > > Hi, > > On Fri, Jul 4, 2025 at 9:21 PM Matheus Alcantara > wrote: > > > > The "autovacuum_max_parallel_workers" declared on guc_tables.c mention > > that is capped by "max_worker_process": > > + { > > +

Re: [PATCH] Generate random dates/times in a specified range

2025-07-14 Thread Dean Rasheed
On Sat, 12 Jul 2025 at 16:15, Greg Sabino Mullane wrote: > > I like the idea, especially the date variant. Unlike Tom, I'm not > particularly concerned about breakage of existing scripts, as > most already are working just fine with raw numbers and I don't see this > patch breaking them. > > In

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread Peter Smith
On Fri, Jul 11, 2025 at 7:18 PM Japin Li wrote: > ... > > Hi, Peter > > 1. > I'm curious if you encountered the following warning during compilation: > > /home/japin/Codes/pg/master/build/../contrib/vci/include/vci_ros.h:745:9: > warning: result of comparison of constant 65536 with expression of

Re: pg_logical_slot_get_changes waits continously for a partial WAL record spanning across 2 pages

2025-07-14 Thread Michael Paquier
On Mon, Jun 30, 2025 at 05:51:51PM +0530, vignesh C wrote: > Thanks, Kuroda-san. I’ve prepared a similar test that doesn’t rely on > injection points. The issue reproduced consistently across all > branches up to PG13. You can use the attached > 049_slot_get_changes_wait_continously_pg17.pl script

Re: ScanKeys passed to table_beginscan in SeqNext

2025-07-14 Thread Tom Lane
Josh Innis writes: > Is there a plan to have scan keys passed to table_beginscan in SeqNext? and > if not, is there a reason why? I know the heap access method has support > for it. AFAIK the only part of the system that relies on HeapKeyTest to do anything useful is the catcache, which has cases

Re: speedup COPY TO for partitioned table.

2025-07-14 Thread jian he
On Mon, Jul 14, 2025 at 10:02 PM Álvaro Herrera wrote: > > On 2025-Jul-02, jian he wrote: > > > @@ -673,11 +680,34 @@ BeginCopyTo(ParseState *pstate, > >errmsg("cannot copy from sequence > > \"%s\"", > >

Re: [PATCH] Generate random dates/times in a specified range

2025-07-14 Thread Robert Treat
On Mon, Jul 14, 2025 at 3:21 AM Dean Rasheed wrote: > On Sat, 12 Jul 2025 at 16:15, Greg Sabino Mullane wrote: > > > > I like the idea, especially the date variant. Unlike Tom, I'm not > > particularly concerned about breakage of existing scripts, as > > most already are working just fine with r

Re: Suggestion to add --continue-client-on-abort option to pgbench

2025-07-14 Thread Yugo Nagata
Hi, On Sun, 13 Jul 2025 23:15:24 +0900 Rintaro Ikeda wrote: > I noticed one small thing I’d like to discuss. I'm not sure that users clearly > understand which aborted in the following error message, the client or the > script. > > pgbench: error: client 0 script 0 aborted in command ... query

ScanKeys passed to table_beginscan in SeqNext

2025-07-14 Thread Josh Innis
Hello PG Devs, For an extension I am working on. I want to create a new storage access method. To do this, I would like/need to get scan_keys passed to table_beginscan in SeqNext. For reference, the comments in SeqRecheck /* * Note that unlike IndexScan, SeqScan never use keys in heap_beginscan

Re: Some ExecSeqScan optimizations

2025-07-14 Thread Amit Langote
Hi, On Fri, Jul 11, 2025 at 11:34 PM Andres Freund wrote: > On 2025-07-11 11:22:36 +0900, Amit Langote wrote: > > On Fri, Jul 11, 2025 at 5:55 AM Andres Freund wrote: > > > On 2025-07-10 17:28:50 +0900, Amit Langote wrote: > > > > Thanks for the patch. > > > > > > > > +/* > > > > + * Use

Re: Requested WAL segment xxx has already been removed

2025-07-14 Thread Japin Li
On Mon, 14 Jul 2025 at 20:33, Fujii Masao wrote: > On 2025/07/14 17:08, Japin Li wrote: >> Hi all, >> I recently hit an error with our streaming replication setup: >>2025-07-14 11:52:59.361 >> CST,"replicator","",728458,"10.9.9.74:35724",68747f1b.b1d8a,1,"START_REPLICATION",2025-07-14 >> 11:52

Re: pg_dump does not dump domain not-null constraint's comments

2025-07-14 Thread jian he
On Tue, Jul 15, 2025 at 2:24 AM Álvaro Herrera wrote: > > On 2025-May-22, jian he wrote: > > > I actually found another bug. > > create schema test; > > CREATE DOMAIN test.d1 AS integer NOT NULL default 11; > > pg_dump --schema=test > 1.sql > > "" > > pg_dump: warning: could not resolve dependenc

Re: Disable parallel query by default

2025-07-14 Thread Laurenz Albe
On Mon, 2025-07-14 at 17:25 -0400, Greg Sabino Mullane wrote: > Tom wrote: > > > BTW, I would say largely the same things about JIT > > Yeah, that would change this from a few people conversing over tea > into a large angry mob bearing pitchforks. Oh yes, I'd much rather disable JIT by default th

Re: Missing NULL check after calling ecpg_strdup

2025-07-14 Thread Aleksander Alekseev
Hi Alvaro, > This looks super baroque. Why not simplify a bit instead? Maybe > something like > > [...] Fair point. Here is the corrected patch. From 35cbf8fa22d0a1e9d1b46784a83adfdfd5c675fb Mon Sep 17 00:00:00 2001 From: Aleksander Alekseev Date: Fri, 11 Jul 2025 17:59:50 +0300 Subject: [PATC

RE: Changing shared_buffers without restart

2025-07-14 Thread Jack Ng
Just brain-storming here... would moving NBuffers to shared memory solve this specific issue? Though I'm pretty sure that would open up a new set of synchronization issues elsewhere, so I'm not sure if there's a net gain. Jack >-Original Message- >From: Andres Freund >Sent: Monday, Jul

Re: POC: make mxidoff 64 bits

2025-07-14 Thread Maxim Orlov
Yet another rebase @ f5a987c0e5 -- Best regards, Maxim Orlov. v16-0001-Use-64-bit-format-output-for-multixact-offsets.patch Description: Binary data v16-0002-Use-64-bit-multixact-offsets.patch Description: Binary data v16-0004-Get-rid-of-MultiXactMemberFreezeThreshold-call.patch Description

Re: domain check constraint should also consider domain's collation

2025-07-14 Thread Tom Lane
jian he writes: > CREATE COLLATION case_insensitive (provider = icu, locale = > '@colStrength=secondary', deterministic = false); > SELECT 'a' = 'A' COLLATE case_insensitive; > CREATE DOMAIN d1 as text collate case_insensitive check (value <> 'a'); > SELECT 'A'::d1; > ``SELECT 'A'::d1`` should er

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 03:18:10PM +, Jack Ng wrote: > Just brain-storming here... would moving NBuffers to shared memory solve this > specific issue? Though I'm pretty sure that would open up a new set of > synchronization issues elsewhere, so I'm not sure if there's a net gain. It's in f

Re: Missing NULL check after calling ecpg_strdup

2025-07-14 Thread Álvaro Herrera
On 2025-Jul-14, Aleksander Alekseev wrote: > @@ -460,7 +461,21 @@ ECPGconnect(int lineno, int c, const char *name, const > char *user, const char *p >*/ > conn_keywords = (const char **) ecpg_alloc((connect_params + 1) * > sizeof(char *), lineno); > conn_values = (const char

Re: speedup COPY TO for partitioned table.

2025-07-14 Thread torikoshia
On 2025-07-02 13:10, jian he wrote: Thanks for updating the patch. now it is: /*  * Scan a single table (which may be a partition) and export its rows to the  * COPY destination. Based on the explanations in the glossary, should 'parition' be partitioned table/relation? | -- https://www.

Re: Changing shared_buffers without restart

2025-07-14 Thread Andres Freund
Hi, On 2025-07-14 15:20:03 +0200, Dmitry Dolgov wrote: > > On Mon, Jul 14, 2025 at 09:14:26AM -0400, Andres Freund wrote: > > > > Clock sweep can find any buffer, independent of whether it's on the > > > > freelist. > > > > > > It does the search based on nextVictimBuffer, where the actual buffer

Re: Log prefix missing for subscriber log messages received from publisher

2025-07-14 Thread Fujii Masao
On 2025/04/15 13:37, vignesh C wrote: Hi, Currently, when a warning is emitted by the publisher, the corresponding log message does not include the log prefix. This makes it harder to correlate such messages with other log entries. For example, in a simulated error scenario where directory re

Re: small fix for pg_overexplain docs

2025-07-14 Thread David G. Johnston
On Mon, Jul 14, 2025 at 12:10 PM Nathan Bossart wrote: > I noticed that pg_overexplain's documentation leads off with this sentence: > > The pg_overexplain extends EXPLAIN with... > > Presumably that's meant to be something like: > > The pg_overexplain MODULE extends EXPLAIN with.

Re: Improve LWLock tranche name visibility across backends

2025-07-14 Thread Sami Imseih
>> Attached is a proof of concept that does not alter the >> LWLockRegisterTranche API. > IMHO we should consider modifying the API, because right now you have to > call LWLockRegisterTranche() in each backend. Why not accept the name as > an argument for LWLockNewTrancheId() and only require it t

pg_overexplain extension name

2025-07-14 Thread Bruce Momjian
I was thinking about the name of our new PG 18 pg_overexplain extension. "Over-explain" has a negative connotation, like how can over-explaining something be useful? Do we want that as the name of this new extension? -- Bruce Momjian https://momjian.us EDB

Re: Requested WAL segment xxx has already been removed

2025-07-14 Thread Japin Li
On Mon, 14 Jul 2025 at 10:21, Alexander Kukushkin wrote: > On Mon, 14 Jul 2025 at 10:08, Japin Li wrote: > > Hi all, > > I recently hit an error with our streaming replication setup: > >2025-07-14 11:52:59.361 > > CST,"replicator","",728458,"10.9.9.74:35724",68747f1b.b1d8a,1,"START_REPLIC

Re: Changing shared_buffers without restart

2025-07-14 Thread Thom Brown
On Mon, 14 Jul 2025, 09:54 Dmitry Dolgov, <9erthali...@gmail.com> wrote: > > On Mon, Jul 14, 2025 at 01:55:39PM +0530, Ashutosh Bapat wrote: > > > You're right of course, a buffer id could be returned from the > > > ClockSweep and from the custom strategy buffer ring. Buf from what I > see > > > t

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 10:24:50AM +0100, Thom Brown wrote: > On Mon, 14 Jul 2025, 09:54 Dmitry Dolgov, <9erthali...@gmail.com> wrote: > > > > On Mon, Jul 14, 2025 at 01:55:39PM +0530, Ashutosh Bapat wrote: > > > > You're right of course, a buffer id could be returned from the > > > > ClockSweep

Re: Logical replication prefetch

2025-07-14 Thread Amit Kapila
On Sun, Jul 13, 2025 at 6:06 PM Konstantin Knizhnik wrote: > > On 13/07/2025 1:28 pm, Amit Kapila wrote: > > On Tue, Jul 8, 2025 at 12:06 PM Konstantin Knizhnik > > wrote: > >> There is well known Postgres problem that logical replication subscriber > >> can not caught-up with publisher just bec

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-07-14 Thread Ryo Kanbayashi
On Mon, Jul 14, 2025 at 9:10 AM Michael Paquier wrote: > > On Thu, Jul 10, 2025 at 02:21:38PM +0900, Michael Paquier wrote: > > After that, I have applied a few cosmetic tweaks here and there, and > > attached is what I have staged for commit, minus proper commit > > messages. The new TAP tests h

Re: Logical replication prefetch

2025-07-14 Thread Amit Kapila
On Sun, Jul 13, 2025 at 5:59 PM Konstantin Knizhnik wrote: > > > Certainly originally intended use case was different: parallel apply is > performed only for large transactions. Number of of such transactions is > not so big and > so there should be enough parallel apply workers in pool to proceed

Re: The same 2PC data maybe recovered twice

2025-07-14 Thread suyu.cmj
> Actually, what you are doing is incorrect because we could miss some > ERRORs for example if a base backup was incorrect if come files were > present in pg_twophase? Yes, you are right, base backup with incorrect files would be a problem. > It's not really true that what you are changing here has

Re: Report bytes and transactions actually sent downtream

2025-07-14 Thread Amit Kapila
On Mon, Jul 14, 2025 at 10:55 AM Ashutosh Bapat wrote: > > On Sun, Jul 13, 2025 at 4:34 PM Amit Kapila wrote: > > > > > > I think we don't want to make it mandatory for plugins to implement > > these stats, so instead of throwing ERROR, the view should show that > > the plugin doesn't provide sta

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-14 Thread Ajin Cherian
On Tue, Jul 8, 2025 at 8:41 PM Ajin Cherian wrote: > Proposed fix: > In process_syncing_tables_for_apply(), acquire an AccessExclusiveLock > on SubscriptionRelRelationId before acquiring the lock on > ReplicationOriginRelationId. > > Patch with fix attached. > I'll continue investigating whether

RE: Logical replication prefetch

2025-07-14 Thread Zhijie Hou (Fujitsu)
On Monday, July 14, 2025 2:36 PM Konstantin Knizhnik wrote: > On 14/07/2025 4:20 am, Zhijie Hou (Fujitsu) wrote: > > Additionally, I was also exploring ways to improve performance and have > > tried an > > alternative version of prefetch for experimentation. The alternative design > > is > > tha

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread shveta malik
On Mon, Jul 14, 2025 at 2:08 PM Peter Smith wrote: > > Hi Shveta, > > Thanks for your README questions. > > On Fri, Jul 11, 2025 at 1:46 PM shveta malik wrote: > > > > Thank You for working on this. I started going through the README and > > tried running simple tests, have few concerns: > > > >

Requested WAL segment xxx has already been removed

2025-07-14 Thread Japin Li
Hi all, I recently hit an error with our streaming replication setup: 2025-07-14 11:52:59.361 CST,"replicator","",728458,"10.9.9.74:35724",68747f1b.b1d8a,1,"START_REPLICATION",2025-07-14 11:52:59 CST,3/0,0,ERROR,58P01,"requested WAL segment 0001000C has already been removed"

Re: Changing shared_buffers without restart

2025-07-14 Thread Ashutosh Bapat
On Mon, Jul 14, 2025 at 1:40 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Mon, Jul 14, 2025 at 10:25:51AM +0530, Ashutosh Bapat wrote: > > > Currently it's being addressed via every backend waiting for each other, > > > but I guess it could be as well managed via handling the freelist,

Re: ABI Compliance Checker GSoC Project

2025-07-14 Thread Mankirat Singh
On Mon, 14 Jul 2025 at 00:32, Tom Lane wrote: > Another idea could be an in-tree file, different in each branch, > that records the hash of the commit we presently want to compare to. > This would require a small amount of additional manual effort to > maintain, but maybe it's the most flexible wa

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 10:25:51AM +0530, Ashutosh Bapat wrote: > > Currently it's being addressed via every backend waiting for each other, > > but I guess it could be as well managed via handling the freelist, so > > that only "available" buffers will be inserted into the lookup table. > > I di

Re: The same 2PC data maybe recovered twice

2025-07-14 Thread Michael Paquier
On Mon, Jul 14, 2025 at 02:46:25PM +0800, CAI, Mengjuan wrote: > Thank you for your reply. I reviewed the thread you mentioned, and > it seems that the issue needing to be fixed is not the same as the > one I previously raised. > I am considering whether the 2PC file check logic in > PrepareRedoAdd

Re: track needed attributes in plan nodes for executor use

2025-07-14 Thread Andrei Lepikhov
On 14/7/2025 06:52, Amit Langote wrote: On Fri, Jul 11, 2025 at 11:09 PM Andrei Lepikhov wrote: To support that efficiently, the slot can store a new tts_min_valid field to indicate the lowest attno that needs deforming. Alternatively, we could use a per-attribute flag array (with TupleDesc->nat

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-07-14 Thread Masahiro Ikeda
On 2025-07-04 20:32, Fujii Masao wrote: On 2025/06/02 16:32, Masahiro Ikeda wrote: OK, I think v5-0002 should be back-patched, since using incorrect error codes is essentially a bug. Thanks for updating the patches! While reviewing the code: amtup = SearchSysCache1(AMOID, Ob

Re: Requested WAL segment xxx has already been removed

2025-07-14 Thread Alexander Kukushkin
On Mon, 14 Jul 2025 at 10:08, Japin Li wrote: > > Hi all, > > I recently hit an error with our streaming replication setup: > > 2025-07-14 11:52:59.361 > CST,"replicator","",728458,"10.9.9.74:35724",68747f1b.b1d8a,1,"START_REPLICATION",2025-07-14 > 11:52:59 CST,3/0,0,ERROR,58P01,"requested WAL

domain check constraint should also consider domain's collation

2025-07-14 Thread jian he
hi. CREATE COLLATION case_insensitive (provider = icu, locale = '@colStrength=secondary', deterministic = false); SELECT 'a' = 'A' COLLATE case_insensitive; CREATE DOMAIN d1 as text collate case_insensitive check (value <> 'a'); SELECT 'A'::d1; ``SELECT 'A'::d1`` should error out as domain check

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread Peter Smith
Hi Shveta, Thanks for your README questions. On Fri, Jul 11, 2025 at 1:46 PM shveta malik wrote: > > Thank You for working on this. I started going through the README and > tried running simple tests, have few concerns: > > 1) > I am not able to understand section 4.2 'WOS-to-ROS conversion'. Wh

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-07-14 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 14 Jul 2025 03:28:16 +0900, Masahiko Sawada wrote: > I've reviewed the 0001 and 0002 patches. The API implemented in the > 0002 patch looks good to me, but I'm concerned about the capsulation > of copy

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread Peter Smith
Hi Japin, Thanks for your README questions. On Fri, Jul 11, 2025 at 7:18 PM Japin Li wrote: ... > > 3. > In the README, 'TID' seems to have conflicting definitions: > Transaction ID (2.1) vs. tuple physical identifier (2.3.1). > > Could you confirm the intended meaning? Suggest using 'XID' for T

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 01:55:39PM +0530, Ashutosh Bapat wrote: > > You're right of course, a buffer id could be returned from the > > ClockSweep and from the custom strategy buffer ring. Buf from what I see > > those are picking a buffer from the set of already utilized buffers, > > meaning that

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-14 Thread Ajin Cherian
On Tue, Jul 8, 2025 at 8:41 PM Ajin Cherian wrote: > > Patch with fix attached. > I'll continue investigating whether this issue also affects HEAD. > While debugging if this problem can occur on HEAD, I found out that on head, it is mostly the tablesync worker that drops the origin on HEAD and si

Re: Improving and extending int128.h to more of numeric.c

2025-07-14 Thread John Naylor
On Thu, Jul 10, 2025 at 9:06 PM Dean Rasheed wrote: > > On Wed, 9 Jul 2025 at 22:31, Dean Rasheed wrote: > > > > On Wed, 9 Jul 2025 at 18:27, Andres Freund wrote: > > > > > > I think we should wire this up to the buildsystem and our testsuite... > > > Having > > > testcode that is not run auto

Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

2025-07-14 Thread Japin Li
On Mon, 14 Jul 2025 at 18:47, Peter Smith wrote: > Hi Japin, > > Thanks for your README questions. > > On Fri, Jul 11, 2025 at 7:18 PM Japin Li wrote: > ... >> >> 3. >> In the README, 'TID' seems to have conflicting definitions: >> Transaction ID (2.1) vs. tuple physical identifier (2.3.1). >> >>

Re: 024_add_drop_pub.pl might fail due to deadlock

2025-07-14 Thread vignesh C
On Mon, 14 Jul 2025 at 15:46, Ajin Cherian wrote: > > On Tue, Jul 8, 2025 at 8:41 PM Ajin Cherian wrote: > > > > Patch with fix attached. > > I'll continue investigating whether this issue also affects HEAD. > > > > While debugging if this problem can occur on HEAD, I found out that on > head, it

Re: Problem with transition tables on partitioned tables with foreign-table partitions

2025-07-14 Thread Etsuro Fujita
Amit-san, On Thu, Jul 10, 2025 at 11:54 PM Amit Langote wrote: > On Thu, Jul 10, 2025 at 22:20 Amit Langote wrote: >> On Wed, Jul 9, 2025 at 5:07 PM Etsuro Fujita wrote: >> > Here is an updated version of the patch, in which I added 1) an Assert >> > to ExecAR* functions to ensure that the pass

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-07-14 Thread Fujii Masao
On 2025/07/14 17:13, Masahiro Ikeda wrote: On 2025-07-04 20:32, Fujii Masao wrote: On 2025/06/02 16:32, Masahiro Ikeda wrote: OK, I think v5-0002 should be back-patched, since using incorrect error codes is essentially a bug. Thanks for updating the patches! While reviewing the code:   

Re: POC: Parallel processing of indexes in autovacuum

2025-07-14 Thread Daniil Davydov
Hi, On Mon, Jul 14, 2025 at 2:10 PM Masahiko Sawada wrote: > > --- > - shared->maintenance_work_mem_worker = > - (nindexes_mwm > 0) ? > - maintenance_work_mem / Min(parallel_workers, nindexes_mwm) : > - maintenance_work_mem; > + > + if (AmAutoVacuumWorkerProcess()) > +

Re: Requested WAL segment xxx has already been removed

2025-07-14 Thread Fujii Masao
On 2025/07/14 17:08, Japin Li wrote: Hi all, I recently hit an error with our streaming replication setup: 2025-07-14 11:52:59.361 CST,"replicator","",728458,"10.9.9.74:35724",68747f1b.b1d8a,1,"START_REPLICATION",2025-07-14 11:52:59 CST,3/0,0,ERROR,58P01,"requested WAL segment 00010

sql/json query function JsonBehavior default expression's collation may differ from returning type's collation

2025-07-14 Thread jian he
hi. create domain d1 as text collate case_insensitive; select json_value('{"a": "A"}', '$.a' returning d1 default 'C' on empty) = 'a'; --return true select json_value('{"a": "A"}', '$.c' returning d1 default 'A' on empty) = 'a'; --return true select json_value('{"a": "A"}', '$.c' returning d1 def

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-07-14 Thread Nathan Bossart
Here is what I have staged for commit, which (barring feedback or objections) I am planning to do towards the end of the week. -- nathan >From 9501f31cdf3ce25012cec8f9e2d7c433df749979 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Mon, 14 Jul 2025 11:17:47 -0500 Subject: [PATCH v4 1/1] pg_u

Re: fix organization wording in psql's \copyright command

2025-07-14 Thread Tom Lane
Nathan Bossart writes: > It looks like \copyright didn't get updated in commit 5987553. I plan to > back-patch this down to v13 once the tag for 18beta2 is pushed. +1. I wonder if we missed any other places ... grepping for "Postgres95" didn't find any, but ... regards,

Re: pg_overexplain extension name

2025-07-14 Thread Aleksander Alekseev
Hi Bruce, > I was thinking about the name of our new PG 18 pg_overexplain extension. > "Over-explain" has a negative connotation, like how can over-explaining > something be useful? Do we want that as the name of this new extension? I think it was an intended pun and to my ears it's sort of funn

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 10:23:23AM -0400, Andres Freund wrote: > > Those steps are separated in time, and I'm currently trying to understand > > what are the consequences of performing them in different order and whether > > there are possible concurrency issues under various scenarios. Does this

Re: Compression of bigger WAL records

2025-07-14 Thread Andrey Borodin
> On 31 Jan 2025, at 08:37, wenhui qiu wrote: > > Hi Andery > I have a question ,If wal_compression_threshold is set to more than the > block size of the wal log, then the FPI is not compressed, and if so, it > might make sense to have a maximum value of this parameter that does not >

Re: AIO v2.5

2025-07-14 Thread Andres Freund
Hi, On 2025-07-11 23:03:53 +0200, Tomas Vondra wrote: > I've been running some benchmarks comparing the io_methods, to help with > resolving this PG18 open item. So here are some results, and my brief > analysis of it. Thanks for doing that! > The TL;DR version > - > > * The "

Re: Disable parallel query by default

2025-07-14 Thread Tom Lane
"Scott Mead" writes: > I'd like to re-open the discussion for this commitfest item. I still have > not been able to find a value for parallel_setup_cost that makes good > decisions about parallelism on a user's behalf. I believe that setting the > SIGHUP-able max_parallel_workers_per_gather t

Re: pg_dump does not dump domain not-null constraint's comments

2025-07-14 Thread Tom Lane
=?utf-8?Q?=C3=81lvaro?= Herrera writes: > I wonder to what extent we promise ABI compatibility of pg_dump.h > structs in stable branches. Those should be private to pg_dump, so I see no objection to changing them. regards, tom lane

Re: AIO v2.5

2025-07-14 Thread Andres Freund
On 2025-07-13 20:04:51 +0200, Tomas Vondra wrote: > On 7/11/25 23:03, Tomas Vondra wrote: > > ... > > > > e) indexscan regression (ryzen-indexscan-uniform-pg17-checksums.png) > > > > There's an interesting difference difference I noticed in the run with > > checksums on PG17. The full PDF is availa

Re: pg_overexplain extension name

2025-07-14 Thread Peter Geoghegan
On Mon, Jul 14, 2025 at 5:23 AM Bruce Momjian wrote: > I was thinking about the name of our new PG 18 pg_overexplain extension. > "Over-explain" has a negative connotation, like how can over-explaining > something be useful? Over-explaining something can be somewhat useful if the person doing the

Re: Improving and extending int128.h to more of numeric.c

2025-07-14 Thread Dean Rasheed
On Mon, 14 Jul 2025 at 11:22, John Naylor wrote: > > (Looking at v3) The new test module runs 10 million rather than a > billion iterations. That still takes 1.2s (after 0005), which seems > excessive for regular buildfarm testing. It seems like we could get by > with fewer than that, by using the

Re: Improve LWLock tranche name visibility across backends

2025-07-14 Thread Sami Imseih
> Ah, I missed the problem with postmaster. Could we have the first backend > that needs to access the table be responsible for creating it and > populating it with the built-in/requested-at-startup entries? We can certainly maintain a flag in the shared state that is set once the first backend l

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-07-14 Thread Nathan Bossart
On Mon, Jul 14, 2025 at 01:28:03PM -0400, Tom Lane wrote: > Is it intentional that this does > > +#include "catalog/pg_largeobject_metadata.h" > +#include "catalog/pg_shdepend.h" > > rather than including the corresponding *_d.h headers? Nope, that was an oversight. -- nathan >From 99551525801

Re: Disable parallel query by default

2025-07-14 Thread Scott Mead
On Wed, May 21, 2025, at 10:55 AM, Scott Mead wrote: > > > On Wed, May 21, 2025, at 3:50 AM, Laurenz Albe wrote: > > On Tue, 2025-05-20 at 16:58 -0400, Scott Mead wrote: > > > On Wed, May 14, 2025, at 4:06 AM, Laurenz Albe wrote: > > > > On Tue, 2025-05-13 at 17:53 -0400, Scott Mead wrote: > >

Re: pg_dump does not dump domain not-null constraint's comments

2025-07-14 Thread Álvaro Herrera
On 2025-May-22, jian he wrote: > I actually found another bug. > create schema test; > CREATE DOMAIN test.d1 AS integer NOT NULL default 11; > pg_dump --schema=test > 1.sql > "" > pg_dump: warning: could not resolve dependency loop among these items: > pg_dump: detail: TYPE d1 (ID 1415 OID 18007

fix organization wording in psql's \copyright command

2025-07-14 Thread Nathan Bossart
It looks like \copyright didn't get updated in commit 5987553. I plan to back-patch this down to v13 once the tag for 18beta2 is pushed. diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c index a2e009ab9be..8c62729a0d1 100644 --- a/src/bin/psql/help.c +++ b/src/bin/psql/help.c @@ -748,7 +748,

Re: Inline non-SQL SRFs using SupportRequestSimplify

2025-07-14 Thread Tom Lane
Paul Jungwirth writes: > I tried a few refactoring approaches but the nicest seemed to be to keep the > shared parts in > inline_set_returning_function, but have it call out to either > inline_sql_set_returning_function or > inline_set_returning_function_with_support. The first patch just refa

Re: Disable parallel query by default

2025-07-14 Thread Greg Sabino Mullane
(picking up the thread again too) >> Five minutes?! That's not been my experience. Not claiming parallelism is perfect yet, >> but there are plenty of parallel performance savings under the five minute mark. > Absolutely, I've seen 1 second queries go to 200ms with parallelism of 2. The problem

Re: Huge commitfest app update upcoming: Tags, Draft CF, Help page, and automated commitfest creat/open/close

2025-07-14 Thread Aleksander Alekseev
Hi, > > It seems like cfbot.cputube.org started to miss a *few* entries since > > CF started. Does it have anything to do with the CF application > > update? > > That's fixed now. Many thanks. I also noticed that cfbot doesn't show entries from the *upcoming* commitfest. I believe it did before a

Re: Improve LWLock tranche name visibility across backends

2025-07-14 Thread Nathan Bossart
On Mon, Jul 14, 2025 at 03:45:02PM -0500, Sami Imseih wrote: >> Ah, I missed the problem with postmaster. Could we have the first backend >> that needs to access the table be responsible for creating it and >> populating it with the built-in/requested-at-startup entries? > > We can certainly main

Re: Missing NULL check after calling ecpg_strdup

2025-07-14 Thread Aleksander Alekseev
Hi Michael, > Should we actually check sqlca_t more seriously if failing one of the > strdup calls used for the port, host, etc. when attempting the > connection? The ecpg_log() assumes that a NULL value equals a > , which would be wrong if we failed one of these allocations > on OOM. If I read

Re: Changing shared_buffers without restart

2025-07-14 Thread Andres Freund
Hi, On 2025-07-14 11:32:25 +0200, Dmitry Dolgov wrote: > > On Mon, Jul 14, 2025 at 10:24:50AM +0100, Thom Brown wrote: > > On Mon, 14 Jul 2025, 09:54 Dmitry Dolgov, <9erthali...@gmail.com> wrote: > > > > > > On Mon, Jul 14, 2025 at 01:55:39PM +0530, Ashutosh Bapat wrote: > > > > > You're right of

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 08:56:56AM -0400, Andres Freund wrote: > > Ah, I see what you mean folks. But I'm talking here only about buffers > > which will be allocated after extending shared memory -- they must go > > through the freelist first (I don't see why not, any other options?), > > and cl

Re: Changing shared_buffers without restart

2025-07-14 Thread Andres Freund
Hi, On 2025-07-14 15:08:28 +0200, Dmitry Dolgov wrote: > > On Mon, Jul 14, 2025 at 08:56:56AM -0400, Andres Freund wrote: > > > Ah, I see what you mean folks. But I'm talking here only about buffers > > > which will be allocated after extending shared memory -- they must go > > > through the free

Re: Missing NULL check after calling ecpg_strdup

2025-07-14 Thread Aleksander Alekseev
Hi, > Here is the corrected patch v3. Changes since v2: > > ``` > for (con = all_connections; con != NULL; con = con->next) > { > - /* XXX strcmp() will segfault if con->name is NULL */ > - if (strcmp(connection_name, con-

Re: Make wal_receiver_timeout configurable per subscription

2025-07-14 Thread Fujii Masao
On 2025/05/28 0:36, Fujii Masao wrote: On 2025/05/22 21:21, Amit Kapila wrote: On Wed, May 21, 2025 at 6:04 PM Fujii Masao wrote: On 2025/05/20 18:13, vignesh C wrote: If we set the wal_receiver_timeout configuration using ALTER ROLE for the subscription owner's role, the apply worker wi

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 09:14:26AM -0400, Andres Freund wrote: > > > Clock sweep can find any buffer, independent of whether it's on the > > > freelist. > > > > It does the search based on nextVictimBuffer, where the actual buffer > > will be a modulo of NBuffers, right? If that's correct and I

Re: Missing NULL check after calling ecpg_strdup

2025-07-14 Thread Aleksander Alekseev
Hi, > While working on it I noticed a potentially problematic strcmp call, > marked with XXX in the patch. I didn't address this issue in v2. Here is the corrected patch v3. Changes since v2: ``` for (con = all_connections; con != NULL; con = con->next) { -

Re: track needed attributes in plan nodes for executor use

2025-07-14 Thread Tom Lane
Amit Langote writes: > Not quite -- the optimization doesn’t require changes to the tuple > header or representation. The existing deforming code already stops > once all requested attributes are filled, using tts_nvalid to track > that. What I’m proposing is to additionally allow the slot to skip

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 09:42:46AM -0400, Andres Freund wrote: > What on earth would be the point of putting a buffer on the freelist but not > make it reachable by the clock sweep? To me that's just nonsensical. To clarify, we're not talking about this scenario as "that's how it would work afte

Re: failover logical replication slots

2025-07-14 Thread Fabrice Chapuis
Thanks for this feedback, I'll remove the check on the sync_replication_slots parameter. I think it is interesting as you suggest to start with the idea of the pg_alter_replication_slot API, I will make a new proposal by opening a new thread, hoping to be supported in my approach. Is there already

Re: Changing shared_buffers without restart

2025-07-14 Thread Dmitry Dolgov
> On Mon, Jul 14, 2025 at 11:11:36AM -0400, Andres Freund wrote: > Hi, > > On July 14, 2025 10:39:33 AM EDT, Dmitry Dolgov <9erthali...@gmail.com> wrote: > >> On Mon, Jul 14, 2025 at 10:23:23AM -0400, Andres Freund wrote: > >> > Those steps are separated in time, and I'm currently trying to underst

Re: Add progressive backoff to XactLockTableWait functions

2025-07-14 Thread Xuneng Zhou
Hi, I spent some time digging deeper into the KnownAssignedXids logic in procarray.c and made a few refinements. Below are some observations and questions I’d like to discuss. 1) Present usage (AFAICS) It appears that logical decoding is the only place that waits for an XID on a standby. The si

Re: pg_overexplain extension name

2025-07-14 Thread David G. Johnston
On Mon, Jul 14, 2025 at 11:55 AM Peter Geoghegan wrote: > On Mon, Jul 14, 2025 at 5:23 AM Bruce Momjian wrote: > > I was thinking about the name of our new PG 18 pg_overexplain extension. > > "Over-explain" has a negative connotation, like how can over-explaining > > something be useful? > > Ove

Re: pg_overexplain extension name

2025-07-14 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Jul 14, 2025 at 5:23 AM Bruce Momjian wrote: >> Do we want that as the name of this new extension? > Personally, I think it's fine. We ought to discourage the idea that > this is just "explain, but better". I agree that it's a tad cutesy, but the time to compla

small fix for pg_overexplain docs

2025-07-14 Thread Nathan Bossart
I noticed that pg_overexplain's documentation leads off with this sentence: The pg_overexplain extends EXPLAIN with... Presumably that's meant to be something like: The pg_overexplain MODULE extends EXPLAIN with... Another pattern in adjacent pages is to leave out the "The":

  1   2   >