Re: index prefetching

2025-07-20 Thread Nazir Bilal Yavuz
Hi, On Mon, 21 Jul 2025 at 03:59, Thomas Munro wrote: > > On Sun, Jul 20, 2025 at 1:07 AM Thomas Munro wrote: > > On Sat, Jul 19, 2025 at 11:23 PM Tomas Vondra wrote: > > > The thing that however concerns me is that what I observed was not the > > > distance getting reset to 1, and then ramping

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 11:24 AM shveta malik wrote: > > On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > > > I'm continuing to think it through and will share any further thoughts > > if something comes to mind. > > > > How about a parameter named 'on_last_logical_slot' with possible > v

Re: Logical Replication of sequences

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 11:15 AM Dilip Kumar wrote: > > 3. Some of the syntaxes works for sequence which doesn't make sense to > me, as listed below, I think there are more > > postgres[154731]=# CREATE PUBLICATION insert_only FOR ALL SEQUENCES > WITH (publish = 'insert'); > CREATE PUBLICATION > >

Re: Skipping schema changes in publication

2025-07-20 Thread Peter Smith
Hi Shlok. Some review comments for patch v16-0003. == Commit message 1. The column "prexcept" of system catalog "pg_publication_rel" is set to "true" when publication is created with EXCEPT table or EXCEPT column list. If column "prattrs" of system catalog "pg_publication_rel" is also set or

Re: add function argument name to substring and substr

2025-07-20 Thread jian he
On Tue, Apr 1, 2025 at 10:41 PM David G. Johnston wrote: > > On Tue, Apr 1, 2025 at 6:15 AM Marcos Pegoraro wrote: >> >> Em ter., 1 de abr. de 2025 às 02:00, David G. Johnston >> escreveu: >> >> Wouldn't it be good to add the use of parentheses using posix ? It's useful >> and rarely documente

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > > > > > One concern I have is regarding the default setting of > > > 'force_slot_drop' . I assume the default value of this new DROP-SUB > > > argument will be 'false' to prevent cu

Re: Proposal: Out-of-Order NOTIFY via GUC to Improve LISTEN/NOTIFY Throughput

2025-07-20 Thread Joel Jacobson
On Mon, Jul 21, 2025, at 00:06, Rishu Bagga wrote: >> "Joel Jacobson" writes: >> >> Unfortunately, the author of [1] jumped to conclusion and assumed >> the global lock was the problem. I'm quite sure it is probably not, >> because: >> >> We know for sure, that current users do LISTEN and NOTIFY >

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-20 Thread Dilip Kumar
On Mon, Jul 21, 2025 at 10:08 AM shveta malik wrote: > > On Sat, Jul 19, 2025 at 5:10 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 11:31 AM Dilip Kumar wrote: > > > > > > On Fri, Jul 18, 2025 at 11:25 AM shveta malik > > > wrote: > > > > > > > > Okay. I see your point. Yes, it was no

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-20 Thread Amit Kapila
On Mon, Jul 21, 2025 at 10:08 AM shveta malik wrote: > > On Sat, Jul 19, 2025 at 5:10 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 11:31 AM Dilip Kumar wrote: > > > > > > On Fri, Jul 18, 2025 at 11:25 AM shveta malik > > > wrote: > > > > > > > > Okay. I see your point. Yes, it was no

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Mon, Jul 21, 2025 at 10:48 AM shveta malik wrote: > > On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila > > > wrote: > > > > > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta ma

Re: Logical Replication of sequences

2025-07-20 Thread Dilip Kumar
On Mon, Jul 21, 2025 at 10:36 AM Dilip Kumar wrote: > > I was just trying a different test, so I realized that ALTER > PUBLICATION ADD SEQUENCE is not supported, any reason for the same? > > postgres[154731]=# ALTER PUBLICATION pub ADD sequence s1; > ERROR: 42601: invalid publication object list

Re-archive the WAL on standby with archive_mode=always?

2025-07-20 Thread Japin Li
Hi, hackers, I was recently reviewing the KeepFileRestoredFromArchive() function and came across a section that raised a question for me: /* * Create .done file forcibly to prevent the restored segment from being * archived again later. */ if (XLogArchiveMode != ARCHIVE_M

Re: Document slot's restart_lsn can go backward

2025-07-20 Thread Amit Kapila
On Fri, Jul 18, 2025 at 5:11 PM Alexander Korotkov wrote: > > While working on the patch fixing the situation when slot's > restart_lsn ends up pointing to a removed WAL segment [1], we > discovered that sometimes slot's restart_lsn can go backward > [2][3][4]. Hayato Kuroda proposed a patch [5],

Add support for specifying tables in pg_createsubscriber.

2025-07-20 Thread Shubham Khanna
Hi hackers, Currently, pg_createsubscriber supports converting streaming replication to logical replication for selected databases or all databases. However, there is no provision to replicate only a few selected tables. For such cases, users are forced to manually set up logical replication using

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-07-20 Thread shveta malik
On Fri, Jul 18, 2025 at 3:03 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 2:27 PM shveta malik wrote: > > > > On Thu, Jul 17, 2025 at 3:06 PM Amit Kapila wrote: > > > > > > On Wed, Jun 18, 2025 at 3:23 PM shveta malik > > > wrote: > > > > > > > > On Wed, Jun 18, 2025 at 2:39 PM Bertrand D

Re: Logical Replication of sequences

2025-07-20 Thread Dilip Kumar
On Sun, Jul 20, 2025 at 7:48 PM vignesh C wrote: > > On Fri, 18 Jul 2025 at 14:11, Dilip Kumar wrote: > > > > On Fri, Jul 18, 2025 at 10:44 AM Dilip Kumar wrote: > > > > > > On Thu, Jul 17, 2025 at 4:52 PM vignesh C wrote: > > > > > > > > I was looking at the high level idea of sequence sync wo

Re: Improve pg_sync_replication_slots() to wait for primary to advance

2025-07-20 Thread shveta malik
On Sat, Jul 19, 2025 at 5:10 PM Amit Kapila wrote: > > On Fri, Jul 18, 2025 at 11:31 AM Dilip Kumar wrote: > > > > On Fri, Jul 18, 2025 at 11:25 AM shveta malik > > wrote: > > > > > > Okay. I see your point. Yes, it was non-blocking earlier but it was > > > not giving ERROR, it was just dumpin

Re: Conflict detection for update_deleted in logical replication

2025-07-20 Thread Amit Kapila
On Sat, Jul 19, 2025 at 10:32 AM Amit Kapila wrote: > > On Sat, Jul 19, 2025 at 3:01 AM Masahiko Sawada wrote: > > > > On Fri, Jul 18, 2025 at 5:03 AM Zhijie Hou (Fujitsu) > > wrote: > > > > > > > Here are some review comments and questions: > > > > > > --- > > + if (inCommitOnly && > > +

Re: array_random

2025-07-20 Thread Tom Lane
jian he writes: > Does performance and other factors justify adding array_random? There hasn't really been field demand for such a function, AFAIR. There's an infinite amount of stuff that would be faster if coded in C than if written in SQL or plpgsql. We can't support it all, so we need some f

Re: array_random

2025-07-20 Thread jian he
On Fri, Jul 11, 2025 at 3:49 PM Dean Rasheed wrote: > > On Tue, 8 Jul 2025 at 15:26, Aleksander Alekseev > wrote: > > > > The proposed function seems to do two things at a time - generating > > random values and transforming them into an array of desired > > dimensions. Generally we try to avoid

Re: Even when the data is already ordered, MergeAppend still adds a Sort node

2025-07-20 Thread feichanghong
> On Jul 21, 2025, at 00:03, Tom Lane wrote: > > feichanghong writes: >> Currently, I have not found a better way to rewrite this, except by >> optimizing >> this scenario from the pg kernel side. > > If you're willing to modify your query, you could fake it out by > spelling the subquery's

Re: Log prefix missing for subscriber log messages received from publisher

2025-07-20 Thread Fujii Masao
On 2025/07/19 15:45, vignesh C wrote: Also, I don't see why there are three patches here instead of just one. Earlier we thought to commit replication changes function firstlly and then commit dblink and fdw changes, but now that we are using a common notice receiver function. I feel it can

Re: index prefetching

2025-07-20 Thread Thomas Munro
On Sun, Jul 20, 2025 at 1:07 AM Thomas Munro wrote: > On Sat, Jul 19, 2025 at 11:23 PM Tomas Vondra wrote: > > The thing that however concerns me is that what I observed was not the > > distance getting reset to 1, and then ramping up. Which should happen > > pretty quickly, thanks to the doublin

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-07-20 Thread Hannu Krosing
I have been evolving details for Direct TOAST design in https://wiki.postgresql.org/wiki/DirectTOAST The top level goals are * 8-byte TOAST pointer - just (header:1, tag:1 and TID:6) * all other info moved from toast pointer to actual toast record(s), so heap rows are smaller and faster. * all ex

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-07-20 Thread Hannu Krosing
Do you think the current patch could be backported to at least some latest versions ? On Fri, Jul 18, 2025 at 6:05 PM Nathan Bossart wrote: > > Committed. > > I'm cautiously optimistic that we can find some better gains for upgrades > from v16 and newer. That would involve dumping lo_create() co

Re: [PATCH] Proposal to Enable/Disable Index using ALTER INDEX

2025-07-20 Thread Robert Treat
On Thu, Jul 17, 2025 at 9:49 PM David Rowley wrote: > > On Wed, 16 Jul 2025 at 05:59, Robert Treat wrote: > > operational risk within the system. Similarly, the nature of the DDL > > change also requires that all sessions be impacted everywhere at once; > > there is no way to slowly roll the chan

[PATCH] Use strchr() to search for a single character

2025-07-20 Thread Dmitry Mityugov
Code in src/port/pgmkdirp.c uses strstr() to find a single character in a string, but strstr() seems to be too generic for this job. Another function, strchr(), might be better suited for this purpose, because it is optimized to search for exactly one character in a string. In addition, if strc

Re: Proposal: Out-of-Order NOTIFY via GUC to Improve LISTEN/NOTIFY Throughput

2025-07-20 Thread Rishu Bagga
> "Joel Jacobson" writes: > > Unfortunately, the author of [1] jumped to conclusion and assumed > the global lock was the problem. I'm quite sure it is probably not, > because: > > We know for sure, that current users do LISTEN and NOTIFY > in the same database. And there is no point in doing NOTI

Re: simple patch for discussion

2025-07-20 Thread Greg Hennessy
On Fri, Jul 18, 2025 at 12:23 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > Framing this differently, how about a patch that lets extension authors > choose to implement alternative formulas or even provide GUC-driven > constants into the planner at the existing spot instead of havin

Re: POC: Parallel processing of indexes in autovacuum

2025-07-20 Thread Daniil Davydov
Hi, On Fri, Jul 18, 2025 at 2:43 AM Masahiko Sawada wrote: > > On Mon, Jul 14, 2025 at 3:49 AM Daniil Davydov <3daniss...@gmail.com> wrote: > > > > This log level is used only "for messages about parallel workers launched". > > I think that such logs relate more to the parallel workers module tha

Re: Even when the data is already ordered, MergeAppend still adds a Sort node

2025-07-20 Thread Tom Lane
feichanghong writes: > Currently, I have not found a better way to rewrite this, except by optimizing > this scenario from the pg kernel side. If you're willing to modify your query, you could fake it out by spelling the subquery's "a = 1" condition in a way that won't produce an EquivalenceClass

Re: Logical Replication of sequences

2025-07-20 Thread vignesh C
On Fri, 18 Jul 2025 at 14:11, Dilip Kumar wrote: > > On Fri, Jul 18, 2025 at 10:44 AM Dilip Kumar wrote: > > > > On Thu, Jul 17, 2025 at 4:52 PM vignesh C wrote: > > > > > I was looking at the high level idea of sequence sync worker patch > i.e. 0005, so far I haven't found anything problematic

Re: Metadata and record block access stats for indexes

2025-07-20 Thread Frédéric Yhuel
On 7/4/25 18:00, Mircea Cadariu wrote: Just attaching v2 of the patch. Hi Mircea, Your patch applies cleanly and seems to work well. IIUC, the index hit ratio should be computed with the following formula: (idx_blks_hit - idx_metadata_blks) / (idx_blks_hit - idx_metadata_blks + idx_blks_

Re: Upgrade from Fedora 40 to Fedora 42, or from PostgreSQL 16.3 to PostgreSQL 16.9

2025-07-20 Thread Michael J. Baars
On Sun, Jul 20, 2025 at 10:13 AM Michael J. Baars wrote: > > On Sat, Jul 19, 2025 at 5:36 PM Tom Lane wrote: > > > > "Michael J. Baars" writes: > > > Compiling from source with a default ./configure --prefix=/usr/local > > > solves the problem. > > > > Cool. I confess I have no idea what the tr

DSA overflow in hash join

2025-07-20 Thread Konstantin Knizhnik
Hi hackers! There is weird error rarely reproduced with sqlancer: `ERROR: invalid DSA memory alloc request size 1140850688`: ** *-* * FinalizeAggregate(cost=114075075706156

Re: Even when the data is already ordered, MergeAppend still adds a Sort node

2025-07-20 Thread feichanghong
> On Jul 19, 2025, at 13:15, Zhang Mingli wrote: > > Hi, > > On Jul 18, 2025 at 22:52 +0800, feichanghong , wrote: > > explain select * from (select * from t where a = 1 and b > 1 order by a, b) > order by a, b limit 1; >QUERY PLAN > --

Re: Support for 8-byte TOAST values (aka the TOAST infinite loop problem)

2025-07-20 Thread Nikhil Kumar Veldanda
Hi, > v26-0014-Design-to-extend-the-varattrib_4b-and-toast-poin.patch: > Design proposal covering varattrib_4b, TOAST pointer layouts, and > related macro updates. > v26-0015-Implement-Zstd-compression-no-dictionary-support.patch: Plain > ZSTD (non dict) support and few basic tests. Sending v27 p

Re: Upgrade from Fedora 40 to Fedora 42, or from PostgreSQL 16.3 to PostgreSQL 16.9

2025-07-20 Thread Michael J. Baars
On Sat, Jul 19, 2025 at 5:36 PM Tom Lane wrote: > > "Michael J. Baars" writes: > > Compiling from source with a default ./configure --prefix=/usr/local > > solves the problem. > > Cool. I confess I have no idea what the triggering difference > was, because the extra options you mentioned before

Re: stats.sql might fail due to shared buffers also used by parallel tests

2025-07-20 Thread Alexander Lakhin
Hello Kuroda-san, Thank you for your attention to this! 15.07.2025 10:33, Hayato Kuroda (Fujitsu) wrote: GetSystemTimePreciseAsFileTime() returns FILETIME structure, which represents the time UTC with 100-nanosecod intervals [1]. The stack overflow seemed to refer it. However, the document fo