Re: POC: Parallel processing of indexes in autovacuum

2025-05-15 Thread Matheus Alcantara
Hi, On 09/05/25 15:33, Daniil Davydov wrote: > Hi, > As I promised - meet parallel index autovacuum with bgworkers > (Parallel-index-autovacuum-with-bgworkers.patch). This is pretty > simple implementation : > 1) Added new table option `parallel_idx_autovac_enabled` that must be > set to `true` if

Re: PG 18 release notes draft committed

2025-05-15 Thread Álvaro Herrera
On 2025-May-13, Laurenz Albe wrote: > On the one hand, the catalogs don't promise to be a stable API, so there > would be no need to enumerate such changes as compatibility breaks. I think it might be useful to distinguish changes to views, which are intended to be user-facing, from changes to "i

Re: Consider explicit incremental sort for Append and MergeAppend

2025-05-15 Thread Andrei Lepikhov
On 12/5/2025 11:29, Richard Guo wrote: For ordered Append or MergeAppend, it seems that incremental sort is currently not considered when injecting an explicit sort into subpaths that are not sufficiently ordered. For instance: Thanks for doing this job. I have reviewed your patch and want to p

Re: Update LDAP Protocol in fe-connect.c to v3

2025-05-15 Thread Pavel Seleznev
I applied your patch. I ran extended application tests relative to vanilla ones, which include various scenarios of working with LDAP and I think that we can safely apply the patch in the PG18. I did not see the need for additional LDAP tests, since compatibility is provided by the LDAP library

Re: Restrict publishing of partitioned table with a foreign table as partition

2025-05-15 Thread Amit Kapila
On Sun, May 11, 2025 at 6:53 AM Álvaro Herrera wrote: > > But the non-idiomatic locking of pg_partitioned_table appears to > continue to be the pain point of this patch. My impression is that > using a lock is the wrong approach to solve the concurrency problem. > Maybe we can use a ConditionVari

Re: Conflict detection for update_deleted in logical replication

2025-05-15 Thread Amit Kapila
On Fri, Apr 25, 2025 at 4:06 PM shveta malik wrote: > > > > > Here is V30 patch set includes the following changes: > > > > Thank You for the patch, please find few concerns: > > 1) > By looking at code of ApplyLauncherMain(), it appears that we stopped > advancing shared-slot's xmin if any of the

Re: [BUG] Skipped initialization of some xl_xact_parsed_prepare fields

2025-05-15 Thread Fujii Masao
On 2025/05/15 14:39, Daniil Davydov wrote: Hi, I noticed that inside ParsePrepareRecord function we are missing initialization of `nstats` and `nabortstats` fields of xl_xact_parsed_prepare structure: *** (current code in master) memset(parsed, 0, sizeof(*parsed)); parsed->xact_time = xlrec->

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Richard Guo
On Thu, May 15, 2025 at 6:50 PM Dilip Kumar wrote: > On Thu, May 15, 2025 at 3:14 PM Fujii Masao > wrote: > > How about adding a check to see whether the target relation has storage, > > using something like RELKIND_HAS_STORAGE()? > Yeah, that makes more sense. +1. FWIW, not long ago we fixed

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Dilip Kumar
On Thu, May 15, 2025 at 3:14 PM Fujii Masao wrote: > > > > On 2025/05/15 18:20, Dilip Kumar wrote: > > On Thu, May 15, 2025 at 2:22 PM Masahiro Ikeda > > wrote: > >> > >> Hi, > >> > >> I encountered an assertion failure when a partitioned table is specified > >> as an argument to pg_prewarm. Bel

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Fujii Masao
On 2025/05/15 18:20, Dilip Kumar wrote: On Thu, May 15, 2025 at 2:22 PM Masahiro Ikeda wrote: Hi, I encountered an assertion failure when a partitioned table is specified as an argument to pg_prewarm. Below are the steps to reproduce the issue: Thanks for the report! This assertion fail

Re: Should we optimize the `ORDER BY random() LIMIT x` case?

2025-05-15 Thread Andrei Lepikhov
On 15/5/2025 11:17, Aleksander Alekseev wrote: What kind of optimisation trick may the optimiser use here to provide an optimal plan? As I see it, it will need to think that all the tuples should be returned from the subquery. The only profit is to skip sorting the massive sample. Doesn't look

Re: Logical Replication of sequences

2025-05-15 Thread Peter Smith
Hi Vignesh. Some minor review comments for the patches in set v20250514. == Patch 0001. 1.1 For function 'pg_sequence_state', the DOCS call the 2nd parameter 'sequence_name', but the pg_proc.dat file calls it 'seq_name'. Should these be made the same? Patch 0004. pg_

Re: Assertion failure in smgr.c when using pg_prewarm with partitioned tables

2025-05-15 Thread Dilip Kumar
On Thu, May 15, 2025 at 2:22 PM Masahiro Ikeda wrote: > > Hi, > > I encountered an assertion failure when a partitioned table is specified > as an argument to pg_prewarm. Below are the steps to reproduce the > issue: > > $ pgbench -i -s 1 --partitions=3 > $ psql < CREATE EXTENSION pg_prewarm; > SE

Re: Should we optimize the `ORDER BY random() LIMIT x` case?

2025-05-15 Thread Aleksander Alekseev
wenhui, Andrei, > if we can optimize the query, that would be great Thanks for the feedback! > I think I got your point, but just to be sure: > Do you want to have some random sampling from an arbitrary subquery with > the guarantee that N distinct (by tid) tuples will be produced or all > the t

Re: Assert failure in base_yyparse

2025-05-15 Thread Richard Guo
On Thu, May 15, 2025 at 1:20 AM Alvaro Herrera wrote: > On 2025-May-14, Richard Guo wrote: > > Attached is a patch that implements this. It also renames the > > internally used option name and updates the error message. > LGTM. Thanks. I've pushed this patch and backpatched it through v15. I

Re: Should we optimize the `ORDER BY random() LIMIT x` case?

2025-05-15 Thread Andrei Lepikhov
On 15/5/2025 01:41, Aleksander Alekseev wrote: One can do `SELECT (the query above) ORDER BY random() LIMIT x` but this produces an inefficient plan. Alternatively one could create temporary tables using `CREATE TEMP TABLE ... AS SELECT * FROM tbl TABLESAMPLE BERNOULLI(20)` but this is inconvenie

Re: Should we optimize the `ORDER BY random() LIMIT x` case?

2025-05-14 Thread wenhui qiu
Hi Aleksander if we can optimize the query, that would be great,Then we won't need to pull a lot of data to the program end and randomly pick the needed data there. On Thu, 15 May 2025 at 07:41, Aleksander Alekseev wrote: > Hi, > > If I didn't miss anything, currently we don't seem to support s

Re: Statistics Import and Export

2025-05-14 Thread Hari Krishna Sunder
Thanks Nathan. Here is the patch with a comment. On Wed, May 14, 2025 at 8:53 AM Nathan Bossart wrote: > On Tue, May 13, 2025 at 05:01:02PM -0700, Hari Krishna Sunder wrote: > > We found a minor issue when testing statistics import with upgrading from > > versions older than v14. (We have VACUUM

Re: Small fixes needed by high-availability tools

2025-05-14 Thread Mihail Nikalayeu
> I think this LSN is simply LSN where crash recovery ends... Yes, you are right and we come back to : > 1. Allow checking standby sync before making data visible after crash recovery Best regards, Mikhail.

Re: PostgreSQL 18 Beta 1 io_max_concurrency

2025-05-14 Thread Tom Lane
"Jafri, Nazneen" writes: > The default setting of parameter io_max_concurrency shows 64 whereas the > documentation says -1, checking if this is the expected behavior. I think it's behaving as designed: the bootstrap default is indeed -1, but later in startup there is an an auto-tuning step that

Re: Small fixes needed by high-availability tools

2025-05-14 Thread Andrey Borodin
> On 14 May 2025, at 05:33, Amit Kapila wrote: > >> It might not necessarily be a flag—it could be some LSN value instead. >> Also, it's not just about a "new transaction," but about any new >> snapshot that could see data not yet replicated to the synchronous >> standby. >> > > Sounds reaso

Re: Assert failure in base_yyparse

2025-05-14 Thread Alvaro Herrera
On 2025-May-14, Richard Guo wrote: > Attached is a patch that implements this. It also renames the > internally used option name and updates the error message. LGTM. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/

Re: Statistics Import and Export

2025-05-14 Thread Nathan Bossart
On Tue, May 13, 2025 at 05:01:02PM -0700, Hari Krishna Sunder wrote: > We found a minor issue when testing statistics import with upgrading from > versions older than v14. (We have VACUUM and ANALYZE disabled) > 3d351d916b20534f973eda760cde17d96545d4c4 >

Re: Disable parallel query by default

2025-05-14 Thread Álvaro Herrera
On 2025-May-13, Scott Mead wrote: > If you have a connection pool from your application and you issue 60 > queries per minute, each that can go from 1 second to 200 ms, That > means that you are making 120 connections per minute back to the DB. > > As we know, connection establishment is brutal..

Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

2025-05-14 Thread Florents Tselai
ths, >> etc. If we were designing this today, suffixes like _stable or _volatile >> might have been more appropriate, but at this point, we’re better off >> staying consistent with the _tz family. > > I get the pragmatism, and don’t want to over-bike-shed, but what a war

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-14 Thread Dilip Kumar
On Wed, May 14, 2025 at 9:16 AM Amit Kapila wrote: > > On Tue, May 13, 2025 at 4:22 PM Dilip Kumar wrote: > > > > On Tue, May 13, 2025 at 3:48 PM shveta malik wrote: > > > > > > Hi All, > > > > > > It is a spin-off thread from earlier discussions at [1] and [2]. > > > > > > While analyzing the s

Re: Incremental Sort Cost Estimation Instability

2025-05-14 Thread Andrei Lepikhov
On 9/12/24 16:57, Tomas Vondra wrote: On 9/12/24 12:12, David Rowley wrote: On Thu, 12 Sept 2024 at 21:51, Andrei Lepikhov wrote: Initial problem causes wrong cost_sort estimation. Right now I think about providing cost_sort() the sort clauses instead of (or in addition to) the pathkeys. I'm

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-14 Thread Dilip Kumar
On Wed, May 14, 2025 at 12:15 PM Amit Kapila wrote: > > On Wed, May 14, 2025 at 11:59 AM Dilip Kumar wrote: > > > > On Wed, May 14, 2025 at 9:16 AM Amit Kapila wrote: > > > > > > On Tue, May 13, 2025 at 4:22 PM Dilip Kumar wrote: > > > > > > > > On Tue, May 13, 2025 at 3:48 PM shveta malik >

Re: Small fixes needed by high-availability tools

2025-05-14 Thread Amit Kapila
On Wed, May 14, 2025 at 2:15 AM Mihail Nikalayeu wrote: > > > One idea to solve this problem could be that whenever we cancel > > sync_rep_wait, we set some system-wide flag that indicates that any > > new transaction must ensure that all the current data is replicated to > > the synchronous stand

Re: Suggestion to add --continue-client-on-abort option to pgbench

2025-05-14 Thread Srinath Reddy Sadipiralla
Hi, On Tue, May 13, 2025 at 11:27 AM Dilip Kumar wrote: > On Tue, May 13, 2025 at 9:20 AM wrote: > > I also appreciate you for pointing out my mistakes in the previous > version of the patch. I fixed the duplicated lines. I’ve attached the > updated patch. > > > This is a useful feature, so +1

Re: using index to speedup add not null constraints to a table

2025-05-14 Thread jian he
On Mon, Apr 28, 2025 at 4:40 PM Álvaro Herrera wrote: > > On 2025-Apr-28, jian he wrote: > > > for tests, just found out i can imitate > > src/test/modules/test_misc/t/001_constraint_validation.pl, > > > > So I created a file: > > src/test/modules/test_misc/t/008_indexscan_validate_notnull.pl > >

Re: Vacuum statistics

2025-05-14 Thread Amit Kapila
On Tue, May 13, 2025 at 3:19 PM Alena Rybakina wrote: > > On 12.05.2025 08:30, Amit Kapila wrote: > > On Fri, May 9, 2025 at 5:34 PM Alena Rybakina > > wrote: > >> I did a rebase and finished the part with storing statistics separately > >> from the relation statistics - now it is possible to d

Re: Disable parallel query by default

2025-05-14 Thread Laurenz Albe
On Tue, 2025-05-13 at 17:53 -0400, Scott Mead wrote: > On Tue, May 13, 2025, at 5:07 PM, Greg Sabino Mullane wrote: > > On Tue, May 13, 2025 at 4:37 PM Scott Mead wrote: > > > I'll open by proposing that we prevent the planner from automatically > > > selecting parallel plans by default > > > > T

Re: Assert failure in base_yyparse

2025-05-14 Thread Richard Guo
On Fri, May 9, 2025 at 6:50 PM Alvaro Herrera wrote: > I agree that blocking the index from using the option name that xmltable > parsing uses internally is okay. Maybe we can rename it to something > like "__pg__is_not_null" or something like that, which would reduce the > chances of troubling p

Re: Allow reading LSN written by walreciever, but not flushed yet

2025-05-14 Thread Alexander Kukushkin
On Mon, 12 May 2025 at 17:48, Andrey Borodin wrote: > Done so. Thanks! TBH, the current function name pg_last_wal_receive_lsn() is confusing, and introducing a new one pg_last_wal_receive_unflushed_lsn() doesn't make it better :( What about actually adding TWO new functions, pg_last_wal_write_l

Re: Allow reading LSN written by walreciever, but not flushed yet

2025-05-13 Thread Alexander Kukushkin
Hi Fujii, On Tue, 13 May 2025 at 13:13, Fujii Masao wrote: > In this case, doesn't the flush LSN typically catch up to the write LSN on > node2 > after a few seconds? Even if the walreceiver exits while there's still > written > but unflushed WAL, it looks like WalRcvDie() ensures everything is

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-13 Thread Amit Kapila
On Wed, May 14, 2025 at 11:59 AM Dilip Kumar wrote: > > On Wed, May 14, 2025 at 9:16 AM Amit Kapila wrote: > > > > On Tue, May 13, 2025 at 4:22 PM Dilip Kumar wrote: > > > > > > On Tue, May 13, 2025 at 3:48 PM shveta malik > > > wrote: > > > > > > > > Hi All, > > > > > > > > It is a spin-off t

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-13 Thread Alexander Kukushkin
Hi Dilip, On Wed, 14 May 2025 at 08:29, Dilip Kumar wrote: > What I meant wasn’t that the subscriber is moving the confirmed LSN > backward, nor was I suggesting we fix it by persisting the LSN on the > subscriber side. My point was: the fact that the subscriber is sending > an LSN older than on

Re: Limiting overshoot in nbtree's parallel SAOP index scans

2025-05-13 Thread Amit Kapila
On Thu, Oct 17, 2024 at 5:03 AM Matthias van de Meent wrote: > > On Thu, 17 Oct 2024 at 00:33, Peter Geoghegan wrote: > > > > On Wed, Oct 16, 2024 at 5:48 PM Matthias van de Meent > > wrote: > > > In v17 and the master branch you'll note 16 buffer hits for the test > > > query. However, when we

Re: Logical Replication of sequences

2025-05-13 Thread vignesh C
On Wed, 14 May 2025 at 09:55, Nisha Moond wrote: > > On Sat, May 3, 2025 at 7:28 PM vignesh C wrote: > > > > > > There was one pending open comment #6 from [1]. This has been > > addressed in the attached patch. > > Thank you for the patches, here are my comments for patch-004: sequencesync.c > >

Re: Logical Replication of sequences

2025-05-13 Thread Nisha Moond
On Sat, May 3, 2025 at 7:28 PM vignesh C wrote: > > > There was one pending open comment #6 from [1]. This has been > addressed in the attached patch. Thank you for the patches, here are my comments for patch-004: sequencesync.c copy_sequences() --- 1) + if (first) + first = fals

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-13 Thread Amit Kapila
On Tue, May 13, 2025 at 4:22 PM Dilip Kumar wrote: > > On Tue, May 13, 2025 at 3:48 PM shveta malik wrote: > > > > Hi All, > > > > It is a spin-off thread from earlier discussions at [1] and [2]. > > > > While analyzing the slot-sync BF failure as stated in [1], it was > > observed that there are

Re: Memoize ANTI and SEMI JOIN inner

2025-05-13 Thread Richard Guo
On Tue, May 13, 2025 at 10:51 PM Andres Freund wrote: > This is failing on CI: > https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F5636 > https://api.cirrus-ci.com/v1/artifact/task/5411026402803712/testrun/build-32/testrun/regress/regress/regression.diffs > > diff -U3 /tmp/cirrus-ci-bu

Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

2025-05-13 Thread David E. Wheeler
; might have been more appropriate, but at this point, we’re better off staying > consistent with the _tz family. I get the pragmatism, and don’t want to over-bike-shed, but what a wart to live with. [I just went back and re-read Robert’s post, and didn’t realize he used exactly the same expres

Re: Add explicit initialization for all PlannerGlobal fields

2025-05-13 Thread Richard Guo
On Tue, May 13, 2025 at 8:26 PM David Rowley wrote: > On Tue, 13 May 2025 at 04:03, Richard Guo wrote: > > While adding a new field to PlannerGlobal in another patch, I noticed > > that although most fields are explicitly initialized, a few are not. > > This doesn't cause any functional issues, s

Re: Statistics Import and Export

2025-05-13 Thread Hari Krishna Sunder
We found a minor issue when testing statistics import with upgrading from versions older than v14. (We have VACUUM and ANALYZE disabled) 3d351d916b20534f973eda760cde17d96545d4c4 changed the def

Re: Disable parallel query by default

2025-05-13 Thread Scott Mead
On Tue, May 13, 2025, at 5:07 PM, Greg Sabino Mullane wrote: > On Tue, May 13, 2025 at 4:37 PM Scott Mead wrote: >> I'll open by proposing that we prevent the planner from automatically >> selecting parallel plans by default > > That seems a pretty heavy hammer, when we have things like > pa

Re: Batch TIDs lookup in ambulkdelete

2025-05-13 Thread Matheus Alcantara
maining all regular index tuples, which > seems to be the most straightforward approach. While further > optimizations are possible, such as performing batch TID lookups for > all index tuples on a single page, these could introduce additional > overhead from sorting and re-sorting TIDs.

Re: Disable parallel query by default

2025-05-13 Thread Greg Sabino Mullane
On Tue, May 13, 2025 at 4:37 PM Scott Mead wrote: > I'll open by proposing that we prevent the planner from automatically > selecting parallel plans by default That seems a pretty heavy hammer, when we have things like parallel_setup_cost that should be tweaked first. The recommendation that I

Re: Disable parallel query by default

2025-05-13 Thread Lukas Fittl
On Tue, May 13, 2025 at 4:36 PM Scott Mead wrote: > I'm looking forward to the upcoming monitoring in e7a9496 (Add two > attributes to pg_stat_database for parallel workers activity), it will be > easier to empirically prove that parallel query is being used. I don't > think the patch goes far e

Re: Small fixes needed by high-availability tools

2025-05-13 Thread Mihail Nikalayeu
Hello, everyone! > On Mon, 12 May 2025 at 18:42, Andrey Borodin wrote: >> >> Problem: user might try to cancel locally committed transaction and if we >> >> do so we will show non-replicated data as committed. This leads to >> >> loosing data with UPSERTs. >> > > >> > > Could you explain why sp

Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

2025-05-13 Thread Florents Tselai
> On 13 May 2025, at 2:07 PM, David E. Wheeler wrote: > > On May 9, 2025, at 15:50, Robert Haas wrote: > >> # We have the kluge of having separate "_tz" functions to support >> # non-immutable datetime operations, but that way doesn't seem like >> # it's going to scale well to multiple source

Re: Allow reading LSN written by walreciever, but not flushed yet

2025-05-13 Thread Mihail Nikalayeu
Hello, everyone! > Or might want LSN "flush everything you have written, and return that LSN". > That will also do the trick, but is not necessary. I think it is a better option. Less things may go wrong in such a case. Best regards, Mikhail.

Re: Remove Instruction Synchronization Barrier in spin_delay() for ARM64 architecture

2025-05-13 Thread Salvatore Dipietro
On Thu, 1 May 2025 at 14:50, Nathan Bossart wrote: > So... > > * The ISB does seem to have a positive effect without commit 3d0b4b1 > applied. > > * With commit 3d0b4b1 applied, removing the ISB seems to have a positive > effect at high concurrencies. This is especially pronounced in the >

Re: PATCH: jsonpath string methods: lower, upper, initcap, l/r/btrim, replace, split_part

2025-05-13 Thread David E. Wheeler
On May 9, 2025, at 15:50, Robert Haas wrote: > # We have the kluge of having separate "_tz" functions to support > # non-immutable datetime operations, but that way doesn't seem like > # it's going to scale well to multiple sources of mutability. > > But I'm not sure I understand why it matters

Re: queryId constant squashing does not support prepared statements

2025-05-13 Thread Dmitry Dolgov
> On Mon, May 12, 2025 at 06:40:43PM GMT, Sami Imseih wrote: > > The static variables was only part of the concern, another part was > > using A_Expr to carry this information, which will have impact on lots > > of unrelated code. > > What would be the problem if A_Expr carries an extra pointer to

Re: fix: propagate M4 env variable to flex subprocess

2025-05-13 Thread Andres Freund
Hi, On 2025-05-12 23:14:59 -0400, J. Javier Maestro wrote: > The pgflex wrapper runs flex with an explicit environment, so it doesn't > inherit environment variables from the parent process. However, flex can > use the M4 env variable and/or the PATH (via execvp) to find the m4 macro > processor.

Re: Add partial :-variable expansion to psql \copy

2025-05-13 Thread Andres Freund
Hi, On 2025-05-01 16:37:18 +, Akshat Jaimini wrote: > I am unable to apply the patch on master. Can you please confirm if the patch > is rebased? cfbot seems to be able to apply the patch. However, the tests have not passed in quite a while: https://cirrus-ci.com/task/5702107409416192 http

Re: Serverside SNI support in libpq

2025-05-13 Thread Andres Freund
Hi, On 2025-02-27 14:38:24 +0100, Daniel Gustafsson wrote: > The attached v6 rebase contains this as well as your tests as well as > general cleanup and comment writing etc. This is not passing CI on windows... https://cirrus-ci.com/build/4765059278176256 Greetings, Andres

Re: Memoize ANTI and SEMI JOIN inner

2025-05-13 Thread Andres Freund
Hi, On 2025-04-10 19:36:14 +0900, Richard Guo wrote: > On Wed, Apr 9, 2025 at 6:18 PM David Rowley wrote: > > On Wed, 9 Apr 2025 at 18:48, Richard Guo wrote: > > > Perhaps we could spend some planner cycles proving inner_unique for > > > anti joins, so that Memoize nodes can be considered for th

Re: Test mail for pgsql-hackers

2025-05-13 Thread Andres Freund
On 2025-03-12 15:56:08 +0530, Ashutosh Bapat wrote: > Large part of your patch is renaming files which are not related to > this patch. Can you please clean that up. Since this has not been fixed, and the patch has not passed CI since at least mid March, I'll close this CF entry.

Re: using index to speedup add not null constraints to a table

2025-05-13 Thread Andres Freund
Hi, On 2025-04-28 12:36:14 +0800, jian he wrote: > On Fri, Apr 18, 2025 at 4:07 PM jian he wrote: > > > > I don't have any good ideas to do the regress tests. > > I use > > ereport(NOTICE, > > errmsg("all not-null constraints on relation > > \"%s\" are vali

Re: Fix incorrect order of params in comment

2025-05-13 Thread Daniel Gustafsson
> On 9 May 2025, at 23:50, Michael Paquier wrote: > > On Fri, May 09, 2025 at 07:21:17PM +0200, Daniel Gustafsson wrote: >> -=item $node->log_check($offset, $test_name, %parameters) >> +=item $node->log_check($test_name, $offset, %parameters) >> >> Check contents of server logs. > > Right, good

Re: Add an option to skip loading missing publication to avoid logical replication failure

2025-05-13 Thread Amit Kapila
On Sat, May 10, 2025 at 5:15 PM Amit Kapila wrote: > > I am planning to proceed with the test-fix proposed by Vignesh [1] > early next week unless we want to discuss more on this issue. > Pushed. > [1] - > https://www.postgresql.org/message-id/CALDaNm3TH3J8fwj%2BNcdjdN8DZCdrnmm1kzBsHBW2nkN%2Bh6

Re: Add explicit initialization for all PlannerGlobal fields

2025-05-13 Thread David Rowley
On Tue, 13 May 2025 at 04:03, Richard Guo wrote: > While adding a new field to PlannerGlobal in another patch, I noticed > that although most fields are explicitly initialized, a few are not. > This doesn't cause any functional issues, since makeNode() zeroes all > fields by default. However, the

Re: Allow reading LSN written by walreciever, but not flushed yet

2025-05-13 Thread Fujii Masao
On 2025/05/13 0:47, Andrey Borodin wrote: Moved off from "Small fixes needed by high-availability tools" On 12 May 2025, at 01:33, Amit Kapila wrote: On Fri, May 2, 2025 at 6:30 PM Andrey Borodin wrote: 3. Allow reading LSN written by walreciever, but not flushed yet Problem: if we hav

Re: Improve monitoring of shared memory allocations

2025-05-13 Thread Amit Kapila
On Fri, Apr 4, 2025 at 9:15 PM Rahila Syed wrote: > > Please find attached the patch which removes the changes for non-shared hash > tables > and keeps them for shared hash tables. > /* Allocate initial segments */ + i = 0; for (segp = hashp->dir; hctl->nsegs < nsegs; hctl->nsegs++, segp++)

Re: Backward movement of confirmed_flush resulting in data duplication.

2025-05-13 Thread Dilip Kumar
On Tue, May 13, 2025 at 3:48 PM shveta malik wrote: > > Hi All, > > It is a spin-off thread from earlier discussions at [1] and [2]. > > While analyzing the slot-sync BF failure as stated in [1], it was > observed that there are chances that confirmed_flush_lsn may move > backward depending on the

Re: Vacuum statistics

2025-05-13 Thread Alena Rybakina
Hi! On 12.05.2025 08:30, Amit Kapila wrote: On Fri, May 9, 2025 at 5:34 PM Alena Rybakina wrote: I did a rebase and finished the part with storing statistics separately from the relation statistics - now it is possible to disable the collection of statistics for relationsh using gucs and thi

Re: make VALIDATE domain constraint lock on related relations as ShareUpdateExclusiveLock

2025-05-13 Thread wenhui qiu
HI Thanks for the patch! and overall this is a valuable improvement in terms of concurrency and consistency with table-level behavior.The refactoring to pass LOCK MODE into validateDomainCheckConstraint() improves flexibility and clarity. and the patch also looks good to me. Regards On Tue, M

Re: make VALIDATE domain constraint lock on related relations as ShareUpdateExclusiveLock

2025-05-12 Thread Dilip Kumar
On Tue, May 13, 2025 at 8:57 AM jian he wrote: > > hi. > > We can still perform DML operations on a table while validating its > check constraint. > Similarly, it should be fine to do DML while validating domain constraints? > but currently, it's not allowed for domain constraints. > > The attache

Re: PG 18 release notes draft committed

2025-05-12 Thread Laurenz Albe
On Mon, 2025-05-12 at 20:27 -0700, David G. Johnston wrote: > On Thursday, May 1, 2025, Bruce Momjian wrote: > > I will continue improving it until beta 1, and until the final release. > > I will probably add markup in 1-3 weeks.  Let the feedback begin.  ;-) > > > Should all columns removed fro

Re: Suggestion to add --continue-client-on-abort option to pgbench

2025-05-12 Thread Dilip Kumar
On Tue, May 13, 2025 at 9:20 AM wrote: > I also appreciate you for pointing out my mistakes in the previous version of > the patch. I fixed the duplicated lines. I’ve attached the updated patch. > This is a useful feature, so +1 from my side. Here are some initial comments on the patch while hav

Re: Why our Valgrind reports suck

2025-05-12 Thread Yasir
On Mon, May 12, 2025 at 12:11 AM Tom Lane wrote: > I wrote: > > And, since there's nothing new under the sun around here, > > we already had a discussion about that back in 2021: > > > https://www.postgresql.org/message-id/flat/3471359.1615937770%40sss.pgh.pa.us > > That thread seems to have led

RE: Changing shared_buffers without restart

2025-05-12 Thread Jack Ng
Hi Ashutosh, > > * During resize, simply calculate the new size and call ftruncate on each > > segment to adjust memory accordingly, no need to mmap/munmap or modify any > > memory mapping. > > > > > That's same as my understanding. Great, thanks for confirming! > I thought I had shared a test pr

Re: Suggestion to add --continue-client-on-abort option to pgbench

2025-05-12 Thread Rintaro.Ikeda
Hi Stepan and Matthias, Thank you both for your replies. I agree with Matthias's detailed explanation regarding the purpose of the patch. > Regarding the patch code, I noticed that there are duplicate case > entries in the command-line option handling (specifically for case 18 > or case ESTATUS_

Re: PG 18 release notes draft committed

2025-05-12 Thread David G. Johnston
On Thursday, May 1, 2025, Bruce Momjian wrote: > > > I will continue improving it until beta 1, and until the final release. > I will probably add markup in 1-3 weeks. Let the feedback begin. ;-) > Should all columns removed from system views and/or catalogs be listed in “Migration” or is there

Re: Useless LEFT JOIN breaks MIN/MAX optimization

2025-05-12 Thread Alena Rybakina
On 12.05.2025 14:05, Tom Lane wrote: Robert Haas writes: But that's also assuming that you're correct here about how to descend through a JoinExpr, which I'm not quite sure whether is true. It's also assuming that we should solve the problem here rather than in some other part of the code e.g.

Re: queryId constant squashing does not support prepared statements

2025-05-12 Thread Sami Imseih
> > On Fri, May 09, 2025 at 12:47:19PM GMT, Sami Imseih wrote: > > So, I think we can create a new parse node ( parsenode.h ) that will only be > > used in parsing (and gram.c only ) to track the start/end locations > > and List and > > based on this node we can create A_ArrayExpr and A_Expr with t

Re: Proposal: Exploring LSM Tree‑Based Storage Engine for PostgreSQL (Inspired by MyRocks)

2025-05-12 Thread Nico Williams
On Sun, May 11, 2025 at 11:06:00AM -0700, Manish Rai Jain wrote: > The motivation stems from the well‑known write‑amplification issues > with B‑trees under high write throughput. An LSM‑based engine could > offer: Consider the ZFS copy-on-write b-tree-ish design, which has _even more_ write amplif

Re: Adding error messages to a few slash commands

2025-05-12 Thread Robin Haberkorn
On Mon Apr 14, 2025 at 12:27:53 GMT +03, Pavel Luzanov wrote: > I recommend to create a new entry for this patch in the open July > commitfest . > I will be able to review this patch in a couple months later in June, > if no one wants to review it earlier. I

Re: Proposal: Exploring LSM Tree‑Based Storage Engine for PostgreSQL (Inspired by MyRocks)

2025-05-12 Thread Aleksander Alekseev
Hi Manish, > I’ve been exploring the idea of integrating an LSM tree–based storage engine > into PostgreSQL — similar in spirit to MyRocks for MySQL — by replacing the > underlying storage while preserving PostgreSQL’s upper layers (planner, > executor, MVCC, etc.). > [...] Personally I wouldn

Re: Request for Implementation of Custom Error Messages for CHECK Constraints

2025-05-12 Thread Bruce Momjian
On Mon, May 12, 2025 at 11:22:21AM -0700, David G. Johnston wrote: > On Monday, May 12, 2025, Bruce Momjian wrote: > > Yeah, you could name the constraint > "Custom_error_message_when_the_condition_is_not_met." and then just > convert underscore to spaces and display that to the user.

Re: Proposal: Exploring LSM Tree‑Based Storage Engine for PostgreSQL (Inspired by MyRocks)

2025-05-12 Thread Matthias van de Meent
On Sun, 11 May 2025 at 14:06, Manish Rai Jain wrote: > > Hi hackers, > > I’ve been exploring the idea of integrating an LSM tree–based storage engine > into PostgreSQL — similar in spirit to MyRocks for MySQL — by replacing the > underlying storage while preserving PostgreSQL’s upper layers (pla

Re: INSERT ... ON CONFLICT DO SELECT [FOR ...] take 2

2025-05-12 Thread Andreas Karlsson
On 3/31/25 5:33 PM, Dean Rasheed wrote: On 3/4/25 10:24 AM, Andreas Karlsson wrote: Rebased the patch to add support for OLD.* and NEW.*. I took a closer look at this, and I have a number of comments: Thanks for taking a look and improving my patch! And thanks to Kirill too. * The changes

Re: Request for Implementation of Custom Error Messages for CHECK Constraints

2025-05-12 Thread David G. Johnston
On Monday, May 12, 2025, Bruce Momjian wrote: > > > Yeah, you could name the constraint > "Custom_error_message_when_the_condition_is_not_met." and then just > convert underscore to spaces and display that to the user. > > The 63 byte limit seems much more likely to be a factor if the name has to

Re: Request for Implementation of Custom Error Messages for CHECK Constraints

2025-05-12 Thread Bruce Momjian
On Sat, May 10, 2025 at 07:58:47PM -0400, Tom Lane wrote: > "Miguel Ferreira" writes: > > I propose extending the syntax of the ALTER TABLE ADD CONSTRAINT > > statement (and potentially CREATE TABLE) to allow for the > > specification of a custom error message for each CHECK constraint. > > Why d

Re: Small fixes needed by high-availability tools

2025-05-12 Thread Ants Aasma
Hi, dropping in my 2 cents here. On Mon, 12 May 2025 at 18:42, Andrey Borodin wrote: > >> Problem: user might try to cancel locally committed transaction and if we > >> do so we will show non-replicated data as committed. This leads to loosing > >> data with UPSERTs. > > > > Could you explain w

Re: Useless LEFT JOIN breaks MIN/MAX optimization

2025-05-12 Thread Tom Lane
Robert Haas writes: > But that's also assuming that you're correct here about how to descend > through a JoinExpr, which I'm not quite sure whether is true. It's > also assuming that we should solve the problem here rather than in > some other part of the code e.g. the join removal code, and I'm n

Re: libcurl in libpq.pc

2025-05-12 Thread Jacob Champion
On Mon, May 12, 2025 at 8:49 AM Jacob Champion wrote: > > On Mon, May 12, 2025 at 3:50 AM Christoph Berg wrote: > > Since nothing in libpq should need curl for compiling, should we drop > > it there instead? > > The static build (libpq.a) still needs libcurl. The module is only > compiled for use

Re: libcurl in libpq.pc

2025-05-12 Thread Jacob Champion
On Mon, May 12, 2025 at 3:50 AM Christoph Berg wrote: > Since nothing in libpq should need curl for compiling, should we drop > it there instead? The static build (libpq.a) still needs libcurl. The module is only compiled for use by the shared library. --Jacob

Re: add tab-complete for ALTER DOMAIN ADD...

2025-05-12 Thread Álvaro Herrera
On 2025-May-12, Fujii Masao wrote: > I have no objection to this commit. However, I had assumed we would > wait to commit changes like this - which aren't bug fixes or > v18-related oversights - until master becomes the development branch > for v19. Maybe I'm missing something.. Yeah, fair questi

Re: Small fixes needed by high-availability tools

2025-05-12 Thread Andrey Borodin
> On 6 May 2025, at 12:00, Matthias van de Meent > wrote: > > On Fri, 2 May 2025 at 15:00, Andrey Borodin wrote: >> >> Hi hackers! >> >> I want to revive attempts to fix some old edge cases of physical quorum >> replication. >> >> Please find attached draft patches that demonstrate ideas.

Re: add tab-complete for ALTER DOMAIN ADD...

2025-05-12 Thread Tom Lane
Fujii Masao writes: > On 2025/05/11 23:22, Álvaro Herrera wrote: >> Done that, and pushed. > I have no objection to this commit. However, I had assumed we would > wait to commit changes like this - which aren't bug fixes or > v18-related oversights - until master becomes the development branch >

Re: [PATCH] Fix replica identity mismatch for partitioned tables with publish_via_partition_root

2025-05-12 Thread Maxim Orlov
Hi! This is probably not the most familiar part of Postgres to me, but does it break anything? Or is it just inconsistency in the replication protocol? A test for the described scenario would be a great addition. And, if it is feasible, provide an example of what would be broken with the way part

Re: Why our Valgrind reports suck

2025-05-12 Thread Tom Lane
=?utf-8?Q?=C3=81lvaro?= Herrera writes: > On 2025-May-11, Tom Lane wrote: >> In particular, I had not realized that autovacuum >> leaks a nontrivial amount of memory per relation processed (cf 0009), >> and apparently has done for a few releases now. This is horrid in >> databases with many table

Re: Why our Valgrind reports suck

2025-05-12 Thread Álvaro Herrera
On 2025-May-11, Tom Lane wrote: > Okay, here is a patch series that updates the > 0001-Make-memory-contexts-themselves-more-visible-to-valg.patch > patch you posted in that thread, and makes various follow-up > fixes that either fix or paper over various leaks. Wow, that's a lot of extra fixes.

Re: Adding skip scan (including MDAM style range skip scan) to nbtree

2025-05-12 Thread Peter Geoghegan
On Sun, May 11, 2025 at 11:09 PM Tomas Vondra wrote: > I where "revert" is master with the removal patch. Sorry about the > confusion, I guess I was distracted and did some mistake. > > So, this seems to be in line with the hypothesis ... That makes way more sense. I wonder if we can fix this pr

Re: Vacuum statistics

2025-05-12 Thread Amit Kapila
On Fri, May 9, 2025 at 5:34 PM Alena Rybakina wrote: > > I did a rebase and finished the part with storing statistics separately from > the relation statistics - now it is possible to disable the collection of > statistics for relationsh using gucs and > this allows us to solve the problem with

[PATCH v2] Re: Cancel problems of query to pg_stat_statements

2025-05-12 Thread Roman Khapov
Hi! Thanks for the review! Its true that qtext_load_file() can be called with acquired lock in pg_stat_statements_internal()… So I perform INTERRUPTS_PENDING_CONDITION in v2 patch and call CHECK_FOR_INTERRUPTS later, after cycle ended and lock released. v2-cancelable-qtext_load_file.patch D

  1   2   3   4   5   6   7   8   9   10   >