Re: Why is parula failing?

2024-04-15 Thread Robins Tharakan
On Mon, 15 Apr 2024 at 16:02, Tom Lane wrote: > David Rowley writes: > > If GetNowFloat() somehow was returning a negative number then we could > > end up with a large delay. But if gettimeofday() was so badly broken > > then wouldn't there be some evidence of this in the log timestamps on > > f

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Bertrand Drouvot
Hi, On Tue, Apr 16, 2024 at 10:00:04AM +0530, shveta malik wrote: > Please find v5 addressing above comments. Thanks! @@ -1634,9 +1677,14 @@ SyncReplicationSlots(WalReceiverConn *wrconn) { PG_ENSURE_ERROR_CLEANUP(slotsync_failure_callback, PointerGetDatum(wrconn)); { +

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Amit Kapila
On Tue, Apr 16, 2024 at 12:03 PM Bertrand Drouvot wrote: > > On Tue, Apr 16, 2024 at 08:21:04AM +0530, Amit Kapila wrote: > > > There is no clear use case for allowing them in parallel and I feel it > > would add more confusion when it can work sometimes but not other > > times. However, if we rec

RE: Race condition in FetchTableStates() breaks synchronization of subscription tables

2024-04-15 Thread Zhijie Hou (Fujitsu)
On Wednesday, March 13, 2024 11:49 AM vignesh C wrote: > > On Tue, 12 Mar 2024 at 09:34, Ajin Cherian wrote: > > > > > > > > On Tue, Mar 12, 2024 at 2:59 PM vignesh C wrote: > >> > >> > >> Thanks, I have created the following Commitfest entry for this: > >> https://commitfest.postgresql.org/47/

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Bertrand Drouvot
Hi, On Tue, Apr 16, 2024 at 08:21:04AM +0530, Amit Kapila wrote: > On Mon, Apr 15, 2024 at 7:47 PM Bertrand Drouvot > wrote: > > > > On Mon, Apr 15, 2024 at 06:29:49PM +0530, Amit Kapila wrote: > > > On Mon, Apr 15, 2024 at 6:21 PM Bertrand Drouvot > > > wrote: > > > > > > > > On Mon, Apr 15, 20

Disallow changing slot's failover option in transaction block

2024-04-15 Thread Zhijie Hou (Fujitsu)
Hi, Kuroda-San reported an issue off-list that: If user execute ALTER SUBSCRIPTION SET (failover) command inside a txn block and rollback, only the subscription option change can be rolled back, while the replication slot's failover change is preserved. This is because ALTER SUBSCRIPTION SET (fa

Re: Typo about the SetDatatabaseHasLoginEventTriggers?

2024-04-15 Thread Michael Paquier
On Tue, Apr 16, 2024 at 02:05:46PM +0800, Japin Li wrote: > Upon reviewing the login event trigger, I noticed a potential typo about > the SetDatabaseHasLoginEventTriggers function name. Indeed, thanks! Will fix and double-check the surroundings. -- Michael signature.asc Description: PGP signat

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-15 Thread Amit Kapila
On Tue, Apr 16, 2024 at 7:48 AM Hayato Kuroda (Fujitsu) wrote: > > > > FYI - We also considered the idea which walsender waits until all prepared > > transactions > > > are resolved before decoding and sending changes, but it did not work well > > > - the restarted walsender sent only COMMIT PREPA

Re: Things I don't like about \du's "Attributes" column

2024-04-15 Thread Pavel Luzanov
On 16.04.2024 01:06, David G. Johnston wrote: At this point I'm on board with retaining the \dr charter of simply being an easy way to access the detail exposed in pg_roles with some display formatting but without any attempt to convey how the system uses said information.  Without changing p

Typo about the SetDatatabaseHasLoginEventTriggers?

2024-04-15 Thread Japin Li
Hi, Upon reviewing the login event trigger, I noticed a potential typo about the SetDatabaseHasLoginEventTriggers function name. diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c index 0d3214df9c..7a5ed6b985 100644 --- a/src/backend/commands/event_trigger

Re: POC: GROUP BY optimization

2024-04-15 Thread Andrei Lepikhov
On 4/12/24 06:44, Tom Lane wrote: * The very first hunk causes get_eclass_for_sort_expr to have side-effects on existing EquivalenceClass data structures. What is the argument that that's not going to cause problems? At the very least it introduces asymmetry, in that the first caller wins the cha

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-04-15 Thread Michael Paquier
On Tue, Apr 16, 2024 at 02:14:21PM +0900, Michael Paquier wrote: > It seems to me that we are going to extend the GUC > default_table_access_method with a "default" mode to be able to force > relam to 0 and make a difference with the non-0 case, in the same way > as ALTER TABLE SET ACCESS METHOD DE

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 10:46:00AM +0900, Michael Paquier wrote: > There is no need for a catalog here to trigger the failure, and it > would have happened as long as a foreign table is used. The problem > introduced in 374c7a229042 fixed by e2395cdbe83a comes from a thinko > on my side, my apolog

Re: Add bump memory context type and use it for tuplesorts

2024-04-15 Thread Amul Sul
Attached is a small patch adding the missing BumpContext description to the README. Regards, Amul 0001-Add-BumpContext-description-to-mmgr-README.patch Description: Binary data

Re: Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 11:56:40PM -0400, Tom Lane wrote: > Thanks! indri has reported on some branches, and is now green for REL_12_STABLE and REL_13_STABLE. The rest should be OK. -- Michael signature.asc Description: PGP signature

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread shveta malik
On Tue, Apr 16, 2024 at 9:27 AM Zhijie Hou (Fujitsu) wrote: > > On Monday, April 15, 2024 6:09 PM shveta malik wrote: > > > > Please find v4 addressing the above comments. > > Thanks for the patch. > > Here are few comments: Thanks for reviewing the patch. > > 1. > > + ere

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread David Rowley
On Tue, 16 Apr 2024 at 14:29, Andres Freund wrote: > I think total_nblocks might also not be entirely stable? I think it is stable for this test. However, I'll let the buildfarm make the final call on that. The reason I want to include it is that I'd like to push the large allocations to the ta

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-04-15 Thread Masahiko Sawada
On Tue, Apr 2, 2024 at 7:34 PM torikoshia wrote: > > On 2024-04-01 11:31, Masahiko Sawada wrote: > > On Fri, Mar 29, 2024 at 11:54 AM torikoshia > > wrote: > >> > >> On 2024-03-28 21:54, Masahiko Sawada wrote: > >> > On Thu, Mar 28, 2024 at 9:38 PM torikoshia > >> > wrote: > >> >> > >> >> On 202

RE: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Zhijie Hou (Fujitsu)
On Monday, April 15, 2024 6:09 PM shveta malik wrote: > > Please find v4 addressing the above comments. Thanks for the patch. Here are few comments: 1. + ereport(ERROR, + errmsg("promotion in progress, can not synchronize replicatio

Re: Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Tom Lane
Michael Paquier writes: > On Mon, Apr 15, 2024 at 07:42:38PM -0400, Tom Lane wrote: >> Please, if you have the time. > Okay, done that in the 12~16 range then, removing all traces of > xmlParseMemory() including for xml_is_well_formed() in 12~14. That > should calm down indri. Thanks!

Re: "backend process" confused with "server process"

2024-04-15 Thread jian he
On Mon, Apr 15, 2024 at 5:27 PM Daniel Gustafsson wrote: > > > On 15 Apr 2024, at 11:07, Andrey M. Borodin wrote: > >> On 15 Apr 2024, at 14:01, jian he wrote: > > >> "is not a PostgreSQL server process" is the same thing as "not a > >> PostgreSQL backend process”? > > > > As far as I understand

Re: Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 07:42:38PM -0400, Tom Lane wrote: > Please, if you have the time. Okay, done that in the 12~16 range then, removing all traces of xmlParseMemory() including for xml_is_well_formed() in 12~14. That should calm down indri. -- Michael signature.asc Description: PGP signatur

Re: Stability of queryid in minor versions

2024-04-15 Thread Michael Paquier
On Tue, Apr 16, 2024 at 02:04:22PM +1200, David Rowley wrote: > It makes sense to talk about the hashing variations closer to the > object identifier part. Mostly what I had in mind. A separate for the new part you are adding at the end of the first part feels a bit more natural split here. Fee

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Amit Kapila
On Mon, Apr 15, 2024 at 7:47 PM Bertrand Drouvot wrote: > > On Mon, Apr 15, 2024 at 06:29:49PM +0530, Amit Kapila wrote: > > On Mon, Apr 15, 2024 at 6:21 PM Bertrand Drouvot > > wrote: > > > > > > On Mon, Apr 15, 2024 at 03:38:53PM +0530, shveta malik wrote: > > > > Now both worker and SQL > > >

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Andres Freund
Hi, On 2024-04-16 13:50:14 +1200, David Rowley wrote: > I think primarily it's good to exercise that code just to make sure it > does not crash. Looking at the output of the above on my machine: Agreed. > name | ident | parent | level | total_bytes | > total_nblocks | free_by

RE: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-15 Thread Hayato Kuroda (Fujitsu)
Dear Amit, > > FYI - We also considered the idea which walsender waits until all prepared > transactions > > are resolved before decoding and sending changes, but it did not work well > > - the restarted walsender sent only COMMIT PREPARED record for > transactions which > > have been prepared bef

Re: pg_combinebackup fails on file named INCREMENTAL.*

2024-04-15 Thread David Steele
On 4/16/24 06:33, Robert Haas wrote: On Sun, Apr 14, 2024 at 11:53 PM David Steele wrote: Since incremental backup is using INCREMENTAL as a keyword (rather than storing incremental info in the manifest) it is vulnerable to any file in PGDATA with the pattern INCREMENTAL.*. Yeah, that's true.

Re: Stability of queryid in minor versions

2024-04-15 Thread David Rowley
On Tue, 16 Apr 2024 at 12:10, Michael Paquier wrote: > Not sure that this is an improvement in clarity. There are a few > bullet points that treat about the instability of the query ID, and > your patch is now mixing the query ID being different for two > mostly-identical queries on the same host

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread David Rowley
On Tue, 16 Apr 2024 at 10:57, Andres Freund wrote: > I guess was thinking more about BumpIsEmpty() and BumpStats() then the "bogus" > cases. But BumpIsEmpty() likely is unreachable as well. The only call to MemoryContextIsEmpty() I see is AtSubCommit_Memory() and it's on an aset.c context type. I

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Tom Lane
Jeff Davis writes: > On Mon, 2024-04-15 at 17:05 -0700, Andres Freund wrote: >> I don't at all like that the tests depend on downloading new unicode >> data. What if there was an update but I just want to test the current >> state? > I was mostly following the precedent for normalization. Should

Re: Bugs in ecpg's macro mechanism

2024-04-15 Thread Tom Lane
Andres Freund writes: > On 2024-04-15 20:47:16 -0400, Tom Lane wrote: >> Ah, thanks. I guess this depends on getopt_long reordering arguments >> (since the "-o outfile" bit will come later). That is safe enough >> in HEAD since 411b72034, but it might fail on weird platforms in v16. >> How much

Add memory context type to pg_backend_memory_contexts view

2024-04-15 Thread David Rowley
In [1] Andres mentioned that there's no way to determine the memory context type in pg_backend_memory_contexts. This is a bit annoying as I'd like to add a test to exercise BumpStats(). Having the context type in the test's expected output helps ensure we are exercising BumpStats() and any future

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Jeff Davis
On Mon, 2024-04-15 at 17:05 -0700, Andres Freund wrote: > Can't we test this as part of the normal testsuite? One thing that complicates things a bit is that the test compares the results against ICU, so a mismatch in Unicode version between ICU and Postgres can cause test failures. The test ignor

Re: Bugs in ecpg's macro mechanism

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 20:47:16 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2024-04-15 17:48:32 -0400, Tom Lane wrote: > >> But I have no idea about making it work in meson. Any suggestions? > > > So you just want to compile define.c twice? The below should suffice: > > > - 'define': ['-

Re: What's our minimum ninja version?

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 20:26:49 -0400, Tom Lane wrote: > installation.sgml says our minimum meson version is 0.54, but it's > silent on what the minimum ninja version is. What RHEL8 supplies > doesn't work for me: Yea. There was some thread where we'd noticed that, think you were on that too. We cou

Re: Bugs in ecpg's macro mechanism

2024-04-15 Thread Tom Lane
Andres Freund writes: > On 2024-04-15 17:48:32 -0400, Tom Lane wrote: >> But I have no idea about making it work in meson. Any suggestions? > So you just want to compile define.c twice? The below should suffice: > - 'define': ['-DCMDLINESYM=123'], > + 'define': ['-DCMDLINESYM=123', files('def

Re: Security lessons from liblzma - libsystemd

2024-04-15 Thread Michael Paquier
On Fri, Apr 12, 2024 at 09:00:11AM -0700, Andres Freund wrote: > I'm actually fairly bothered by us linking to libxml2. It was effectively > unmaintained for most of the last decade, with just very occasional drive-by > commits. And it's not that there weren't significant bugs or such. Maintenance

What's our minimum ninja version?

2024-04-15 Thread Tom Lane
installation.sgml says our minimum meson version is 0.54, but it's silent on what the minimum ninja version is. What RHEL8 supplies doesn't work for me: $ meson setup build ... Found ninja-1.8.2 at /usr/bin/ninja ninja: error: build.ninja:7140: multiple outputs aren't (yet?) supported by depslog

Re: Stability of queryid in minor versions

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 02:54:52PM +1200, David Rowley wrote: > pg_stat_statements will consider two > apparently-identical > queries to be distinct, if they reference a table that was dropped > and recreated between the executions of the two queries. > + Two servers participating in

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 16:53:48 -0700, Jeff Davis wrote: > On Sun, 2024-04-14 at 15:33 -0700, Andres Freund wrote: > > - Coverage for some of the new unicode code is pretty poor: > >   > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/common/unicode_category.c.gcov.html#L122 > > Thank you

Re: pg17 issues with not-null contraints

2024-04-15 Thread Justin Pryzby
On Mon, Apr 15, 2024 at 03:47:38PM +0200, Alvaro Herrera wrote: > On 2024-Apr-15, Justin Pryzby wrote: > > > I think there are other issues related to b0e96f3119 (Catalog not-null > > constraints) - if I dump a v16 server using v17 tools, the backup can't > > be restored into the v16 server. I'm

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Jeff Davis
On Sun, 2024-04-14 at 15:33 -0700, Andres Freund wrote: > - Coverage for some of the new unicode code is pretty poor: >   > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/common/unicode_category.c.gcov.html#L122 Thank you for looking. Those functions are tested by category_test.c which

Re: Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Tom Lane
Michael Paquier writes: > On Mon, Apr 15, 2024 at 07:14:22PM -0400, Tom Lane wrote: >> I could switch the animal to use -Wno-deprecated-declarations in the >> back branches, but I'd rather not. I think the right answer is to >> back-patch Michael's 65c5864d7 (xml2: Replace deprecated routines wit

[18] clarify the difference between pg_wchar, wchar_t, and Unicode code points

2024-04-15 Thread Jeff Davis
I'm not sure I understand all of the history behind pg_wchar, but it seems to be some blend of: (a) Postgres's own internal representation of a decoded character (b) libc's wchar_t (c) Unicode code point For example, Postgres has its own encoding/decoding routines, so (a) is the most obviou

Re: SQL function which allows to distinguish a server being in point in time recovery mode and an ordinary replica

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 04:06:03PM -0500, Tristan Partin wrote: > Saw your patch for the first time today. Looks like your patch is messed up? > You seem to have more of the diff at the bottom which seems to add a test. > Want to send a v2 with a properly formatted patch? FWIW, complicating more X

Re: Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 07:14:22PM -0400, Tom Lane wrote: > I could switch the animal to use -Wno-deprecated-declarations in the > back branches, but I'd rather not. I think the right answer is to > back-patch Michael's 65c5864d7 (xml2: Replace deprecated routines with > recommended ones). We spe

Re: Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 19:14:22 -0400, Tom Lane wrote: > I think the right answer is to > back-patch Michael's 65c5864d7 (xml2: Replace deprecated routines with > recommended ones). We speculated about that at the time (see e.g., > 400928b83) but didn't pull the trigger. I think 65c5864d7 has now > b

Time to back-patch libxml deprecation fixes?

2024-04-15 Thread Tom Lane
I just noticed that my animal indri has been failing in the back branches since I updated its MacPorts packages a few days ago: ccache gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attri

Re: Bugs in ecpg's macro mechanism

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 17:48:32 -0400, Tom Lane wrote: > I started looking into the ideas discussed at [1] about reimplementing > ecpg's string handling. Before I could make any progress I needed > to understand the existing input code, part of which is the macro > expansion mechanism ... and the more

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 11:07:05AM +0200, Daniel Gustafsson wrote: > On 15 Apr 2024, at 07:04, Michael Paquier wrote: >> On Fri, Apr 12, 2024 at 02:42:57PM +0200, Daniel Gustafsson wrote: >>> Is the attached split in line with how you were thinking about it? >> >> If I may, 0001 looks sensible he

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Andres Freund
Hi, On 2024-04-16 10:26:57 +1200, David Rowley wrote: > On Mon, 15 Apr 2024 at 10:33, Andres Freund wrote: > > - The new bump allocator has a fair amount of uncovered functionality: > > > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/backend/utils/mmgr/bump.c.gcov.html#L293 > >

Re: Stack overflow issue

2024-04-15 Thread Andres Freund
Hi, On 2024-03-06 14:17:23 +0200, Alexander Korotkov wrote: > 0001 Turn tail recursion into iteration in CommitTransactionCommand() > I did minor revision of comments and code blocks order to improve the > readability. After sending https://www.postgresql.org/message-id/20240414223305.m3i5eju6zyl

Re: Removing GlobalVisTestNonRemovableHorizon

2024-04-15 Thread Michael Paquier
On Mon, Apr 15, 2024 at 11:57:20AM -0700, Andres Freund wrote: > GlobalVisTestNonRemovableHorizon()/GlobalVisTestNonRemovableFullHorizon() only > existed for snapshot_too_old - but that was removed in f691f5b80a8. I'm > inclined to think we should remove those functions for 17. No new code should

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread David Rowley
On Mon, 15 Apr 2024 at 10:33, Andres Freund wrote: > - The new bump allocator has a fair amount of uncovered functionality: > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/backend/utils/mmgr/bump.c.gcov.html#L293 The attached adds a test to tuplesort to exercise BumpAllocLarge()

Re: Things I don't like about \du's "Attributes" column

2024-04-15 Thread David G. Johnston
On Sun, Feb 18, 2024 at 4:14 AM Pavel Luzanov wrote: > 2. Tom's advise: > > Not sure it's worth worrying about > > Show real values for 'Valid until' and 'Connection limit' without any hints. > > At this point I'm on board with retaining the \dr charter of simply being an easy way to access the d

Bugs in ecpg's macro mechanism

2024-04-15 Thread Tom Lane
I started looking into the ideas discussed at [1] about reimplementing ecpg's string handling. Before I could make any progress I needed to understand the existing input code, part of which is the macro expansion mechanism ... and the more I looked at that the more bugs I found, not to mention tha

Optimize numeric.c mul_var() using the Karatsuba algorithm

2024-04-15 Thread Joel Jacobson
Hi, This patch introduces the Karatsuba algorithm to speed up multiplication operations in numeric.c, where the operands have many digits. It is implemented via a new conditional in mul_var() that determines whether the sizes of the factors are sufficiently large to justify its use. This decisio

Re: Things I don't like about \du's "Attributes" column

2024-04-15 Thread David G. Johnston
On Sat, Apr 13, 2024 at 7:02 PM Wen Yi wrote: > I think we can change the output like this: > > postgres=# \du > List of roles > Role name | Login | Attributes | Password | Valid until | Connection > limit > > ---+---+-+--+

Re: make dist using git archive

2024-04-15 Thread Tristan Partin
On Wed Jan 24, 2024 at 11:57 AM CST, Tristan Partin wrote: On Wed Jan 24, 2024 at 10:18 AM CST, Tristan Partin wrote: > On Tue Jan 23, 2024 at 3:30 AM CST, Peter Eisentraut wrote: > > On 22.01.24 21:04, Tristan Partin wrote: > > 3. Meson does not support tar.bz2 archives. Submitted https://githu

Re: SQL function which allows to distinguish a server being in point in time recovery mode and an ordinary replica

2024-04-15 Thread Tristan Partin
On Tue Mar 26, 2024 at 9:28 AM CDT, m.litsarev wrote: Hi, At present time, an existing pg_is_in_recovery() method is not enough to distinguish a server being in point in time recovery (PITR) mode and an ordinary replica because it returns true in both cases. That is why pg_is_standby_requeste

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 15:36:04 -0400, Robert Haas wrote: > On Sun, Apr 14, 2024 at 6:33 PM Andres Freund wrote: > > - Some of the new walsummary code could use more tests. > > > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/backend/backup/walsummaryfuncs.c.gcov.html#L69 > > So this

Re: pg_combinebackup fails on file named INCREMENTAL.*

2024-04-15 Thread Robert Haas
On Sun, Apr 14, 2024 at 11:53 PM David Steele wrote: > Since incremental backup is using INCREMENTAL as a keyword (rather than > storing incremental info in the manifest) it is vulnerable to any file > in PGDATA with the pattern INCREMENTAL.*. Yeah, that's true. I'm not greatly concerned about th

Re: Table AM Interface Enhancements

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 16:02:00 -0400, Robert Haas wrote: > On Mon, Apr 15, 2024 at 3:47 PM Andres Freund wrote: > > That said, I don't like the state after applying > > https://postgr.es/m/CAPpHfdvuT6DnguzaV-M1UQ2whYGDojaNU%3D-%3DiHc0A7qo9HBEJw%40mail.gmail.com > > because there's too much coupling.

Re: Table AM Interface Enhancements

2024-04-15 Thread Andres Freund
Hi, On 2024-04-12 01:04:03 +0300, Alexander Korotkov wrote: > 1) If we just apply my revert patch and leave c6fc50cb4028 and > 041b96802ef in the tree, then we get our table AM API narrowed. As > you expressed the current API requires block numbers to be 1:1 with > the actual physical on-disk loc

Re: documentation structure

2024-04-15 Thread Robert Haas
On Mon, Apr 8, 2024 at 10:15 AM Peter Eisentraut wrote: > > Here is a new version of this patch. I think this is v18 material at > > this point, absent an outcry to the contrary. Sometimes we're flexible > > about doc patches. > > Looks good to me. I think this could go into PG17. Hearing no obj

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 3:47 PM Andres Freund wrote: > That said, I don't like the state after applying > https://postgr.es/m/CAPpHfdvuT6DnguzaV-M1UQ2whYGDojaNU%3D-%3DiHc0A7qo9HBEJw%40mail.gmail.com > because there's too much coupling. Hence talking about needing to iterate on > the interface in s

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Dave Cramer
On Mon, 15 Apr 2024 at 15:38, Jelte Fennema-Nio wrote: > On Mon, 15 Apr 2024 at 19:43, Robert Haas wrote: > > > > On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > > > I think for clients/drivers, the work would generally be pretty > > > minimal. For almost all proposed changes, clients

Re: Table AM Interface Enhancements

2024-04-15 Thread Andres Freund
Hi, On 2024-04-15 23:14:01 +0400, Pavel Borisov wrote: > Why it makes a difference looks a little bit unclear to me, I can't comment > on this. I noticed that before 041b96802ef we had a block number and block > sampler state that tied acquire_sample_rows() to the actual block > structure. That,

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Jelte Fennema-Nio
On Mon, 15 Apr 2024 at 19:43, Robert Haas wrote: > > On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > > I think for clients/drivers, the work would generally be pretty > > minimal. For almost all proposed changes, clients can "support" the > > protocol version update by simply not using

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Robert Haas
On Sun, Apr 14, 2024 at 6:33 PM Andres Freund wrote: > - Some of the new walsummary code could use more tests. > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/backend/backup/walsummaryfuncs.c.gcov.html#L69 So this is pg_wal_summary_contents() and pg_get_wal_summarizer_state(). I

Re: Table AM Interface Enhancements

2024-04-15 Thread Pavel Borisov
On Mon, 15 Apr 2024 at 22:09, Robert Haas wrote: > On Mon, Apr 15, 2024 at 12:37 PM Pavel Borisov > wrote: > > In my understanding, the downside of 041b96802ef is bringing > read_stream* things from being heap-only-related up to the level of > acquire_sample_rows() that is not supposed to be tie

Re: Removing GlobalVisTestNonRemovableHorizon

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 2:57 PM Andres Freund wrote: > GlobalVisTestNonRemovableHorizon()/GlobalVisTestNonRemovableFullHorizon() only > existed for snapshot_too_old - but that was removed in f691f5b80a8. I'm > inclined to think we should remove those functions for 17. No new code should > use th

Re: SET ROLE documentation improvement

2024-04-15 Thread Nathan Bossart
On Fri, Apr 12, 2024 at 09:54:24AM +0900, Michael Paquier wrote: > On Thu, Apr 11, 2024 at 02:21:49PM -0500, Nathan Bossart wrote: >> No objections here. I'll give this a few days for others to comment. I'm >> not particularly interested in back-patching this since it's arguably not >> fixing any

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 12:41 PM Nazir Bilal Yavuz wrote: > I agree with you but I did not understand one thing. If out-of-core > AMs are used, does not all block sampling logic (BlockSampler_Init(), > BlockSampler_Next() etc.) need to be edited as well since these > functions assume block numbers

Removing GlobalVisTestNonRemovableHorizon

2024-04-15 Thread Andres Freund
Hi, GlobalVisTestNonRemovableHorizon()/GlobalVisTestNonRemovableFullHorizon() only existed for snapshot_too_old - but that was removed in f691f5b80a8. I'm inclined to think we should remove those functions for 17. No new code should use them. Greetings, Andres Freund

Re: Differential code coverage between 16 and HEAD

2024-04-15 Thread Peter Geoghegan
On Sun, Apr 14, 2024 at 6:33 PM Andres Freund wrote: > - Some of the new nbtree code could use a bit more tests: > > https://anarazel.de/postgres/cov/16-vs-HEAD-2024-04-14/src/backend/access/nbtree/nbtutils.c.gcov.html#L1468 I made a conscious decision to not add coverage for the function that

Re: Parallel CREATE INDEX for BRIN indexes

2024-04-15 Thread Tomas Vondra
On 4/15/24 10:18, Tomas Vondra wrote: > ... > > I'll try a bit more to make this work without the temp table. > Considering the earlier discussion in e2933a6e1, I think making the table TEMP is the best fix, so I'll do that. Thanks for remembering that change, Alexander! Attached is the cleanup

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 12:37 PM Pavel Borisov wrote: > In my understanding, the downside of 041b96802ef is bringing read_stream* > things from being heap-only-related up to the level of acquire_sample_rows() > that is not supposed to be tied to heap. And changing *_analyze_next_block() > funct

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Robert Haas
[ Hit send too early, sorry. ] On Mon, Apr 15, 2024 at 1:43 PM Robert Haas wrote: > But the wire protocol changes very slowly, and I think that is a > difference that actually matters quite a bit here. Broadly speaking, I > can use a psq ...a psql that I just built today to talk to a server from

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-04-15 Thread Robert Haas
On Sat, Apr 6, 2024 at 6:14 PM Jelte Fennema-Nio wrote: > I think for clients/drivers, the work would generally be pretty > minimal. For almost all proposed changes, clients can "support" the > protocol version update by simply not using the new features, ... I mean, I agree if a particular proto

Re: allow changing autovacuum_max_workers without restarting

2024-04-15 Thread Imseih (AWS), Sami
> Another option could be to just remove the restart-only GUC and hard-code > the upper limit of autovacuum_max_workers to 64 or 128 or something. While > that would simplify matters, I suspect it would be hard to choose an > appropriate limit that won't quickly become outdated. Hardcoded values a

Re: Table AM Interface Enhancements

2024-04-15 Thread Nazir Bilal Yavuz
Hi, On Mon, 15 Apr 2024 at 18:36, Robert Haas wrote: > > On Sat, Apr 13, 2024 at 5:28 AM Alexander Korotkov > wrote: > > Yes, I think so. Table AM API deals with TIDs and block numbers, but > > doesn't force on what they actually mean. For example, in ZedStore > > [1], data is stored on per-c

Re: allow changing autovacuum_max_workers without restarting

2024-04-15 Thread Nathan Bossart
On Mon, Apr 15, 2024 at 11:28:33AM -0500, Nathan Bossart wrote: > On Mon, Apr 15, 2024 at 08:33:33AM -0500, Justin Pryzby wrote: >> On Wed, Apr 10, 2024 at 04:23:44PM -0500, Nathan Bossart wrote: >>> The proof-of-concept patch keeps autovacuum_max_workers as the maximum >>> number of slots to reser

Re: Table AM Interface Enhancements

2024-04-15 Thread Pavel Borisov
On Mon, 15 Apr 2024 at 19:36, Robert Haas wrote: > On Sat, Apr 13, 2024 at 5:28 AM Alexander Korotkov > wrote: > > Yes, I think so. Table AM API deals with TIDs and block numbers, but > > doesn't force on what they actually mean. For example, in ZedStore > > [1], data is stored on per-column B

Re: pg17 issues with not-null contraints

2024-04-15 Thread Alvaro Herrera
On 2024-Apr-15, Alvaro Herrera wrote: > On 2024-Apr-15, Justin Pryzby wrote: > > postgres=# CREATE TABLE iparent(id serial PRIMARY KEY); CREATE TABLE child > > (id int) INHERITS (iparent); ALTER TABLE child ALTER id DROP NOT NULL; > > ALTER TABLE child ADD CONSTRAINT p PRIMARY KEY (id); > > >

Re: allow changing autovacuum_max_workers without restarting

2024-04-15 Thread Nathan Bossart
On Mon, Apr 15, 2024 at 08:33:33AM -0500, Justin Pryzby wrote: > On Wed, Apr 10, 2024 at 04:23:44PM -0500, Nathan Bossart wrote: >> The proof-of-concept patch keeps autovacuum_max_workers as the maximum >> number of slots to reserve for workers, but I think we should instead >> rename this paramete

Re: Potential stack overflow in incremental base backup

2024-04-15 Thread Robert Haas
On Wed, Apr 10, 2024 at 9:55 PM Thomas Munro wrote: > Pushed. That fixes the stack problem. Cool. > Of course we still have this: > > /* > * Since this array is relatively large, avoid putting it on the stack. > * But we don't need it at all if this is not an incremental backup. >

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-15 Thread Robert Haas
On Mon, Apr 15, 2024 at 11:00 AM Alexander Lakhin wrote: > Initially I was confused by that message, because of: > CREATE TABLE t (i int) PARTITION BY RANGE (i); > CREATE FOREIGN TABLE ftp_0_1 PARTITION OF t >FOR VALUES FROM (0) TO (1) >SERVER loopback OPTIONS (table_name 'lt_0_1'); > CREA

Re: Table AM Interface Enhancements

2024-04-15 Thread Robert Haas
On Sat, Apr 13, 2024 at 5:28 AM Alexander Korotkov wrote: > Yes, I think so. Table AM API deals with TIDs and block numbers, but > doesn't force on what they actually mean. For example, in ZedStore > [1], data is stored on per-column B-trees, where TID used in table AM > is just a logical key of

Re: Slow catchup of 2PC (twophase) transactions on replica in LR

2024-04-15 Thread Давыдов Виталий
Dear All, On Wednesday, April 10, 2024 17:16 MSK, Давыдов Виталий wrote:  Hi Amit, Ajin, All Thank you for the patch and the responses. I apologize for my delayed answer due to some curcumstances. On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila wrote: Vitaly, does the minimal solution

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-15 Thread Dmitry Koval
Hi! Please, find a my version of this fix attached. Is it possible to make a small addition to the file v6-0001 ... .patch (see attachment)? Most important: 1) Line 19: + mergePartName = makeRangeVar(cmd->name->schemaname, tmpRelName, -1); (temporary table should use the same schema as th

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-15 Thread Alexander Lakhin
Hello Robert, 15.04.2024 17:30, Robert Haas wrote: On Sat, Apr 13, 2024 at 6:05 AM Alexander Korotkov wrote: Please, find a my version of this fix attached. I think we need to check relpersistence in a similar way ATTACH PARTITION or CREATE TABLE ... PARTITION OF do. I'm going to polish this

Re: Issue with the PRNG used by Postgres

2024-04-15 Thread Robert Haas
On Fri, Apr 12, 2024 at 3:33 PM Andres Freund wrote: > Here's a patch implementing this approach. I confirmed that before we trigger > the stuck spinlock logic very quickly and after we don't. However, if most > sleeps are interrupted, it can delay the stuck spinlock detection a good > bit. But th

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

2024-04-15 Thread Robert Haas
On Sat, Apr 13, 2024 at 6:05 AM Alexander Korotkov wrote: > Please, find a my version of this fix attached. I think we need to > check relpersistence in a similar way ATTACH PARTITION or CREATE TABLE > ... PARTITION OF do. I'm going to polish this a little bit more. + errmsg("\"%s\" is not an o

Re: cpluspluscheck/headerscheck require build in REL_16_STABLE

2024-04-15 Thread Marina Polyakova
On 2024-04-13 08:40, John Naylor wrote: On Fri, Apr 12, 2024 at 11:51 PM Marina Polyakova wrote: ... In the other branches everything is fine: these problems begin with commits [2] (jsonpath_gram.h) and [3] (gram.h) and in the master branch there're no such problems after commit [4]. The att

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Bertrand Drouvot
Hi, On Mon, Apr 15, 2024 at 06:29:49PM +0530, Amit Kapila wrote: > On Mon, Apr 15, 2024 at 6:21 PM Bertrand Drouvot > wrote: > > > > On Mon, Apr 15, 2024 at 03:38:53PM +0530, shveta malik wrote: > > > Now both worker and SQL > > > function set 'syncing' when they start and reset it when they exit

Re: pg17 issues with not-null contraints

2024-04-15 Thread Alvaro Herrera
On 2024-Apr-15, Justin Pryzby wrote: > 9b581c5341 can break dump/restore from old versions, including > pgupgrade. > > postgres=# CREATE TABLE iparent(id serial PRIMARY KEY); CREATE TABLE child > (id int) INHERITS (iparent); ALTER TABLE child ALTER id DROP NOT NULL; ALTER > TABLE child ADD CONS

Re: allow changing autovacuum_max_workers without restarting

2024-04-15 Thread Justin Pryzby
On Wed, Apr 10, 2024 at 04:23:44PM -0500, Nathan Bossart wrote: > The attached proof-of-concept patch demonstrates what I have in mind. > Instead of trying to dynamically change the global process table, etc., I'm > proposing that we introduce a new GUC that sets the effective maximum > number of a

Re: cataloguing NOT NULL constraints

2024-04-15 Thread Alvaro Herrera
(I think I had already argued this point, but I don't see it in the archives, so here it is again). On 2024-Feb-07, jian he wrote: > if you place CommandCounterIncrement inside the `if (recurse)` branch, > then the regression test will be ok. Yeah, but don't you think this is too magical? I mea

Re: promotion related handling in pg_sync_replication_slots()

2024-04-15 Thread Amit Kapila
On Mon, Apr 15, 2024 at 6:21 PM Bertrand Drouvot wrote: > > On Mon, Apr 15, 2024 at 03:38:53PM +0530, shveta malik wrote: > > On Mon, Apr 15, 2024 at 2:29 PM Amit Kapila wrote: > > > > > > On Fri, Apr 12, 2024 at 5:25 PM shveta malik > > > wrote: > > > > > > > > Please find v3 addressing race-c

  1   2   >