Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
Robert Haas writes: > On Wed, Feb 1, 2023 at 4:12 PM Tom Lane wrote: >> That being the case, I don't think moving the goalposts for hash >> function stability is going to lead to a workable solution. > I don't see that there is any easy, clean way to solve this in > released branches. The idea t

Re: pg_dump versus hash partitioning

2023-02-01 Thread Peter Geoghegan
On Wed, Feb 1, 2023 at 2:49 PM Tom Lane wrote: > It's precisely because you want to analyze it in the same terms > as range/list partitioning that we have these issues. Or we could > have built it on some other infrastructure than hash index opclasses > ... but we didn't do that, and now we have

Re: Perform streaming logical transactions by background workers and parallel apply

2023-02-01 Thread Peter Smith
Some minor review comments for v91-0001 == doc/src/sgml/config.sgml 1. -Allows streaming or serializing changes immediately in logical decoding. -The allowed values of logical_replication_mode are -buffered and immediate. When set -to immediate, stream

Re: pg_dump versus hash partitioning

2023-02-01 Thread David Rowley
On Thu, 2 Feb 2023 at 11:38, Tom Lane wrote: > > Peter Geoghegan writes: > > You mentioned "minor releases" here. Who said anything about that? > > I did: I'd like to back-patch the fix if possible. I think changing > the default --load-via-partition-root choice could be back-patchable. > > If R

Re: CI and test improvements

2023-02-01 Thread Thomas Munro
On Fri, Dec 30, 2022 at 4:59 PM Thomas Munro wrote: > On Wed, Nov 23, 2022 at 11:57 AM Justin Pryzby wrote: > > [PATCH 03/10] cirrus/macos: update to macos ventura > > I don't know any reason not to push this one too, but it's not time critical. Some observations: * macOS has a new release ever

Re: pg_dump versus hash partitioning

2023-02-01 Thread Tom Lane
David Rowley writes: > Digging into the history a bit, I found [2] and particularly [3] that > seem to indicate this option was thought about due to concerns about > hash functions not returning consistent results on different > architectures. I suspect it might have been defaulted to load into th

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

2023-02-01 Thread Michael Paquier
On Wed, Feb 01, 2023 at 03:06:16PM +1300, Thomas Munro wrote: > +1, go for it. It shouldn't affect Unix build releases, and on > Windows the function does nothing. Thanks for confirming. I am wondering what these animals may complain about next, but based on some tests on this buildfarm host wit

Re: CI and test improvements

2023-02-01 Thread Tom Lane
Thomas Munro writes: > Some observations: > * macOS has a new release every year in June[1] > * updates cease after three years[1] > * thus three releases are in support (by that definition) at a time > * we need an image on Cirrus; 13 appeared ~1 month later[2] > * we need Homebrew support; 13 a

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Michael Paquier
On Wed, Feb 01, 2023 at 10:18:27AM -0800, Andres Freund wrote: > On 2023-02-01 12:27:19 -0500, Tom Lane wrote: >> Andres Freund writes: >> The main thing that system() brings to the table is platform-specific >> knowledge of where the shell is. I'm not very sure that we want to >> wire in "/bin/s

Fwd: pgsql: Remove over-optimistic Assert.

2023-02-01 Thread Richard Guo
Resend this email to -hackers. Sorry for the noise. Thanks Richard -- Forwarded message - From: Richard Guo Date: Thu, Feb 2, 2023 at 9:51 AM Subject: Re: pgsql: Remove over-optimistic Assert. To: Tom Lane Cc: , PostgreSQL-development < pgsql-hack...@postgresql.org> On Thu,

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Michael Paquier
On Wed, Feb 01, 2023 at 02:35:55PM -0800, Nathan Bossart wrote: > Here is a first draft for the proposed stopgap fix. If we want to proceed > with this, I can provide patches for the back branches. > + /* > + * PreRestoreCommand() is used to tell the SIGTERM handler for the > startup >

Re: Worth using personality(ADDR_NO_RANDOMIZE) for EXEC_BACKEND on linux?

2023-02-01 Thread Michael Paquier
On Thu, Feb 02, 2023 at 10:06:15AM +0900, Michael Paquier wrote: > Thanks for confirming. I am wondering what these animals may complain > about next, but based on some tests on this buildfarm host with the > same configuration, things are looking OK once this stuff is applied > on 11~14. Actuall

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Tom Lane
Michael Paquier writes: > Hmm. Isn't that something that we should also document in startup.c > where both routines are defined? If we begin to use > PreRestoreCommand() and PostRestoreCommand() in more code paths in the > future, that could be again an issue. I was vaguely wondering about remo

Re: [PATCH] New [relation] option engine

2023-02-01 Thread vignesh C
On Wed, 1 Feb 2023 at 14:49, Alvaro Herrera wrote: > > On 2023-Jan-31, vignesh C wrote: > > > On Tue, 3 Jan 2023 at 18:38, vignesh C wrote: > > > > The patch does not apply on top of HEAD as in [1], please post a rebased > > > patch: > > > === Applying patches on top of PostgreSQL commit ID > >

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-02-01 Thread Amit Kapila
On Wed, Feb 1, 2023 at 3:10 PM shiy.f...@fujitsu.com wrote: > > On Mon, Jan 30, 2023 6:05 PM Takamichi Osumi (Fujitsu) > wrote: > > > > Kindly have a look at the attached v25. > > > > Thanks for your patch. Here are some comments. > > 1. > + /* > +* The min_apply_delay parameter is

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-02-01 Thread shveta malik
On Wed, Feb 1, 2023 at 5:42 PM Melih Mutlu wrote: > > Hi Shveta, > > shveta malik , 1 Şub 2023 Çar, 15:01 tarihinde şunu > yazdı: >> >> On Wed, Feb 1, 2023 at 5:05 PM Melih Mutlu wrote: >> 2) I found a crash in the previous patch (v9), but have not tested it >> on the latest yet. Crash happens w

Re: Weird failure with latches in curculio on v15

2023-02-01 Thread Michael Paquier
On Wed, Feb 01, 2023 at 09:34:44PM -0500, Tom Lane wrote: > Michael Paquier writes: >> Hmm. Isn't that something that we should also document in startup.c >> where both routines are defined? If we begin to use >> PreRestoreCommand() and PostRestoreCommand() in more code paths in the >> future, t

Re: Exit walsender before confirming remote flush in logical replication

2023-02-01 Thread Kyotaro Horiguchi
At Wed, 1 Feb 2023 14:58:14 +0530, Amit Kapila wrote in > On Wed, Feb 1, 2023 at 2:09 PM Masahiko Sawada wrote: > > > > Otherwise, we will end up terminating > > the WAL stream without the done message. Which will lead to an error > > message "ERROR: could not receive data from WAL stream: ser

Re: Fix GUC_NO_SHOW_ALL test scenario in 003_check_guc.pl

2023-02-01 Thread Michael Paquier
On Wed, Feb 01, 2023 at 02:29:23PM +0900, Michael Paquier wrote: > As also mentioned upthread by Tom, I am not sure that this combination > makes much sense, actually, because I don't see why one would never > want to know what is the effective value loaded for a parameter stored > in a file when h

Re: Exit walsender before confirming remote flush in logical replication

2023-02-01 Thread Masahiko Sawada
On Wed, Feb 1, 2023 at 6:28 PM Amit Kapila wrote: > > On Wed, Feb 1, 2023 at 2:09 PM Masahiko Sawada wrote: > > > > On Fri, Jan 20, 2023 at 7:45 PM Amit Kapila wrote: > > > > > > On Tue, Jan 17, 2023 at 2:41 PM Amit Kapila > > > wrote: > > > > > > > > Let me try to summarize the discussion til

Re: recovery modules

2023-02-01 Thread Michael Paquier
On Wed, Feb 01, 2023 at 01:23:26PM -0800, Nathan Bossart wrote: > Yeah, that's nicer. cfbot is complaining about a missing #include, so I > need to send a new revision anyway. Okay, the changes done here look straight-forward seen from here. I got one small-ish comment. +basic_archive_startup(A

Re: Exit walsender before confirming remote flush in logical replication

2023-02-01 Thread Amit Kapila
On Thu, Feb 2, 2023 at 10:48 AM Masahiko Sawada wrote: > > On Wed, Feb 1, 2023 at 6:28 PM Amit Kapila wrote: > > > > > > > In a case where pq_is_send_pending() doesn't become false > > > for a long time, (e.g., the network socket buffer got full due to the > > > apply worker waiting on a lock), I

Re: Exit walsender before confirming remote flush in logical replication

2023-02-01 Thread Amit Kapila
On Thu, Feb 2, 2023 at 10:04 AM Kyotaro Horiguchi wrote: > > At Wed, 1 Feb 2023 14:58:14 +0530, Amit Kapila > wrote in > > On Wed, Feb 1, 2023 at 2:09 PM Masahiko Sawada > > wrote: > > > > > > Otherwise, we will end up terminating > > > the WAL stream without the done message. Which will lead

Re: Exposing the lock manager's WaitForLockers() to SQL

2023-02-01 Thread Will Mortensen
Here is a first attempt at a WIP patch. Sorry about the MIME type. It doesn't take any locks on the tables, but I'm not super confident that that's safe, so any input would be appreciated. I omitted view support for simplicity, but if that seems like a requirement I'll see about adding it. I assu

Re: Add progress reporting to pg_verifybackup

2023-02-01 Thread Masahiko Sawada
On Wed, Feb 1, 2023 at 10:25 AM Michael Paquier wrote: > > On Fri, Jan 06, 2023 at 04:28:42PM +0900, Masahiko Sawada wrote: > > I've attached the simple patch to add the progress reporting option to > > pg_verifybackup. The progress information is displayed with --progress > > option only during t

Re: heapgettup refactoring

2023-02-01 Thread David Rowley
On Thu, 2 Feb 2023 at 10:12, Melanie Plageman wrote: > FWIW, I like setting rs_inited in heapgettup_initial_block() better in > the final refactor, but I agree with you that in this patch on its own > it is better in the body of heapgettup() and heapgettup_pagemode(). We can reconsider that when

Re: Add progress reporting to pg_verifybackup

2023-02-01 Thread Michael Paquier
On Thu, Feb 02, 2023 at 02:57:44PM +0900, Masahiko Sawada wrote: > It seems that the --progress option doesn't work with command_like() > since the progress information is written in stderr but command_like() > doesn't want it. What about command_checks_all()? It should check for stderr, stdout a

Re: run pgindent on a regular basis / scripted manner

2023-02-01 Thread Noah Misch
On Mon, Jan 30, 2023 at 03:42:09PM -0500, Bruce Momjian wrote: > On Sat, Jan 28, 2023 at 05:06:03PM -0800, Noah Misch wrote: > > On Tue, Jan 24, 2023 at 02:04:02PM -0500, Bruce Momjian wrote: > > > On Tue, Jan 24, 2023 at 09:54:57AM -0500, Tom Lane wrote: > > > > As another example, the mechanisms

Re: Add index scan progress to pg_stat_progress_vacuum

2023-02-01 Thread Masahiko Sawada
On Fri, Jan 20, 2023 at 11:38 PM Imseih (AWS), Sami wrote: > > >Number of indexes that will be vacuumed or cleaned up. This counter only > >advances when the phase is vacuuming indexes or cleaning up indexes. > > I agree, this reads better. > > --- > -/* Report that we are

RE: Deadlock between logrep apply worker and tablesync worker

2023-02-01 Thread houzj.f...@fujitsu.com
On Tuesday, January 31, 2023 1:07 AM vignesh C wrote: > On Mon, 30 Jan 2023 at 17:30, vignesh C wrote: > > > > On Mon, 30 Jan 2023 at 13:00, houzj.f...@fujitsu.com > > wrote: > > > > > > On Monday, January 30, 2023 2:32 PM Amit Kapila > wrote: > > > > > > > > On Mon, Jan 30, 2023 at 9:20 AM vig

Re: run pgindent on a regular basis / scripted manner

2023-02-01 Thread Tom Lane
Noah Misch writes: > Regarding the concern about a pre-receive hook blocking an emergency push, the > hook could approve every push where a string like "pgindent: no" appears in a > commit message within the push. You'd still want to make the tree clean > sometime the same week or so. It's cheap

Re: RLS makes COPY TO process child tables

2023-02-01 Thread Yugo NAGATA
On Wed, 01 Feb 2023 11:47:23 -0500 Tom Lane wrote: > Yugo NAGATA writes: > > Antonin Houska wrote: > >> While working on [1] I noticed that if RLS gets enabled, the COPY TO > >> command > >> includes the contents of child table into the result, although the > >> documentation says it should no

Re: RLS makes COPY TO process child tables

2023-02-01 Thread Antonin Houska
Yugo NAGATA wrote: > On Wed, 01 Feb 2023 12:45:57 +0100 > Antonin Houska wrote: > > > While working on [1] I noticed that if RLS gets enabled, the COPY TO command > > includes the contents of child table into the result, although the > > documentation says it should not: > > > > "COPY TO c

<    1   2