Progress report removal of temp files and temp relation files using ereport_startup_progress

2022-04-29 Thread Bharath Rupireddy
Hi, At times, there can be many temp files (under pgsql_tmp) and temp relation files (under removal which after crash may take longer during which users have no clue about what's going on in the server before it comes up online. Here's a proposal to use ereport_startup_progress to report the prog

Re: bogus: logical replication rows/cols combinations

2022-04-29 Thread Amit Kapila
On Sat, Apr 30, 2022 at 2:02 AM Tomas Vondra wrote: > > On 4/29/22 06:48, Amit Kapila wrote: > > On Thu, Apr 28, 2022 at 11:00 PM Tomas Vondra > > I think such issues due to ALTER of the publication are somewhat > expected, and I think users will understand they might need to resync > the subscrip

Re: failures in t/031_recovery_conflict.pl on CI

2022-04-29 Thread Tom Lane
Andres Freund writes: > Questions: > - I'm planning to backpatch the test as 031_recovery_conflict.pl, even though > preceding numbers are unused. It seems way more problematic to use a > different number in the backbranches than have gaps? +1 > - The test uses pump_until() and wait_for_log(

Re: bogus: logical replication rows/cols combinations

2022-04-29 Thread Tomas Vondra
On 4/29/22 06:48, Amit Kapila wrote: > On Thu, Apr 28, 2022 at 11:00 PM Tomas Vondra > wrote: >> >> On 4/28/22 14:26, Peter Eisentraut wrote: >>> On 27.04.22 12:33, Amit Kapila wrote: >>> >>> I wonder how we handle the combination of >>> >>> pub1: publish=insert WHERE (foo) >>> pub2: publish=updat

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread David G. Johnston
On Fri, Apr 29, 2022 at 12:31 PM Tom Lane wrote: > "David G. Johnston" writes: > > The fact that (baserel.rows > path->subpath->rows) here seems like a > > straight bug: there are no filters involved in this case but in the > > presence of filters baserel->rows should be strictly (<= > > path->s

Re: [RFC] building postgres with meson -v8

2022-04-29 Thread Andres Freund
Hi, On 2022-04-29 11:00:43 -0700, Andres Freund wrote: > On 2022-04-27 21:56:27 +0200, Peter Eisentraut wrote: > > Here is a patch that adds in NLS. > > Cool! I know very little about translations, so I was reticent tackling > this... > > The annoying thing is that the i18n module doesn't appear

Re: failures in t/031_recovery_conflict.pl on CI

2022-04-29 Thread Andres Freund
Hi, Attached are patches for this issue. It adds a test case for deadlock conflicts to make sure that case isn't broken. I also tested the recovery conflict tests in the back branches, and they work there with a reasonably small set of changes. Questions: - I'm planning to backpatch the test as

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Tom Lane
I wrote: > So perhaps we should do it more like the attached, which produces > this plan for the UNION case: sigh ... actually attached this time. regards, tom lane diff --git a/src/backend/optimizer/path/costsize.c b/src/backend/optimizer/path/costsize.c index b787c6f81a

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Tom Lane
"David G. Johnston" writes: > The fact that (baserel.rows > path->subpath->rows) here seems like a > straight bug: there are no filters involved in this case but in the > presence of filters baserel->rows should be strictly (<= > path->subpath->rows), right? No, because the subpath's rowcount has

Re: Use standard SIGHUP and SIGTERM handlers in autoprewarm module

2022-04-29 Thread Andres Freund
Hi, On 2020-12-16 18:12:39 +0900, Fujii Masao wrote: > - /* Wait to be signaled by UnpinBuffer() */ > + /* > + * Wait to be signaled by UnpinBuffer(). > + * > + * We assume that only UnpinBuffer() and the timeout requests > established > + * above can wake us up here.

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread David G. Johnston
On Fri, Apr 29, 2022 at 11:09 AM Tom Lane wrote: > > In short, these SubqueryScans are being labeled as producing 6 rows > when their input only produces 25000 rows, which is surely insane. > > So: even though the SubqueryScan itself isn't parallel-aware, the number > of rows it processes has

Re: allow specifying action when standby encounters incompatible parameter settings

2022-04-29 Thread Cary Huang
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed Hello The patch applied and tested fine. I think for this ki

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Tom Lane
I wrote: > So I think the actual problem here is that we leave the decision > to elide no-op SubqueryScan nodes till setrefs.c. We should detect > that earlier, and when it applies, label the SubqueryScanPath with > the exact cost of its child. Hmm ... actually, while doing that seems like it'd b

Re: [RFC] building postgres with meson -v8

2022-04-29 Thread Andres Freund
Hi, On 2022-04-27 21:56:27 +0200, Peter Eisentraut wrote: > Here is a patch that adds in NLS. Cool! I know very little about translations, so I was reticent tackling this... > For example, we could move the list of languages from the meson.build files > into separate LINGUAS files, which could

Re: [RFC] building postgres with meson -v8

2022-04-29 Thread Andres Freund
Hi, On 2022-04-20 15:09:31 +0200, Peter Eisentraut wrote: > On 13.04.22 12:26, Peter Eisentraut wrote: > > Some feedback and patches for your branch at > > 3274198960c139328fef3c725cee1468bbfff469: > > Here is another patch. It adds support for building ecpg. Cool! I have merged this, with a f

Re: Support logical replication of DDLs

2022-04-29 Thread Zheng Li
Hello, > You should forbid it. Unless you can decompose the command into multiple SQL > commands to make it a safe operation for logical replication. > > Let's say you want to add a column with a volatile default. > > ALTER TABLE foo ADD COLUMN bar double precision DEFAULT random(); > > If you rep

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Tom Lane
I wrote: > -ENOCAFFEINE, sorry about that. As penance for that blunder, I spent a little time looking into this issue (responding to Robert's upthread complaint that it wasn't explained clearly). See the test case in parallel_subquery.sql, attached, which is adapted from the test in incremental_s

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Tom Lane
I wrote: > Really? Maybe I misunderstand the case under consideration, but > what I think will be happening is that each worker will re-execute > the pushed-down subquery in full. Oh ... nevermind that: what I was thinking about was the SubLink/SubPlan case, which is unrelated to SubqueryScan. -

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread David G. Johnston
On Fri, Apr 29, 2022 at 7:02 AM Tom Lane wrote: > Richard Guo writes: > > Currently subquery scan is using rel->rows (if no parameterization), > > which I believe is not correct. That's not the size the subquery scan > > node in each worker needs to handle, as the rows have been divided > > acro

Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Tom Lane
Richard Guo writes: > Currently subquery scan is using rel->rows (if no parameterization), > which I believe is not correct. That's not the size the subquery scan > node in each worker needs to handle, as the rows have been divided > across workers by the parallel-aware node. Really? Maybe I mis

Re: Assorted small doc patches

2022-04-29 Thread David G. Johnston
Updated status of the set. On Wed, Apr 20, 2022 at 5:59 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > > v0001-database-default-name (-bugs, with a related cleanup suggestion as > well) > > https://www.postgresql.org/message-id/flat/CAKFQuwZvHH1HVSOu7EYjvshynk4pnDwC5RwkF%3DVfZJvmUskw

Re: SQL JSON compliance

2022-04-29 Thread Andrew Dunstan
On 2022-04-29 Fr 04:13, Peter Eisentraut wrote: > On 13.04.22 22:43, Andrew Dunstan wrote: >> Simon has just pointed out to me that as a result of recent commits, a >> number of things now should move from the unsupported table to the >> supported table in features.sgml. In particular, it looks t

Re: Building Postgres with lz4 on Visual Studio

2022-04-29 Thread Andrew Dunstan
On 2022-04-29 Fr 08:50, Andrew Dunstan wrote: > On 2022-04-26 Tu 16:26, Robert Haas wrote: >> On Wed, Apr 13, 2022 at 10:22 AM Melih Mutlu wrote: >>> What I realized is that c:\lz4\lib\liblz4.lib actually does not exist. >>> The latest versions of lz4, downloaded from [2], do not contain \liblz4.

Re: Building Postgres with lz4 on Visual Studio

2022-04-29 Thread Andrew Dunstan
On 2022-04-26 Tu 16:26, Robert Haas wrote: > On Wed, Apr 13, 2022 at 10:22 AM Melih Mutlu wrote: >> What I realized is that c:\lz4\lib\liblz4.lib actually does not exist. >> The latest versions of lz4, downloaded from [2], do not contain \liblz4.lib >> anymore, as opposed to what's written here

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

2022-04-29 Thread Bharath Rupireddy
On Tue, Apr 26, 2022 at 12:09 AM Stephen Frost wrote: > > I was thinking more specifically along the lines of "if there's > X GB > of WAL that hasn't been archived, give up on archiving anything new" > (which is how the pgbackrest option works). IMO, this option is dangerous because the users mig

Re: Skipping schema changes in publication

2022-04-29 Thread vignesh C
On Thu, Apr 28, 2022 at 4:50 PM osumi.takami...@fujitsu.com wrote: > > On Wednesday, April 27, 2022 9:50 PM vignesh C wrote: > > Thanks for the comments, the attached v3 patch has the changes for the same. > Hi > > Thank you for updating the patch. Several minor comments on v3. > > (1) commit mes

Re: Handle infinite recursion in logical replication setup

2022-04-29 Thread vignesh C
On Fri, Apr 29, 2022 at 8:08 AM Peter Smith wrote: > > Here are my review comments for the v11* patches. > > == > > v11-0001 - no more comments. LGTM > > == > > V11-0002 > > 1. doc/src/sgml/logical-replication.sgml > > + > + Bidirectional replication is useful in creating multi mast

Re: Add WAL recovery messages with log_wal_traffic GUC (was: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display)

2022-04-29 Thread Alvaro Herrera
Did we ever consider the idea of using a new pg_stat_wal_activity_progress view or something like that, using the backend_progress.c functionality? I don't see it mentioned in the thread. It's not an *exact* fit, since this is not about one "command" being executed by a "backend", but it seems a n

Re: Multi-Master Logical Replication

2022-04-29 Thread vignesh C
On Fri, Apr 29, 2022 at 2:35 PM Peter Smith wrote: > > On Fri, Apr 29, 2022 at 2:16 PM Yura Sokolov wrote: > > > > В Чт, 28/04/2022 в 17:37 +0530, vignesh C пишет: > > > On Thu, Apr 28, 2022 at 4:24 PM Yura Sokolov > > > wrote: > > > > В Чт, 28/04/2022 в 09:49 +1000, Peter Smith пишет: > > > >

Re: add log messages when replication slots become active and inactive (was Re: Is it worth adding ReplicationSlot active_pid to ReplicationSlotPersistentData?)

2022-04-29 Thread Alvaro Herrera
If I read the code right, this patch emits logs when pg_logical_slot_get_changes and pg_replication_slot_advance SQL functions are called. Is this desirable/useful, for the use case you stated at start of thread? I think it is most likely pointless. If you get rid of those, then the only acquisi

add log messages when replication slots become active and inactive (was Re: Is it worth adding ReplicationSlot active_pid to ReplicationSlotPersistentData?)

2022-04-29 Thread Bharath Rupireddy
On Sun, Feb 27, 2022 at 3:16 PM Bharath Rupireddy wrote: > > On Mon, Jan 10, 2022 at 6:50 AM Bharath Rupireddy > wrote: > > > > On Wed, Jan 5, 2022 at 12:13 PM Kyotaro Horiguchi > > wrote: > > > > Here's the v2 patch. Please provide your thoughts. > > > > > > Thanks! I have three comments on t

Re: Add WAL recovery messages with log_wal_traffic GUC (was: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display)

2022-04-29 Thread Bharath Rupireddy
On Sun, Feb 27, 2022 at 12:50 PM Bharath Rupireddy wrote: > > On Wed, Dec 29, 2021 at 6:50 AM Bharath Rupireddy > wrote: > > > > On Thu, Dec 9, 2021 at 9:28 PM Alvaro Herrera > > wrote: > > > > > > Maybe some tunable like > > > log_wal_traffic = { none, medium, high } > > > where "none" is curr

Re: Multi-Master Logical Replication

2022-04-29 Thread Peter Smith
On Fri, Apr 29, 2022 at 2:16 PM Yura Sokolov wrote: > > В Чт, 28/04/2022 в 17:37 +0530, vignesh C пишет: > > On Thu, Apr 28, 2022 at 4:24 PM Yura Sokolov > > wrote: > > > В Чт, 28/04/2022 в 09:49 +1000, Peter Smith пишет: > > > > > > > 1.1 ADVANTAGES OF MMLR > > > > > > > > - Increases write sca

Re: Re: fix cost subqueryscan wrong parallel cost

2022-04-29 Thread Richard Guo
On Fri, Apr 29, 2022 at 12:53 AM Robert Haas wrote: > Gather doesn't require a parallel aware subpath, just a parallel-safe > subpath. In a case like this, the parallel seq scan will divide the > rows from the underlying relation across the three processes executing > it. Each process will pass t

Re: SQL JSON compliance

2022-04-29 Thread Peter Eisentraut
On 13.04.22 22:43, Andrew Dunstan wrote: Simon has just pointed out to me that as a result of recent commits, a number of things now should move from the unsupported table to the supported table in features.sgml. In particular, it looks to me like all of these should move: This all looks correc