RE: Design of pg_stat_subscription_workers vs pgstats

2022-02-26 Thread osumi.takami...@fujitsu.com
On Saturday, February 26, 2022 11:51 AM Amit Kapila wrote: > I have reviewed the latest version and made a few changes along with fixing > some of the pending comments by Peter Smith. The changes are as > follows: (a) Removed m_databaseid in PgStat_MsgSubscriptionError as that is > not required n

Re: [PATCH] pg_permissions

2022-02-26 Thread Joel Jacobson
On Fri, Feb 25, 2022, at 22:12, Chapman Flack wrote: > I would be happy to review this patch, but a look through the email leaves me > thinking it may still be waiting on a C implementation of pg_get_acl(). Is > that > right? Not sure. > And perhaps a view rename to pg_privileges, following Pete

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-26 Thread Bharath Rupireddy
On Sat, Feb 26, 2022 at 1:08 AM Nathan Bossart wrote: > > On Fri, Feb 25, 2022 at 08:31:37PM +0530, Bharath Rupireddy wrote: > > Thanks Satya and others for the inputs. Here's the v1 patch that > > basically allows async wal senders to wait until the sync standbys > > report their flush lsn back t

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-26 Thread Gunnar "Nick" Bluth
Am 26.02.22 um 06:51 schrieb Michael Paquier: On Fri, Feb 25, 2022 at 10:35:49AM +0100, Gunnar "Nick" Bluth wrote: Am 24.02.22 um 14:46 schrieb Daniel Gustafsson: Actually, I think this looks like a saner approach. Putting a config setting in two place (postgresql.conf and on the commandline f

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-26 Thread Bharath Rupireddy
On Sat, Feb 26, 2022 at 3:22 AM Hsu, John wrote: > > > On Fri, Feb 25, 2022 at 08:31:37PM +0530, Bharath Rupireddy wrote: > >> Thanks Satya and others for the inputs. Here's the v1 patch that > >> basically allows async wal senders to wait until the sync standbys > >> report their flush lsn back t

Re: C++ Trigger Framework

2022-02-26 Thread Shmuel Kamensky
OK, great thank you, Buce! I'll check back in with any interesting results when I have something running (probably in a few months to a year since I don't yet know C++ very well :-)).

Document ordering guarantees on INSERT/UPDATE RETURNING clause

2022-02-26 Thread Shay Rojansky
Hi all, I've seen various discussions around whether PG makes any guarantees on the ordering of rows returned by the RETURNING clause (e.g. [1]). In a nutshell, when executing a statement such as the following: CREATE TABLE foo (id INT PRIMARY KEY GENERATED ALWAYS AS IDENTITY, data INT); INSERT I

Re: Document ordering guarantees on INSERT/UPDATE RETURNING clause

2022-02-26 Thread David G. Johnston
On Sat, Feb 26, 2022 at 5:42 AM Shay Rojansky wrote: > FWIW I've received feedback from a SQL Server engineer that one definitely > should *not* depend on such ordering there, and that future optimizations > (e.g. parallel insertion of many rows) could result in row ordering which > differs from

Re: Document ordering guarantees on INSERT/UPDATE RETURNING clause

2022-02-26 Thread Julien Rouhaud
Hi, On Sat, Feb 26, 2022 at 06:25:22AM -0700, David G. Johnston wrote: > On Sat, Feb 26, 2022 at 5:42 AM Shay Rojansky wrote: > > > FWIW I've received feedback from a SQL Server engineer that one definitely > > should *not* depend on such ordering there, and that future optimizations > > (e.g. p

Re: why do hash index builds use smgrextend() for new splitpoint pages

2022-02-26 Thread Melanie Plageman
On Fri, Feb 25, 2022 at 11:17 PM Amit Kapila wrote: > > On Sat, Feb 26, 2022 at 3:01 AM Melanie Plageman > wrote: > > > > Since _hash_alloc_buckets() WAL-logs the last page of the > > splitpoint, is it safe to skip the smgrimmedsync()? What if the last > > page of the splitpoint doesn't end up ha

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-26 Thread Nathan Bossart
On Sat, Feb 26, 2022 at 02:17:50PM +0530, Bharath Rupireddy wrote: > A global min LSN of SendRqstPtr of all the sync standbys can be > calculated and the async standbys can send WAL up to global min LSN. > This is unlike what the v1 patch does i.e. async standbys will wait > until the sync standbys

Re: Mingw task for Cirrus CI

2022-02-26 Thread Andrew Dunstan
On 2/25/22 19:27, Andres Freund wrote: > Hi, > > Andrew, CCIng you both because you might be interested in the CI bit, and > because you might know the answer. > > > >> 2- src/pl/plpython/Makefile looks under "C:/Windows/system32/" for >> PYTHONDLL. gendef cannot open that file, give an error li

Fix a typo in pg_receivewal.c's get_destination_dir()

2022-02-26 Thread Bharath Rupireddy
Hi, It looks like there's a typo in pg_receivewal.c's get_destination_dir(), that is, use the function parameter dest_folder instead of global variable basedir in the error message. Although there's no harm in it as basedir is a global variable in pg_receivewal.c, let's use the function parameter

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-02-26 Thread Chapman Flack
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested This patch applies cleanly for me and passes installcheck-world. I ha

Re: Document ordering guarantees on INSERT/UPDATE RETURNING clause

2022-02-26 Thread Shay Rojansky
> > > That seems very reasonable; if the situation is similar on PostgreSQL, > > > then I'd suggest making that very clear in the INSERT[2] and UPDATE[3] docs. > > > > There is clearly no mention of such a guarantee in our documentation. > > Yes, which is just how SQL works: a set doesn't have any

Re: Reducing power consumption on idle servers

2022-02-26 Thread Magnus Hagander
On Mon, Feb 21, 2022 at 5:11 PM Simon Riggs wrote: > > On Sat, 19 Feb 2022 at 17:03, Andres Freund wrote: > > > > Hi, > > > > On 2022-02-19 14:10:39 +, Simon Riggs wrote: > > IMO we should instead consider either deprecating file based promotion, or > > adding an optional dependency on filesy

Re: Reducing power consumption on idle servers

2022-02-26 Thread Tom Lane
Magnus Hagander writes: >> Deprecating explicit file-based promotion is possible and simple, so >> that is the approach in the latest version of the patch. > Is there any actual use-case for this other than backwards > compatibility? The fundamental problem with signal-based promotion is that it

Missed condition-variable wakeups on FreeBSD

2022-02-26 Thread Tom Lane
About once a month over the last six months, my buildfarm animal florican has gotten stuck while running the core regression tests. The symptoms have looked very much the same each time: there is a backend with two parallel worker processes that are just sitting and not consuming any CPU time. Eac

Re: Missed condition-variable wakeups on FreeBSD

2022-02-26 Thread Andres Freund
Hi, On 2022-02-26 14:07:05 -0500, Tom Lane wrote: > About once a month over the last six months, my buildfarm animal > florican has gotten stuck while running the core regression tests. > The symptoms have looked very much the same each time: there is > a backend with two parallel worker processes

Re: Missed condition-variable wakeups on FreeBSD

2022-02-26 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 02:07:05PM -0500, Tom Lane wrote: > I don't know much about how gdb interacts with kernel calls on > FreeBSD, but I speculate that the poll(2) call returns with EINTR > after gdb releases the process, and then things resume fine, > suggesting that we lost an interrupt somewh

Re: explain_regress, explain(MACHINE), and default to explain(BUFFERS) (was: BUFFERS enabled by default in EXPLAIN (ANALYZE))

2022-02-26 Thread Justin Pryzby
Rebased over ebf6c5249b7db525e59563fb149642665c88f747. It looks like that patch handles only query_id, and this patch also tries to handle a bunch of other stuff. If it's helpful, feel free to kick this patch to a future CF. >From e58fffedc6f1cf471228fb3234faba35898678c3 Mon Sep 17 00:00:00 2001 F

Re: Commitfest manager for 2022-03

2022-02-26 Thread Greg Stark
On Sat, 26 Feb 2022 at 01:33, Julien Rouhaud wrote: > > On Sat, Feb 26, 2022 at 02:42:33PM +0900, Michael Paquier wrote: > > On Fri, Feb 25, 2022 at 01:58:55PM -0600, David Steele wrote: > > > On 2/25/22 12:39, Greg Stark wrote: > > >> I would like to volunteer. > > > > > I've been hoping somebody

Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-02-26 Thread Andres Freund
Hi, In two recent investigations in occasional test failures (019_replslot_limit.pl failures, AIO rebase) the problems are somehow tied to checkpointer. I don't yet know if actually causally related to precisely those failures, but when running e.g. 027_stream_regress.pl, I see phases in which ma

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-02-26 Thread Chapman Flack
On 02/26/22 11:48, Chapman Flack wrote: > This patch applies cleanly for me and passes installcheck-world. > I have not yet studied all of the changes in detail. I've now looked through the rest, and the only further thing I noticed was that xlog.c's do_pg_start_backup still has a tablespaces para

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-02-26 Thread Nathan Bossart
On Sat, Feb 26, 2022 at 04:48:52PM +, Chapman Flack wrote: > My biggest concerns are the changes to the SQL-visible pg_start_backup > and pg_stop_backup functions. When the non-exclusive API was implemented > (in 7117685), that was done with care (with a new optional argument to > pg_start_back

Re: Postgres restart in the middle of exclusive backup and the presence of backup_label file

2022-02-26 Thread Nathan Bossart
On Sat, Feb 26, 2022 at 05:03:04PM -0500, Chapman Flack wrote: > I've now looked through the rest, and the only further thing I noticed > was that xlog.c's do_pg_start_backup still has a tablespaces parameter > to receive a List* of tablespaces if the caller wants, but this patch > removes the comm

Re: Missed condition-variable wakeups on FreeBSD

2022-02-26 Thread Melanie Plageman
On Sat, Feb 26, 2022 at 2:07 PM Tom Lane wrote: > > About once a month over the last six months, my buildfarm animal > florican has gotten stuck while running the core regression tests. > The symptoms have looked very much the same each time: there is > a backend with two parallel worker processes

Re: [Proposal] Global temporary tables

2022-02-26 Thread Justin Pryzby
I read through this. Find attached some language fixes. You should be able to apply each "fix" patch on top of your own local branch with git am, and then squish them together. Let me know if you have trouble with that. I think get_seqence_start_value() should be static. (Or otherwise, it shoul

Re: Commitfest manager for 2022-03

2022-02-26 Thread Justin Pryzby
Can I suggest to update the CF APP to allow: | Target version: 16 I also suggest to update patches to indicate which are (not) being considered for v15. A few specific ones from myself: |https://commitfest.postgresql.org/37/2573/ |pg_dump - read data for some options from external fileRe

Re: Fix a typo in pg_receivewal.c's get_destination_dir()

2022-02-26 Thread Andres Freund
Hi, On 2022-02-26 22:04:13 +0530, Bharath Rupireddy wrote: > It looks like there's a typo in pg_receivewal.c's > get_destination_dir(), that is, use the function parameter dest_folder > instead of global variable basedir in the error message. Although > there's no harm in it as basedir is a global

Re: Adding CI to our tree

2022-02-26 Thread Andres Freund
Hi, > Subject: [PATCH 2/7] cirrus: upload changed html docs as artifacts I still think the determination of the base branch needs to be resolved before this can be considered. > Always run doc build; to allow them to be shown in cfbot, they should not be > skipped if the linux build fails. > >

Re: Adding CI to our tree

2022-02-26 Thread Andres Freund
On 2022-02-26 17:09:08 -0800, Andres Freund wrote: > You could put the script in src/tools/ci and call it from the script to avoid > the quoting issues. Might also be a good idea for the bulk of the docs / coverage stuff, even if there are no quoting issues.

Re: range_agg with multirange inputs

2022-02-26 Thread Chapman Flack
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested This applies (with some fuzz) and passes installcheck-world, but a re

Re: convert libpq uri-regress tests to tap test

2022-02-26 Thread Andres Freund
Hi, On 2022-02-25 17:52:29 -0800, Andres Freund wrote: > I'd like to commit 0001 and 0002 soon, unless somebody sees a reason not to? Pushed. Attached is the remainder, 0003, the move of libpq_pipeline to src/interfaces/libpq that I'm not planning to push for now. Regards, Andres >From 61a8972

Re: Adding CI to our tree

2022-02-26 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 05:09:08PM -0800, Andres Freund wrote: > > XXX: if this is run in the same task, the configure flags should probably be > > consistent ? > > What do you mean? I mean that commit to run CompilerWarnings unconditionally built docs with different flags than the other stuff in

Re: Adding CI to our tree

2022-02-26 Thread Andres Freund
Hi, On 2022-02-26 20:43:52 -0600, Justin Pryzby wrote: > This doesn't do the right thing - I just tried. > https://cirrus-ci.org/guide/writing-tasks/#environment-variables > | changesInclude function can be very useful for skipping some tasks when no > changes to sources have been made since the

Re: Adding CI to our tree

2022-02-26 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 06:50:00PM -0800, Andres Freund wrote: > Hi, > > On 2022-02-26 20:43:52 -0600, Justin Pryzby wrote: > > This doesn't do the right thing - I just tried. > > https://cirrus-ci.org/guide/writing-tasks/#environment-variables > > | changesInclude function can be very useful for

Re: [Proposal] Global temporary tables

2022-02-26 Thread Pavel Stehule
Hi You redirect stats from pg_class and pg_statistics to a local hash table. > This is pretty hairy :( > I guess you'd also need to handle pg_statistic_ext and ext_data. > pg_stats doesn't work, since the data isn't in pg_statistic - it'd need to > look > at pg_get_gtt_statistics. > Without this,

Re: Adding CI to our tree

2022-02-26 Thread Andres Freund
Hi, On 2022-02-26 21:10:57 -0600, Justin Pryzby wrote: > I did git commit --amend --no-edit and repushed to github to trigger a new CI > run, and it did this: https://github.com/justinpryzby/postgres/runs/5347878714 > > This is in a branch with changes to doc. I wasn't intending it to skip > buil

Re: [Proposal] Global temporary tables

2022-02-26 Thread Andres Freund
Hi, On 2022-02-27 04:17:52 +0100, Pavel Stehule wrote: > > You redirect stats from pg_class and pg_statistics to a local hash table. > > This is pretty hairy :( As is I think the patch is architecturally completely unacceptable. Having code everywhere to redirect to manually written in-memory cat

Re: [Proposal] Global temporary tables

2022-02-26 Thread Pavel Stehule
ne 27. 2. 2022 v 5:13 odesílatel Andres Freund napsal: > Hi, > > On 2022-02-27 04:17:52 +0100, Pavel Stehule wrote: > > > You redirect stats from pg_class and pg_statistics to a local hash > table. > > > This is pretty hairy :( > > As is I think the patch is architecturally completely unacceptabl

Add WAL recovery messages with log_wal_traffic GUC (was: add recovery, backup, archive, streaming etc. activity messages to server logs along with ps display)

2022-02-26 Thread Bharath Rupireddy
On Wed, Dec 29, 2021 at 6:50 AM Bharath Rupireddy wrote: > > On Thu, Dec 9, 2021 at 9:28 PM Alvaro Herrera wrote: > > > > Maybe some tunable like > > log_wal_traffic = { none, medium, high } > > where "none" is current behavior of no noise, "medium" gets (say) once > > every 256 segments (e.g., w