Re: proposal - psql - use pager for \watch command

2022-05-09 Thread Pavel Stehule
Hi út 13. 7. 2021 v 19:50 odesílatel Tom Lane napsal: > Thomas Munro writes: > > Pushed, after retesting on macOS (with the fixed pspg that has by now > > arrived in MacPorts), FreeBSD and Linux. Thanks! > > After playing with this along the way to fixing the sigwait issues, > I have a gripe/s

Re: Mark all GUC variable as PGDLLIMPORT

2022-05-09 Thread Michael Paquier
On Fri, May 06, 2022 at 04:49:24PM -0700, Andres Freund wrote: > Just noticed that > extern sigset_t UnBlockSig, > BlockSig, > StartupBlockSig; > are unadorned. mark_pgdllimport.pl is not able to capture this part of the change because of this logic, whe

Re: Support logical replication of DDLs

2022-05-09 Thread Bharath Rupireddy
On Sun, May 8, 2022 at 12:39 PM Dilip Kumar wrote: > > On Sat, May 7, 2022 at 9:38 AM Amit Kapila wrote: > > > > On Fri, May 6, 2022 at 10:21 PM Zheng Li wrote: > > > > > > > Attached is a set of two patches as an attempt to evaluate this > > > > approach. > > > > > > > > > > Thanks for explori

Re: Configuration Parameter/GUC value validation hook

2022-05-09 Thread Bharath Rupireddy
On Sat, May 7, 2022 at 12:11 AM Euler Taveira wrote: > > On Wed, May 4, 2022, at 8:12 AM, Bharath Rupireddy wrote: > > How about we provide a sample extension (limiting some important > parameters say shared_buffers, work_mem and so on to some > "reasonable/recommended" limits) in the core along w

Indent C code in flex and bison files

2022-05-09 Thread Peter Eisentraut
I suppose we'll do pgindent soon. I have semi-manually gone through the various .l and .y files and fixed up the formatting of the C code to be more in line with pgindent style. Most of that code was old, so I don't expect this to be such a large effort going forward. I also think a lot of

Re: Logical replication timeout problem

2022-05-09 Thread Masahiko Sawada
On Mon, May 9, 2022 at 3:47 PM Amit Kapila wrote: > > On Fri, May 6, 2022 at 12:42 PM wangw.f...@fujitsu.com > wrote: > > > > On Fri, May 6, 2022 at 9:54 AM Masahiko Sawada > > wrote: > > > On Wed, May 4, 2022 at 7:18 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, May 2, 2022 at 8:07 AM

Re: Support logical replication of DDLs

2022-05-09 Thread Amit Kapila
On Mon, May 9, 2022 at 12:46 PM Bharath Rupireddy wrote: > > On Sun, May 8, 2022 at 12:39 PM Dilip Kumar wrote: > > > > On Sat, May 7, 2022 at 9:38 AM Amit Kapila wrote: > > > > > > On Fri, May 6, 2022 at 10:21 PM Zheng Li wrote: > > > > > > > > > Attached is a set of two patches as an attempt

Indent protocol.sgml

2022-05-09 Thread Peter Eisentraut
protocol.sgml has some unusual indentation that keeps getting added on to with new additions in that file. I have gone through and reindented it to be more in line with the style elsewhere, so future editing hopefully won't be that painful on the eye. ;-) The diff can be seen here: https:

Re: Support logical replication of DDLs

2022-05-09 Thread Alvaro Herrera
On 2022-May-08, Dilip Kumar wrote: > On Sat, May 7, 2022 at 9:38 AM Amit Kapila wrote: > > I agree that it adds to our maintainability effort, like every time we > > enhance any DDL or add a new DDL that needs to be replicated, we > > probably need to change the deparsing code. But OTOH this app

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-05-09 Thread Bharath Rupireddy
On Tue, Apr 26, 2022 at 11:57 AM Laurenz Albe wrote: > > On Mon, 2022-04-25 at 19:51 +0530, Bharath Rupireddy wrote: > > With synchronous replication typically all the transactions (txns) > > first locally get committed, then streamed to the sync standbys and > > the backend that generated the tra

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-05-09 Thread Dilip Kumar
On Mon, May 9, 2022 at 2:50 PM Bharath Rupireddy wrote: > > On Tue, Apr 26, 2022 at 11:57 AM Laurenz Albe > wrote: > > > > On Mon, 2022-04-25 at 19:51 +0530, Bharath Rupireddy wrote: > > > With synchronous replication typically all the transactions (txns) > > > first locally get committed, then

Re: Finer grain log timestamps

2022-05-09 Thread Dagfinn Ilmari Mannsåker
David Fetter writes: > diff --git src/backend/utils/error/elog.c src/backend/utils/error/elog.c > index 55ee5423af..4698e32ab7 100644 > --- src/backend/utils/error/elog.c > +++ src/backend/utils/error/elog.c > @@ -2295,7 +2295,7 @@ char * > get_formatted_log_time(void) > { > pg_time_t

Re: An attempt to avoid locally-committed-but-not-replicated-to-standby-transactions in synchronous replication

2022-05-09 Thread Andrey Borodin
> On 9 May 2022, at 14:20, Bharath Rupireddy > wrote: > > On Tue, Apr 26, 2022 at 11:57 AM Laurenz Albe > wrote: >> >> While this may mitigate the problem, I don't think it will deal with >> all the cases which could cause a transaction to end up committed locally, >> but not on the synchr

Re: [PATCH] Add native windows on arm64 support

2022-05-09 Thread Niyas Sait
> Thanks for doing that! Your post is just a couple of days old, let's > see if we get a reply on that. Microsoft updated documentation [1] and clarified that ASLR cannot be disabled for Arm64 targets. Because ASLR can't be disabled on ARM, ARM64, or ARM64EC architectures, > /DYNAMICBASE:NO isn'

Re: configure openldap crash warning

2022-05-09 Thread Peter Eisentraut
On 06.05.22 21:25, Tom Lane wrote: I wrote: After thinking about this for awhile, it seems like the best solution is to make configure proceed like this: 1. Find libldap. 2. Detect whether it's OpenLDAP 2.5 or newer. 3. If not, try to find libldap_r. Here's a proposed patch for that. It se

Re: configure openldap crash warning

2022-05-09 Thread Peter Eisentraut
On 04.05.22 17:16, Tom Lane wrote: Hmm, I just tried it with up-to-date MacPorts, and it was a*complete* fail: I got all the deprecation warnings (so the system include headers were used), and both postgres and libpq.dylib still ended up linked against /System/Library/Frameworks/LDAP.framework/Ve

2022-05-12 release announcement draft

2022-05-09 Thread Jonathan S. Katz
Hi, Please see attached draft for the 2022-05-12 release announcement. One change: while normally we start the EOL notices for $NEXT_VERSION_EOL ~6 months from $EOL_DATE, I moved the EOL section up to be closer to the top of the announcement. This comes from feedback around ensuring we're giv

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-05-09 Thread Robert Haas
On Sun, May 8, 2022 at 7:30 PM Thomas Munro wrote: > Simple idea: how about logging the PID of processes that block > progress for too long? In the attached, I arbitrarily picked 5 > seconds as the wait time between LOG messages. Also, DEBUG1 messages > let you see the processing speed on eg bui

Re: should check interrupts in BuildRelationExtStatistics ?

2022-05-09 Thread Robert Haas
On Sun, May 8, 2022 at 11:36 PM Tom Lane wrote: > Michael Paquier writes: > > How long can the backend remain unresponsive? I don't think that > > anybody would object to the addition of some CHECK_FOR_INTERRUPTS() in > > areas where it would be efficient to make the shutdown quicker, but > > we

Re: Can postgres ever delete the recycled future WAL files to free-up disk space if max_wal_size is reduced or wal_recycle is set to off?

2022-05-09 Thread Bharath Rupireddy
On Fri, May 6, 2022 at 10:20 PM Tom Lane wrote: > > Bharath Rupireddy writes: > > Can postgres delete the recycled future WAL files once max_wal_size is > > reduced and/or wal_recycle is set to off? > > A checkpoint should do that, see RemoveOldXlogFiles. > > Maybe you have a broken WAL archiving

Re: Configuration Parameter/GUC value validation hook

2022-05-09 Thread Robert Haas
On Mon, May 9, 2022 at 3:44 AM Bharath Rupireddy wrote: > Thanks for providing thoughts. I'm personally not in favour of adding > any new syntax, as the new syntax would require some education and > changes to other layers. I see some downsides with new syntax: > 1) It will be a bit difficult to d

Re: Mark all GUC variable as PGDLLIMPORT

2022-05-09 Thread Robert Haas
On Mon, May 9, 2022 at 3:15 AM Michael Paquier wrote: > On Fri, May 06, 2022 at 04:49:24PM -0700, Andres Freund wrote: > > Just noticed that > > extern sigset_t UnBlockSig, > > BlockSig, > > StartupBlockSig; > > are unadorned. > > mark_pgdllimport.pl is

Re: Logical replication timeout problem

2022-05-09 Thread Euler Taveira
On Mon, May 9, 2022, at 3:47 AM, Amit Kapila wrote: > Thanks. The patch LGTM. I'll push and back-patch this after the > current minor release is done unless there are more comments related > to this work. Looks sane to me. (I only tested the HEAD version) + boolend_xact = ctx->end_xact;

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Euler Taveira
On Sun, May 8, 2022, at 5:51 PM, Jonathan S. Katz wrote: > Hi, > > Please see attached draft for the 2022-05-12 release announcement. LGTM. > Please provide feedback on accuracy and if there are any notable omissions. * Several fixes for `contrib/pageinspect` to improve overall stability. * Disab

Re: configure openldap crash warning

2022-05-09 Thread Tom Lane
Peter Eisentraut writes: > Btw., I was a bit puzzled about all this talk about deprecation > warnings, which I have not seen. I turns out that you only get those if > you use the OS compiler, not a third-party gcc installation. Ah-hah. > So in terms of my original message, my installation is

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Tom Lane
"Jonathan S. Katz" writes: > Please provide feedback on accuracy and if there are any notable omissions. Please remove this bit: > * Infinite endpoints are now disallowed in the timestamp variants of > `generate_series()`. Per last-minute discussion [1], I'm going to revert that change for now.

Privileges on PUBLICATION

2022-05-09 Thread Antonin Houska
Now that the user can specify rows and columns to be omitted from the logical replication [1], I suppose hiding rows and columns from the subscriber is an important use case. However, since the subscription connection user (i.e. the user specified in the CREATE SUBSCRIPTION ... CONNECTION ... comma

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Jonathan S. Katz
On 5/9/22 9:43 AM, Euler Taveira wrote: On Sun, May 8, 2022, at 5:51 PM, Jonathan S. Katz wrote: Hi, Please see attached draft for the 2022-05-12 release announcement. LGTM. Please provide feedback on accuracy and if there are any notable omissions. * Several fixes for `contrib/pageinspect`

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Jonathan S. Katz
On 5/9/22 10:06 AM, Tom Lane wrote: "Jonathan S. Katz" writes: Please provide feedback on accuracy and if there are any notable omissions. Please remove this bit: * Infinite endpoints are now disallowed in the timestamp variants of `generate_series()`. Per last-minute discussion [1], I'm

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Euler Taveira
On Mon, May 9, 2022, at 11:54 AM, Jonathan S. Katz wrote: > You raise a good point, which is in the release announcement, should we > prefix contrib modules with "contrib"? Perhaps instead of shorthand we > spell it out, e.g. "Several fixes for the `pageinspect` module..." Nowadays 'contrib' refe

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Robert Treat
On Mon, May 9, 2022 at 11:16 AM Euler Taveira wrote: > On Mon, May 9, 2022, at 11:54 AM, Jonathan S. Katz wrote: > You raise a good point, which is in the release announcement, should we > prefix contrib modules with "contrib"? Perhaps instead of shorthand we > spell it out, e.g. "Several fixes fo

postgres_fdw "parallel_commit" docs

2022-05-09 Thread Jonathan S. Katz
Hi, While researching PG15 features, I was trying to read through the docs[1] for the "parallel_commit" (04e706d4) feature in postgres_fdw to better understand what it does. I found myself becoming lost with the references to (sub)transaction and a few other items that, while accurate, may be

psql now shows zero elapsed time after an error

2022-05-09 Thread Tom Lane
Example (you need up-to-the-minute HEAD for this particular test case, but anything that runs a little while before failing will do): regression=# \timing Timing is on. regression=# select * from generate_series('2022-01-01 00:00'::timestamptz, 'infinity'::timestamptz,

Re: postgres_fdw "parallel_commit" docs

2022-05-09 Thread Justin Pryzby
On Mon, May 09, 2022 at 11:37:35AM -0400, Jonathan S. Katz wrote: > @@ -473,27 +473,25 @@ OPTIONS (ADD password_required 'false'); > parallel_commit (boolean) > > > - This option controls whether postgres_fdw commits > - remote (sub)transactions opened on a foreign

Re: Query generates infinite loop

2022-05-09 Thread Tom Lane
Corey Huinker writes: > The infinite-upper-bound-withlimit-pushdown counterexample makes sense, but > seems like we're using generate_series() only because we lack a function > that generates a series of N elements, without a specified upper bound, > something like > generate_finite_series(

Re: Privileges on PUBLICATION

2022-05-09 Thread Euler Taveira
On Mon, May 9, 2022, at 11:09 AM, Antonin Houska wrote: > Now that the user can specify rows and columns to be omitted from the logical > replication [1], I suppose hiding rows and columns from the subscriber is an > important use case. However, since the subscription connection user (i.e. the > us

Re: SQL/JSON: FOR ORDINALITY bug

2022-05-09 Thread Andrew Dunstan
On 2022-05-04 We 16:09, Erik Rijkers wrote: > Op 04-05-2022 om 21:12 schreef Andrew Dunstan: >> > > I don't see how rowseq can be anything but 1.  Each invocation of >>> >>> >>> After some further experimentation, I now think you must be right, >>> David. >>> >>> Also, looking at the DB2

Re: [PATCH] Add native windows on arm64 support

2022-05-09 Thread Michael Paquier
On Mon, May 09, 2022 at 12:44:22PM +0100, Niyas Sait wrote: > Microsoft updated documentation [1] and clarified that ASLR cannot be > disabled for Arm64 targets. > > Because ASLR can't be disabled on ARM, ARM64, or ARM64EC architectures, > > /DYNAMICBASE:NO isn't supported for these targets. Bett

Proposal: add a debug message about using geqo

2022-05-09 Thread KAWAMOTO Masaya
Hi, During query tuning, users may want to check if GEQO is used or not to generate a plan. However, users can not know it by simply counting the number of tables that appear in SQL. I know we can know it by enabling GEQO_DEBUG flag, but it needs recompiling, so I think it is inconvenient. So, I

waiting for reload in tests

2022-05-09 Thread Andres Freund
Hi, In a couple tests I (IIRC others as well) had the problem that a config reload isn't actually synchronous. I.e. a sequence like $node_primary->reload; $node_primary->safe_psql('postgres',...) isn't actually guaranteed to observe the config as reloaded in the the safe_psql(). It *typically* w

Re: waiting for reload in tests

2022-05-09 Thread Tom Lane
Andres Freund writes: > In a couple tests I (IIRC others as well) had the problem that a config reload > isn't actually synchronous. I.e. a sequence like > $node_primary->reload; > $node_primary->safe_psql('postgres',...) > isn't actually guaranteed to observe the config as reloaded in the the >

Re: waiting for reload in tests

2022-05-09 Thread Michael Paquier
On Mon, May 09, 2022 at 09:29:32PM -0400, Tom Lane wrote: > Brute force way: s/reload/restart/ That was my first thought, as it can be tricky to make sure that all the processes got the update because we don't publish such a state. One thing I was also thinking about would be to update pg_stat_ac

Re: waiting for reload in tests

2022-05-09 Thread Tom Lane
Michael Paquier writes: > On Mon, May 09, 2022 at 09:29:32PM -0400, Tom Lane wrote: >> Less brute force: wait for "SHOW variable-you-changed" to report the >> value you expect. > This method may still be unreliable in some processes like a logirep > launcher/receiver or just autovacuum, no? Yeah

Re: [PoC] Improve dead tuple storage for lazy vacuum

2022-05-09 Thread Masahiko Sawada
Hi, On Sun, Feb 13, 2022 at 12:39 PM Andres Freund wrote: > > On 2022-02-13 12:36:13 +0900, Masahiko Sawada wrote: > > Actually, I'm working on simplifying and improving radix tree > > implementation for PG16 dev cycle. From the discussion so far I think > > it's better to have a data structure t

Re: waiting for reload in tests

2022-05-09 Thread Andres Freund
Hi, On 2022-05-09 21:42:20 -0400, Tom Lane wrote: > Michael Paquier writes: > > On Mon, May 09, 2022 at 09:29:32PM -0400, Tom Lane wrote: > >> Less brute force: wait for "SHOW variable-you-changed" to report the > >> value you expect. > > > This method may still be unreliable in some processes l

Re: 2022-05-12 release announcement draft

2022-05-09 Thread Jonathan S. Katz
On 5/9/22 11:19 AM, Robert Treat wrote: On Mon, May 9, 2022 at 11:16 AM Euler Taveira wrote: On Mon, May 9, 2022, at 11:54 AM, Jonathan S. Katz wrote: You raise a good point, which is in the release announcement, should we prefix contrib modules with "contrib"? Perhaps instead of shorthand we s

Re: strange slow query - lost lot of time somewhere

2022-05-09 Thread Justin Pryzby
On Tue, May 03, 2022 at 02:13:18PM +1200, David Rowley wrote: > I'm wishing I put the initial hash table size and the final hash table > size in EXPLAIN + EXPLAIN ANALYZE now. Perhaps it's not too late for > v15 to do that so that it might help us figure things out in the > future. On Fri, May 06,

Re: psql now shows zero elapsed time after an error

2022-05-09 Thread Richard Guo
On Mon, May 9, 2022 at 11:56 PM Tom Lane wrote: > Example (you need up-to-the-minute HEAD for this particular test > case, but anything that runs a little while before failing will do): > > regression=# \timing > Timing is on. > regression=# select * from generate_series('2022-01-01 00:00'::times

Re: Logical replication timeout problem

2022-05-09 Thread Amit Kapila
On Mon, May 9, 2022 at 7:01 PM Euler Taveira wrote: > > On Mon, May 9, 2022, at 3:47 AM, Amit Kapila wrote: > > Thanks. The patch LGTM. I'll push and back-patch this after the > current minor release is done unless there are more comments related > to this work. > > Looks sane to me. (I only teste

RE: Logical replication timeout problem

2022-05-09 Thread wangw.f...@fujitsu.com
On Mon, May 9, 2022 at 11:23 AM Amit Kapila wrote: > On Mon, May 9, 2022 at 7:01 PM Euler Taveira wrote: > > > > On Mon, May 9, 2022, at 3:47 AM, Amit Kapila wrote: > > > > Thanks. The patch LGTM. I'll push and back-patch this after the > > current minor release is done unless there are more comm

Re: Skipping schema changes in publication

2022-05-09 Thread vignesh C
On Fri, May 6, 2022 at 8:05 AM Peter Smith wrote: > > On Thu, Apr 28, 2022 at 9:32 PM Amit Kapila wrote: > > > ... > > > > Another idea that occurred to me today for tables this is as follows: > > 1. Allow to mention except during create publication ... For All Tables. > > CREATE PUBLICATION pub1

Re: Privileges on PUBLICATION

2022-05-09 Thread Amit Kapila
On Tue, May 10, 2022 at 12:16 AM Euler Taveira wrote: > > On Mon, May 9, 2022, at 11:09 AM, Antonin Houska wrote: > > Now that the user can specify rows and columns to be omitted from the logical > replication [1], I suppose hiding rows and columns from the subscriber is an > important use case. H

pglogical setup in cascade replication architecture

2022-05-09 Thread Perumal Raj
Hi Team, I have cascade level streaming replication & WAL Apply setup . *Primary -> Secondary 1 -> Secondary 2,Secondary 3* *WAL Archive -> Secondary 4* I am expanding setup by adding another Replica from *Primary -> Secondary 5* for logical streaming. Do I need to install and configure pglogic

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-05-09 Thread Thomas Munro
On Tue, Apr 12, 2022 at 10:50 AM Andres Freund wrote: > On 2022-04-12 10:33:28 +1200, Thomas Munro wrote: > > Instead of bothering to create N different XXXPending variables for > > the different conflict "reasons", I used an array. Other than that, > > it's much like existing examples. > > It ki

Re: Perform streaming logical transactions by background workers and parallel apply

2022-05-09 Thread Masahiko Sawada
On Wed, May 4, 2022 at 12:50 PM Amit Kapila wrote: > > On Tue, May 3, 2022 at 9:45 AM Amit Kapila wrote: > > > > On Mon, May 2, 2022 at 5:06 PM Masahiko Sawada > > wrote: > > > > > > On Mon, May 2, 2022 at 6:09 PM Amit Kapila > > > wrote: > > > > > > > > On Mon, May 2, 2022 at 11:47 AM Masahi

Re: Perform streaming logical transactions by background workers and parallel apply

2022-05-09 Thread Masahiko Sawada
On Wed, May 4, 2022 at 8:44 AM Peter Smith wrote: > > On Tue, May 3, 2022 at 5:16 PM Peter Smith wrote: > > > ... > > > Avoiding unexpected differences like this is why I suggested the > > option should have to be explicitly enabled instead of being on by > > default as it is in the current patch