Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-03-02 Thread Robert Haas
On Wed, Mar 2, 2022 at 3:00 PM Andres Freund wrote: > What I am stuck on is what we can do for the released branches. Data > corruption after two consecutive ALTER DATABASE SET TABLESPACEs seems like > something we need to address. I think we should consider back-porting the ProcSignalBarrier stu

Re: libpq compression (part 2)

2022-03-02 Thread Justin Pryzby
If there's no objection, I'd like to move this to the next CF for consideration in PG16. On Mon, Jan 17, 2022 at 10:39:19PM -0600, Justin Pryzby wrote: > On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > > => Since March, errmsg doesn't need extra parenthesis around it (e3a87b

Re: BufferAlloc: don't take two simultaneous locks

2022-03-02 Thread Yura Sokolov
В Вт, 01/03/2022 в 10:24 +0300, Yura Sokolov пишет: > Ok, here is v4. And here is v5. First, there was compilation error in Assert in dynahash.c . Excuse me for not checking before sending previous version. Second, I add third commit that reduces HASHHDR allocation size for non-partitioned dynah

Re: Logical replication timeout problem

2022-03-02 Thread Peter Smith
On Wed, Mar 2, 2022 at 1:06 PM wangw.f...@fujitsu.com wrote: > ... > Attach the new patch. [suggestion by Kuroda-San] It is difficult to read the thread and to keep track of who reviewed what, and what patch is latest etc, when every patch name is the same. Can you please introduce a version num

Re: Column Filtering in Logical Replication

2022-03-02 Thread Justin Pryzby
I applied this patch in my branch with CI hacks to show code coverage on cirrus. https://api.cirrus-ci.com/v1/artifact/task/6186186539532288/coverage/coverage/00-index.html Eyeballing it looks good. But GetActionsInPublication() isn't being hit at all? I think the queries in pg_dump should be wr

Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Kyotaro Horiguchi
At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy wrote in > On Wed, Mar 2, 2022 at 5:41 PM Nitin Jadhav > wrote: > > > > Hi, > > > > I have noticed that the CHECKPOINT_REQUESTED flag information is not > > present in the log message of LogCheckpointStart() function. I would > > like to unde

Re: PG DOCS - logical replication filtering

2022-03-02 Thread Peter Smith
On Wed, Mar 2, 2022 at 8:43 PM Aleksander Alekseev wrote: ... > Here is an updated version of the patch. Thanks for your review comments and fixes. The updated v2 patch looks good to me. -- Kind Regards, Peter Smith. Fujitsu Australia

Re: PG DOCS - logical replication filtering

2022-03-02 Thread Peter Smith
On Wed, Mar 2, 2022 at 8:26 PM Amit Kapila wrote: > > On Wed, Mar 2, 2022 at 2:37 PM Aleksander Alekseev > wrote: > > > > > > I see that a large part of the documentation is commented and marked as TBA > > (Column Filters, Combining Different Kinds of Filters). Could you please > > clarify if i

Re: Proposal: Support custom authentication methods using hooks,Re: Proposal: Support custom authentication methods using hooks

2022-03-02 Thread Tatsuo Ishii
> On 2/25/22 12:39 PM, Tom Lane wrote: >> Jeff Davis writes: >>> On Thu, 2022-02-24 at 20:47 -0500, Tom Lane wrote: ... and, since we can't readily enforce that the client only sends those cleartext passwords over suitably-encrypted connections, this could easily be a net negative f

Why do spgbuildempty(), btbuildempty(), and blbuildempty() use smgrwrite()?

2022-03-02 Thread Melanie Plageman
If you enable the CHECK_WRITE_VS_EXTEND-protected assert in mdwrite(), you'll trip it anytime you exercise btbuildempty(), blbuildempty(), or spgbuildempty(). In this case, it may not make any meaningful difference if smgrwrite() or smgrextend() is called (_mdfd_getseg() behavior won't differ even

Re: Avoiding smgrimmedsync() during nbtree index builds

2022-03-02 Thread Justin Pryzby
Rebased to appease cfbot. I ran these paches under a branch which shows code coverage in cirrus. It looks good to my eyes. https://api.cirrus-ci.com/v1/artifact/task/5212346552418304/coverage/coverage/00-index.html Are these patches being considered for v15 ? >From 30707da3e5eb68d1efbc5594696da4

Re: pg_stop_backup() v2 incorrectly marked as proretset

2022-03-02 Thread Michael Paquier
On Thu, Mar 03, 2022 at 12:36:32AM +0800, Julien Rouhaud wrote: > I don't see strong evidence for that pattern being wildly used with some naive > grepping: Yes, I don't recall either seeing the style with an undef a lot when it came to system functions. I'll move on and apply the fix in a minute

Re: pg_stop_backup() v2 incorrectly marked as proretset

2022-03-02 Thread Michael Paquier
On Wed, Mar 02, 2022 at 12:04:59PM -0500, Chapman Flack wrote: > I had just recently noticed that while reviewing [0], but shrugged, > as I didn't know what the history was. Okay. I did not see you mention it on the thread, but the discussion is long so it is easy to miss some of its details. >

Re: Changing "Hot Standby" to "hot standby"

2022-03-02 Thread Kyotaro Horiguchi
At Wed, 2 Mar 2022 15:22:44 +, "Daniel Westermann (DWE)" wrote in > > Pretty sure that for titles we should keep English capitalization rules. > > Done like that. Thanks for taking a look. -Hot Standby feedback propagates upstream, whatever the cascaded arrangement. +hot stan

Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Michael Paquier
On Thu, Mar 03, 2022 at 09:39:37AM +0900, Kyotaro Horiguchi wrote: > At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy > wrote in >> I don't think that's useful. Being in LogCheckpointStart >> (CreateCheckPoint or CreateRestartPoint) itself means that somebody >> has requested a checkpoint. H

Re: [Proposal] Global temporary tables

2022-03-02 Thread Pavel Stehule
Hi > I think you'd basically have to come up with a generic design for > partitioning > catalog tables into local / non-local storage, without needing explicit > code > for each catalog. That could also be used to store the default catalog > contents separately from user defined ones (e.g. pg_pro

Re: Design of pg_stat_subscription_workers vs pgstats

2022-03-02 Thread Andres Freund
Hi, On 2022-02-25 11:32:24 -0800, Andres Freund wrote: > On 2022-02-25 16:25:01 -0300, Euler Taveira wrote: > > On Fri, Feb 25, 2022, at 11:52 AM, Greg Stark wrote: > > > On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote: > > > > > > > > Hi, > > > > > > > > I was looking the shared memory stats p

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-02 Thread Ashutosh Sharma
On Wed, Mar 2, 2022 at 10:37 PM Bharath Rupireddy wrote: > > On Wed, Mar 2, 2022 at 8:12 PM Ashutosh Sharma wrote: > > > > Some review comments on v5 patch (v5-0001-pg_walinspect.patch) > > Thanks for reviewing. > > > +-- > > +-- pg_get_wal_records_info() > > +-- > > +CREATE FUNCTION pg_get_wal_r

Re: row filtering for logical replication

2022-03-02 Thread Amit Kapila
On Wed, Mar 2, 2022 at 5:42 PM Euler Taveira wrote: > > On Wed, Mar 2, 2022, at 8:45 AM, Tomas Vondra wrote: > > While working on the column filtering patch, which touches about the > same places, I noticed two minor gaps in testing: > > 1) The regression tests do perform multiple ALTER PUBLICATIO

Doc about how to set max_wal_senders when setting minimal wal_level

2022-03-02 Thread Japin Li
Hi, hackers When I try to change wal_level to minimal and restart the database, it complains max_wal_senders > 0. 2022-03-03 10:10:16.938 CST [6389] FATAL: WAL streaming (max_wal_senders > 0) requires wal_level "replica" or "logical" However, the documentation about wal_level [1] doesn't ment

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-03-02 Thread Kyotaro Horiguchi
Hi. +#ifdef FRONTEND +/* + * Functions that are currently not needed in the backend, but are better + * implemented inside xlogreader.c because of the internal facilities available + * here. + */ + #endif /* FRONTEND */ Why didn't you remove the em

Re: Why do spgbuildempty(), btbuildempty(), and blbuildempty() use smgrwrite()?

2022-03-02 Thread Kyotaro Horiguchi
At Wed, 2 Mar 2022 20:07:14 -0500, Melanie Plageman wrote in > If you enable the CHECK_WRITE_VS_EXTEND-protected assert in mdwrite(), > you'll trip it anytime you exercise btbuildempty(), blbuildempty(), or > spgbuildempty(). > > In this case, it may not make any meaningful difference if smgrwr

Re: Add CHECKPOINT_REQUESTED flag to the log message in LogCheckpointStart()

2022-03-02 Thread Kyotaro Horiguchi
At Thu, 3 Mar 2022 10:27:10 +0900, Michael Paquier wrote in > On Thu, Mar 03, 2022 at 09:39:37AM +0900, Kyotaro Horiguchi wrote: > > At Wed, 2 Mar 2022 18:18:10 +0530, Bharath Rupireddy > > wrote in > >> I don't think that's useful. Being in LogCheckpointStart > >> (CreateCheckPoint or Create

Re: PG DOCS - logical replication filtering

2022-03-02 Thread Amit Kapila
On Wed, Mar 2, 2022 at 8:00 PM Peter Eisentraut wrote: > > On 02.03.22 05:47, Peter Smith wrote: > > This patch introduces a new "Filtering" page to give a common place > > where all kinds of logical replication filtering can be described. > > (e.g. It is envisaged that a "Column Filters" section

Re: Doc about how to set max_wal_senders when setting minimal wal_level

2022-03-02 Thread David G. Johnston
On Wed, Mar 2, 2022 at 7:44 PM Japin Li wrote: > > Hi, hackers > > When I try to change wal_level to minimal and restart the database, it > complains > max_wal_senders > 0. > > 2022-03-03 10:10:16.938 CST [6389] FATAL: WAL streaming (max_wal_senders > > 0) requires wal_level "replica" or "logica

RE: Failed transaction statistics to measure the logical replication progress

2022-03-02 Thread osumi.takami...@fujitsu.com
On Wednesday, March 2, 2022 2:18 PM Masahiko Sawada wrote: > On Wed, Mar 2, 2022 at 10:21 AM osumi.takami...@fujitsu.com > wrote: > > Also, I quickly checked other similar views(pg_stat_slru, > > pg_stat_wal_receiver) commit logs, especially when they introduce columns. > > But, I couldn't find

RE: Failed transaction statistics to measure the logical replication progress

2022-03-02 Thread osumi.takami...@fujitsu.com
On Wednesday, March 2, 2022 5:29 PM Shi, Yu/侍 雨 wrote: > A comments on the v26 patch. Thank you for checking the patch ! > > The following document about pg_stat_subscription_stats view only says that > "showing statistics about errors", should we add something about transactions > here? > >

Re: logical replication empty transactions

2022-03-02 Thread Ajin Cherian
On Wed, Mar 2, 2022 at 1:01 PM shiy.f...@fujitsu.com wrote: > > Hi, > > Here are some comments on the v21 patch. > > 1. > + WalSndKeepalive(false, 0); > > Maybe we can use InvalidXLogRecPtr here, instead of 0. > Fixed. > 2. > + pq_sendint64(&output_message, writePtr ?

Re: Doc about how to set max_wal_senders when setting minimal wal_level

2022-03-02 Thread Japin Li
On Thu, 03 Mar 2022 at 11:25, David G. Johnston wrote: > I would suggest a wording more like: > > "A precondition for using minimal WAL is to disable WAL archiving and > streaming replication by setting max_wal_senders to 0, and archive_mode to > off." > > While accurate, the phrase "you must se

Re: PG DOCS - logical replication filtering

2022-03-02 Thread Peter Smith
On Thu, Mar 3, 2022 at 2:15 PM Amit Kapila wrote: > > On Wed, Mar 2, 2022 at 8:00 PM Peter Eisentraut > wrote: > > > > On 02.03.22 05:47, Peter Smith wrote: > > > This patch introduces a new "Filtering" page to give a common place > > > where all kinds of logical replication filtering can be desc

Re: Rewriting the test of pg_upgrade as a TAP test - take three - remastered set

2022-03-02 Thread Michael Paquier
On Wed, Mar 02, 2022 at 12:07:29AM -0800, Andres Freund wrote: >> +++ b/src/bin/pg_upgrade/t/001_basic.pl >> @@ -0,0 +1,9 @@ >> +use strict; >> +use warnings; >> + >> +use PostgreSQL::Test::Utils; >> +use Test::More tests => 8; > > Outdated. Fixed. >> +program_help_ok('pg_upgrade'); >> +program_

RE: row filtering for logical replication

2022-03-02 Thread shiy.f...@fujitsu.com
On Thu, Mar 3, 2022 10:40 AM Amit Kapila wrote: > > On Wed, Mar 2, 2022 at 5:42 PM Euler Taveira wrote: > > > > On Wed, Mar 2, 2022, at 8:45 AM, Tomas Vondra wrote: > > > > While working on the column filtering patch, which touches about the > > same places, I noticed two minor gaps in testing:

Re: Proposal: Support custom authentication methods using hooks

2022-03-02 Thread Tatsuo Ishii
> Yes, really, it's a known-broken system which suffers from such an old > and well known attack that it's been given a name: pass-the-hash. As > was discussed on this thread even, just the fact that it's not trivial > to break on the wire doesn't make it not-broken, particularly when we > use the

Re: Proposal: Support custom authentication methods using hooks

2022-03-02 Thread Michael Paquier
On Wed, Mar 02, 2022 at 11:15:28AM -0500, Stephen Frost wrote: > With... which? We removed recovery.conf without any warning between > major releases, yet it was used by every single PG file-based backup and > restore solution out there and by every single organization that had > ever done a resto

Re: Add the replication origin name and commit-LSN to logical replication worker errcontext

2022-03-02 Thread Amit Kapila
On Wed, Mar 2, 2022 at 1:05 PM Masahiko Sawada wrote: > > On Wed, Mar 2, 2022 at 4:14 PM Amit Kapila wrote: > > I've attached updated patches. > The first patch LGTM. Some comments on the second patch: 1. @@ -3499,6 +3503,17 @@ ApplyWorkerMain(Datum main_arg) myslotname = MemoryContextStrdup(

Re: Changing "Hot Standby" to "hot standby"

2022-03-02 Thread Daniel Westermann (DWE)
Hi Kyotaro, >>    >>-    Hot Standby is the term used to describe the ability to connect to >>+    hot standby is the term used to describe the ability to connect to >They look like decapitalizing the first word in a sentsnce. Thanks for having a look. Are you suggesting to change it like this?

Re: Schema variables - new implementation for Postgres 15

2022-03-02 Thread Julien Rouhaud
Hi, On Wed, Mar 02, 2022 at 06:03:06AM +0100, Pavel Stehule wrote: > > I lost commit with this change. I am sending updated patch. Thanks a lot Pavel! I did a more thorough review of the patch. I'm attaching a diff (in .txt extension) for comment improvement suggestions. I may have misunderst

Re: Schema variables - new implementation for Postgres 15

2022-03-02 Thread Julien Rouhaud
On Thu, Mar 03, 2022 at 03:06:52PM +0800, Julien Rouhaud wrote: > Hi, > > On Wed, Mar 02, 2022 at 06:03:06AM +0100, Pavel Stehule wrote: > > > > I lost commit with this change. I am sending updated patch. Also, another thing is the size of the patch. It's probably the minimum to have a consistent

Re: Changing "Hot Standby" to "hot standby"

2022-03-02 Thread Kyotaro Horiguchi
At Thu, 3 Mar 2022 06:55:43 +, "Daniel Westermann (DWE)" wrote in > Hi Kyotaro, > > >>    > >>-    Hot Standby is the term used to describe the ability to connect to > >>+    hot standby is the term used to describe the ability to connect to > > >They look like decapitalizing the first wo

Re: Commitfest 2022-03 Patch Triage Part 1b

2022-03-02 Thread Fabien COELHO
Hello Greg, Peter posted an updated version of Fabiens patch about a month ago (which at this point no longer applies) Attached a v15 which is a rebase, after some minor changes in the source and some new test cases added (good!). fixing a few issues, but also point at old review comments

Re: Changing "Hot Standby" to "hot standby"

2022-03-02 Thread Daniel Westermann (DWE)
>> Thanks for having a look. Are you suggesting to change it like this? >> -    Hot Standby is the term used to describe the ability to connect to >> +    Hot standby is the term used to describe the ability to connect to >Yes.  Isn't it the right form of a sentence? Done like that. Regards Dani

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-03-02 Thread Michael Paquier
On Wed, Mar 02, 2022 at 01:27:40PM -0800, Andres Freund wrote: > I don't think we should commit this without synchronizing the authn between > worker / leader (in a separate commit). Too likely that some function that's > marked parallel ok queries the authn_id, opening up a security/monitoring hol

<    1   2