Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-06-20 Thread Michael Paquier
On Fri, May 27, 2022 at 07:01:37PM +, Imseih (AWS), Sami wrote: > What we found: > > 1. missingContrecPtr is set when >StandbyMode is false, therefore >only a writer should set this value >and a record is then sent downstream. > >But a standby going through crash >recove

Re: [PATCH] Completed unaccent dictionary with many missing characters

2022-06-20 Thread Przemysław Sztoch
Michael Paquier wrote on 20.06.2022 03:49: On Wed, Jun 15, 2022 at 01:01:37PM +0200, Przemysław Sztoch wrote: Two fixes (bad comment and fixed Latin-ASCII.xml). if codepoint.general_category.startswith('L') and \ - len(codepoint.combining_ids) > 1: + len(codepoin

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-06-20 Thread Matthias van de Meent
On Mon, 20 Jun 2022 at 07:02, Michael Paquier wrote: > > On Sat, Jun 11, 2022 at 09:25:48PM +0200, Matthias van de Meent wrote: > > Did you initiate a new cluster or otherwise skip the invalid record > > you generated when running the instance based on master? It seems to > > me you're trying to r

Re: Handle infinite recursion in logical replication setup

2022-06-20 Thread Dilip Kumar
On Thu, Jun 16, 2022 at 3:48 PM vignesh C wrote: > > On Wed, Jun 15, 2022 at 12:09 PM Peter Smith wrote: > > > Thanks for the comments, the attached v21 patch has the changes for the same. I have done some basic review of v21 and I have a few comments, 1. +/* + * The subscription will request

Re: Handle infinite recursion in logical replication setup

2022-06-20 Thread vignesh C
On Mon, Jun 20, 2022 at 2:37 PM Dilip Kumar wrote: > > On Thu, Jun 16, 2022 at 3:48 PM vignesh C wrote: > > > > On Wed, Jun 15, 2022 at 12:09 PM Peter Smith wrote: > > > > > > Thanks for the comments, the attached v21 patch has the changes for the > > same. > > I have done some basic review of

Re: Handle infinite recursion in logical replication setup

2022-06-20 Thread Dilip Kumar
On Mon, Jun 20, 2022 at 3:16 PM vignesh C wrote: > > On Mon, Jun 20, 2022 at 2:37 PM Dilip Kumar wrote: > > > > On Thu, Jun 16, 2022 at 3:48 PM vignesh C wrote: > > > > > > On Wed, Jun 15, 2022 at 12:09 PM Peter Smith > > > wrote: > > > > > > > > > Thanks for the comments, the attached v21 pat

Re: pltcl crash on recent macOS

2022-06-20 Thread Peter Eisentraut
On 14.06.22 05:05, Tom Lane wrote: I'd be okay with just dropping the -lc from pl/tcl/Makefile and seeing what the buildfarm says. The fact that we needed it in 1998 doesn't mean that we still need it on supported versions of Tcl; nor was it ever anything but a hack for us to be overriding what

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-06-20 Thread Kyotaro Horiguchi
At Mon, 20 Jun 2022 16:13:43 +0900, Michael Paquier wrote in > On Fri, May 27, 2022 at 07:01:37PM +, Imseih (AWS), Sami wrote: > > What we found: > > > > 1. missingContrecPtr is set when > >StandbyMode is false, therefore > >only a writer should set this value > >and a record i

Re: Handle infinite recursion in logical replication setup

2022-06-20 Thread vignesh C
On Mon, Jun 20, 2022 at 3:16 PM vignesh C wrote: > > On Mon, Jun 20, 2022 at 2:37 PM Dilip Kumar wrote: > > > > On Thu, Jun 16, 2022 at 3:48 PM vignesh C wrote: > > > > > > On Wed, Jun 15, 2022 at 12:09 PM Peter Smith > > > wrote: > > > > > > > > > Thanks for the comments, the attached v21 pat

Re: SGML doc file references

2022-06-20 Thread Peter Eisentraut
On 17.06.22 21:33, Tom Lane wrote: Peter Eisentraut writes: On 17.06.22 19:52, Josh Soref wrote: ok, are they worth fixing? That would require renaming either the output files or the input files, and people would really not like either one. Agreed that renaming those files is not desirabl

Re: Add TAP test for auth_delay extension

2022-06-20 Thread Dong Wook Lee
On 22/06/18 12:07오후, Michael Paquier wrote: > On Sat, Jun 18, 2022 at 11:06:02AM +0900, Dong Wook Lee wrote: > > I have written a test for the auth_delay extension before, > > but if it is okay, can you review it? > > +# check enter wrong password > +my $t0 = [gettimeofday]; > +test_login($node, '

Re: Skipping logical replication transactions on subscriber side

2022-06-20 Thread Peter Eisentraut
On 16.06.22 18:35, Robert Haas wrote: On Thu, Jun 16, 2022 at 3:26 AM Masahiko Sawada wrote: FWIW, looking at the manual, there might have been a solution for AIX to specify -qalign=natural compiler option in order to enforce the alignment of double to 8. Well if that can work it sure seems b

Re: Privileges on PUBLICATION

2022-06-20 Thread Antonin Houska
Euler Taveira wrote: > --eeab359ad6094efd84562cddd7fb9e89 > Content-Type: text/plain > > On Wed, May 18, 2022, at 6:44 AM, Antonin Houska wrote: > > ok, please see the next version. > The new paragraph looks good to me. I'm not sure if the CREATE PUBLICATION is > the right place to provide such

Re: Skipping logical replication transactions on subscriber side

2022-06-20 Thread Robert Haas
On Mon, Jun 20, 2022 at 9:52 AM Peter Eisentraut wrote: > That means changing the system's ABI, so in the extreme case you then > need to compile everything else to match as well. I think we wouldn't want to do that in a minor release, but doing it in a new major release seems fine -- especially

Re: Finer grain log timestamps

2022-06-20 Thread Alvaro Herrera
On 2022-Jun-14, David Fetter wrote: > On Mon, Jun 13, 2022 at 04:22:42PM -0400, Tom Lane wrote: > > A different line of thought is to extend %t to provide a precision > > field a la sprintf, so that for example "%.3t" is equivalent to > > "%m" and "%.6t" does what David wants, and we won't have t

Re: Finer grain log timestamps

2022-06-20 Thread Tom Lane
Alvaro Herrera writes: > Do we *have* to provide support for arbitrary numbers of digits, though? > We could provide support for only %.3t and %.6t specifically, and not > worry about other cases (error: width not supported). If I were coding it, I would allow only exactly 1 digit (%.Nt) to simpl

Make COPY extendable in order to support Parquet and other formats

2022-06-20 Thread Aleksander Alekseev
Hi hackers, In several conversations I had recently with colleagues it was pointed out that it would be great if PostgreSQL supported COPY to/from Parquet and other formats. I've found a corresponding discussion [1] on pgsql-general@. The consensus reached back in 2018 seems to be that this should

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-06-20 Thread Robert Haas
On Mon, Jun 20, 2022 at 7:28 AM Kyotaro Horiguchi wrote: > > Hmm. I have not looked at that in depth, but if the intention is to > > check that the database is able to write WAL, looking at > > XLogCtl->SharedRecoveryState would be the way to go because that's the > > flip switching between crash

Re: SLRUs in the main buffer pool, redux

2022-06-20 Thread Robert Haas
On Thu, Jun 16, 2022 at 1:13 PM Konstantin Knizhnik wrote: > I wonder which workload can cause CLOG to become a bottleneck? > Usually Postgres uses hint bits to avoid clog access. So standard pgbench > doesn't demonstrate any degrade of performance even in case of presence of > long living trans

Re: Finer grain log timestamps

2022-06-20 Thread Isaac Morland
On Mon, 20 Jun 2022 at 11:01, Tom Lane wrote: > Alvaro Herrera writes: > > If I were coding it, I would allow only exactly 1 digit (%.Nt) to simplify > the parsing side of things and bound the required buffer size. Without > having written it, it's not clear to me whether further restricting

CFM for 2022-07

2022-06-20 Thread Jacob Champion
Hi all, If there's not already a manager for the upcoming (July 2022) commitfest, I'd like to volunteer. (And if there is, I'm happy to assist and start learning the ropes, if that would be helpful.) --Jacob

Re: CFM for 2022-07

2022-06-20 Thread Daniel Gustafsson
> On 20 Jun 2022, at 19:29, Jacob Champion wrote: > If there's not already a manager for the upcoming (July 2022) > commitfest, I'd like to volunteer. +1 > (And if there is, I'm happy to assist and start learning the ropes, if that > would be helpful.) Turning it on it's head, I'm happy to as

Re: CFM for 2022-07

2022-06-20 Thread Jacob Champion
On Mon, Jun 20, 2022 at 12:38 PM Daniel Gustafsson wrote: > > On 20 Jun 2022, at 19:29, Jacob Champion wrote: > > (And if there is, I'm happy to assist and start learning the ropes, if that > > would be helpful.) > > Turning it on it's head, I'm happy to assist you. Thanks Daniel, that'd be grea

Re: Assorted small doc patches

2022-06-20 Thread David G. Johnston
On Wed, Jun 1, 2022 at 7:05 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 31.05.22 22:12, David G. Johnston wrote: > > Anything I should be doing differently here to get a bit of > > reviewer/committer time on these? I'll add them to the commitfest for > > next month if nee

Re: Remove trailing newlines from pg_upgrade's messages

2022-06-20 Thread Greg Stark
On Wed, 15 Jun 2022 at 11:54, Tom Lane wrote: > > Yeah, that is sort of the inverse problem. I think those are there > to ensure that the text appears on a fresh line even if the current > line has transient status on it. We could get rid of those perhaps > if we teach pg_log_v to remember wheth

Re: Remove trailing newlines from pg_upgrade's messages

2022-06-20 Thread Tom Lane
Greg Stark writes: > On Wed, 15 Jun 2022 at 11:54, Tom Lane wrote: >> Yeah, that is sort of the inverse problem. I think those are there >> to ensure that the text appears on a fresh line even if the current >> line has transient status on it. We could get rid of those perhaps >> if we teach pg

Re: CFM for 2022-07

2022-06-20 Thread Michael Paquier
On Mon, Jun 20, 2022 at 12:57:22PM -0500, Jacob Champion wrote: > On Mon, Jun 20, 2022 at 12:38 PM Daniel Gustafsson wrote: >> Turning it on it's head, I'm happy to assist you. > > Thanks Daniel, that'd be great! Wow. Thanks, both of you. -- Michael signature.asc Description: PGP signature

Re: [PATCH] Completed unaccent dictionary with many missing characters

2022-06-20 Thread Michael Paquier
On Mon, Jun 20, 2022 at 10:37:57AM +0200, Przemysław Sztoch wrote: > But ligature check is performed on combining_ids (result of translation), > not on base codepoint. > Without it, you will get assertions in get_plain_letters. Hmm. I am wondering if we could make the whole logic a bit more intui

Re: [PATCH] Completed unaccent dictionary with many missing characters

2022-06-20 Thread Thomas Munro
On Tue, Jun 21, 2022 at 12:11 PM Michael Paquier wrote: > Yeah, Latin-ASCII.xml is getting it wrong here, then. unaccent > fetches the thing from this URL currently: > https://raw.githubusercontent.com/unicode-org/cldr/release-41/common/transforms/Latin-ASCII.xml Oh, we're using CLDR 41, which r

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-06-20 Thread Kyotaro Horiguchi
At Mon, 20 Jun 2022 11:57:20 -0400, Robert Haas wrote in > It seems to me that what we want to do is: if we're about to start > allowing WAL writes, then consider whether to insert an aborted > contrecord record. Now, if we are about to start allowing WAL write, > we must determine the LSN at wh

Re: Perform streaming logical transactions by background workers and parallel apply

2022-06-20 Thread Peter Smith
Here are some review comments for the v11-0001 patch. (I will review the remaining patches 0002-0005 and post any comments later) == 1. General I still feel that 'apply' seems like a meaningless enum value for this feature because from a user point-of-view every replicated change gets "appl

Re: Non-replayable WAL records through overflows and >MaxAllocSize lengths

2022-06-20 Thread Michael Paquier
On Mon, Jun 20, 2022 at 11:01:51AM +0200, Matthias van de Meent wrote: > On Mon, 20 Jun 2022 at 07:02, Michael Paquier wrote: >> + if (unlikely(!AllocSizeIsValid(total_len))) >> + XLogErrorDataLimitExceeded(); >> Rather than a single check at the end of XLogRecordAssemble(), you'd >> bette

Re: Replica Identity check of partition table on subscriber

2022-06-20 Thread Amit Langote
On Mon, Jun 20, 2022 at 3:46 PM shiy.f...@fujitsu.com wrote: > On Mon, Jun 20, 2022 1:33 PM Amit Kapila wrote: > > One minor comment: > > + /* > > + * If it is a partitioned table, we don't check it, we will check its > > + * partition later. > > + */ > > > > Can we change the above comment to: "

Re: Replica Identity check of partition table on subscriber

2022-06-20 Thread Amit Kapila
On Tue, Jun 21, 2022 at 7:49 AM Amit Langote wrote: > > On Mon, Jun 20, 2022 at 3:46 PM shiy.f...@fujitsu.com > wrote: > > On Mon, Jun 20, 2022 1:33 PM Amit Kapila wrote: > > > One minor comment: > > > + /* > > > + * If it is a partitioned table, we don't check it, we will check its > > > + * pa

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-06-20 Thread Kyotaro Horiguchi
At Fri, 17 Jun 2022 20:31:50 +0200, Alvaro Herrera wrote in > On 2022-Jun-16, Kyotaro Horiguchi wrote: > > > The attached is a crude patch to separate the state for PIPELINE-IDLE > > from PGASYNC_IDLE. I haven't found a better way.. > > Ah, yeah, this might be a way to fix this. > > Somethin

Re: Perform streaming logical transactions by background workers and parallel apply

2022-06-20 Thread Amit Kapila
On Tue, Jun 21, 2022 at 7:11 AM Peter Smith wrote: > > Here are some review comments for the v11-0001 patch. > > (I will review the remaining patches 0002-0005 and post any comments later) > > == > > 1. General > > I still feel that 'apply' seems like a meaningless enum value for this > featur

Re: Is RecoveryConflictInterrupt() entirely safe in a signal handler?

2022-06-20 Thread Thomas Munro
Here's a new version, but there's something wrong that I haven't figured out yet (see CI link below). Here's one thing I got a bit confused about along the way, but it seems the comment was just wrong: + /* +* If we can abort just the current subtrans

Re: Replica Identity check of partition table on subscriber

2022-06-20 Thread Amit Kapila
On Tue, Jun 21, 2022 at 8:02 AM Amit Kapila wrote: > > On Tue, Jun 21, 2022 at 7:49 AM Amit Langote wrote: > > > > > > I think it should spell out REPLICA IDENTITY explicitly to avoid the > > commit being confused to have to do with "Referential Integrity > > checking". > > > > This makes sense.

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-06-20 Thread Kyotaro Horiguchi
At Tue, 21 Jun 2022 11:42:59 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 17 Jun 2022 20:31:50 +0200, Alvaro Herrera > wrote in > > Others to think about: > > > > PQisBusy (I think no changes are needed), > > Agreed. > > > PQfn (I think it should accept a call in PGASYNC_PIPELINE_IDLE

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-06-20 Thread Kyotaro Horiguchi
At Tue, 21 Jun 2022 14:56:40 +0900 (JST), Kyotaro Horiguchi wrote in > By the way, I noticed that "libpq_pipeline uniqviol" intermittently > fails for uncertain reasons. > > > result 574/575: pipeline aborted > > ... > > done writing > >

Re: Using PQexecQuery in pipeline mode produces unexpected Close messages

2022-06-20 Thread Kyotaro Horiguchi
At Tue, 21 Jun 2022 14:59:07 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 21 Jun 2022 14:56:40 +0900 (JST), Kyotaro Horiguchi > wrote in > > By the way, I noticed that "libpq_pipeline uniqviol" intermittently > > fails for uncertain reasons. > > > > > result 574/575: pipeline aborted >

RE: Replica Identity check of partition table on subscriber

2022-06-20 Thread houzj.f...@fujitsu.com
On Tuesday, June 21, 2022 1:29 PM Amit Kapila : > > On Tue, Jun 21, 2022 at 8:02 AM Amit Kapila wrote: > > > > On Tue, Jun 21, 2022 at 7:49 AM Amit Langote > wrote: > > > > > > > > > I think it should spell out REPLICA IDENTITY explicitly to avoid the > > > commit being confused to have to do wi