Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-08-07 Thread Michael Paquier
On Sat, Aug 06, 2022 at 10:59:26AM -0400, Joe Conway wrote: > I am not sure how else we should interpret SYSTEM_USER -- if it isn't > port->authn_id what else would you propose it should be? What you say sounds rather right, but I was wondering mainly what Oracle and SQL server report when it come

Re: failing to build preproc.c on solaris with sun studio

2022-08-07 Thread John Naylor
On Sun, Aug 7, 2022 at 7:05 AM Tom Lane wrote: > Even on a modern Linux: > > $ size src/backend/parser/gram.o >textdata bss dec hex filename > 656568 0 0 656568 a04b8 src/backend/parser/gram.o > $ size src/interfaces/ecpg/preproc/preproc.o >textdata

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-08-07 Thread Michael Paquier
On Sun, Aug 07, 2022 at 10:41:49AM +0530, Bharath Rupireddy wrote: > Agree. I separated out the changes. + +/* + * A convenience wrapper for pwritev() that retries on partial write. If an + * error is returned, it is unspecified how much has been written. + */ +ssize_t +pg_pwritev_with_retry(int

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-08-07 Thread Thomas Munro
On Sun, Aug 7, 2022 at 7:56 PM Michael Paquier wrote: > FWIW, when it comes to that we have a couple of routines that just use > '0' to mean such a thing, aka palloc0(). I find 0002 confusing, as it > introduces in fe_utils.c a new wrapper > (pg_pwritev_with_retry_and_write_zeros) on what's alrea

Re: logical decoding and replication of sequences

2022-08-07 Thread Tomas Vondra
On 8/7/22 02:36, Noah Misch wrote: > On Thu, Apr 07, 2022 at 08:34:50PM +0200, Tomas Vondra wrote: >> I've pushed a revert af all the commits related to this - decoding of >> sequences and test_decoding / built-in replication changes. > > Two July buildfarm runs failed with PANIC during standby

bug on log generation ?

2022-08-07 Thread Marcos Pegoraro
I have to fix log files because its content is not properly formatted, I´m using version 14.4 but that happened when I was using version 11 too. It happens only when that statement is huge, or because it is a long sequence of updates in a WITH or DO statements, or because i´m updating a bytea field

Re: default result formats setting

2022-08-07 Thread Dave Cramer
On Sun, 7 Aug 2022 at 09:58, Robert Haas wrote: > On Wed, Mar 24, 2021 at 12:01 PM Tom Lane wrote: > > I don't think I buy the premise that there are exactly two levels > > on the client side. > > Thanks for sharing your thoughts on this. I agree it's a complex > issue, and the idea that there a

Re: bug on log generation ?

2022-08-07 Thread Tom Lane
Marcos Pegoraro writes: > I have to fix log files because its content is not properly formatted, What mechanism are you using to store the log? If syslog is involved, it's reputed to drop data under load. regards, tom lane

Re: automatically generating node support functions

2022-08-07 Thread Tom Lane
Amit Kapila writes: > On Wed, Aug 3, 2022 at 7:16 PM Tom Lane wrote: >> More likely, we need to add something explicit to Mkvcbuild.pm >> for this. I recall that it has stanzas to deal with updating >> other autogenerated files; I bet we either missed that or >> fat-fingered it for node-support-

Re: bug on log generation ?

2022-08-07 Thread Marcos Pegoraro
it´s csvlog only Atenciosamente, Em dom., 7 de ago. de 2022 às 11:12, Tom Lane escreveu: > Marcos Pegoraro writes: > > I have to fix log files because its content is not properly formatted, > > What mechanism are you using to store the log? If syslog is involved, > it's reputed to drop dat

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?

2022-08-07 Thread Bharath Rupireddy
On Sun, Aug 7, 2022 at 3:19 PM Thomas Munro wrote: > > > A second thing is that pg_pwritev_with_retry_and_write_zeros() is > > designed to work on WAL segments initialization and it uses > > XLOG_BLCKSZ and PGAlignedXLogBlock for the job, but there is nothing > > in its name that tells us so. Thi

Re: Use fadvise in wal replay

2022-08-07 Thread Andrey Borodin
> On 7 Aug 2022, at 06:39, Bharath Rupireddy > wrote: > > Agree. Why can't we just prefetch the entire WAL file once whenever it > is opened for the first time? Does the OS have any limitations on max > size to prefetch at once? It may sound aggressive, but it avoids > fadvise() system calls,

Re: bug on log generation ?

2022-08-07 Thread Andres Freund
Hi, On 2022-08-07 11:56:44 -0300, Marcos Pegoraro wrote: > it´s csvlog only How are you running postgres? If the logger process runs into trouble it might write to stderr. Is there a chance your huge statements would make you run out of space? Greetings, Andres Freund

Re: Cleaning up historical portability baggage

2022-08-07 Thread Andres Freund
Hi, On 2022-08-06 18:29:14 -0700, Andres Freund wrote: > 0003: aix: aix3.2.5, aix4.1 are not even of historical interest at this point > - 4.1 was released before the first commit in our commit history hoverfly clearly doesn't like this: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=h

Re: logical decoding and replication of sequences

2022-08-07 Thread Noah Misch
On Sun, Aug 07, 2022 at 03:18:52PM +0200, Tomas Vondra wrote: > On 8/7/22 02:36, Noah Misch wrote: > > On Thu, Apr 07, 2022 at 08:34:50PM +0200, Tomas Vondra wrote: > >> I've pushed a revert af all the commits related to this - decoding of > >> sequences and test_decoding / built-in replication cha

Re: logical decoding and replication of sequences

2022-08-07 Thread Thomas Munro
On Mon, Aug 8, 2022 at 7:12 AM Noah Misch wrote: > On Sun, Aug 07, 2022 at 03:18:52PM +0200, Tomas Vondra wrote: > > I'd bet it's about WAL prefetching, not the revert, and the bisect was a > > bit incorrect, because the commits are close and the failures happen to > > be rare. (Presumably you fir

Re: Cleaning up historical portability baggage

2022-08-07 Thread Tom Lane
Thomas Munro writes: > On Thu, Aug 4, 2022 at 4:09 PM Tom Lane wrote: >> clock_gettime is required by SUSv2 (1997), so I have to admit that >> macOS 10.4 doesn't have a lot of excuse not to have it. In any case, >> prairiedog is just sitting there doing its thing until I find cycles >> to instal

old_snapshot: add test for coverage

2022-08-07 Thread Dong Wook Lee
Hi hackers, I wrote a test of the old_snapshot extension for coverage. I hope that this is written correctly. before: 0% after: 100% --- regards, Lee Dong Wook. v1_add_test_oldsnapshot.patch Description: Binary data

Re: [BUG] Logical replication failure "ERROR: could not map filenode "base/13237/442428" to relation OID" with catalog modifying txns

2022-08-07 Thread Amit Kapila
On Wed, Aug 3, 2022 at 1:20 PM Masahiko Sawada wrote: > > On Wed, Aug 3, 2022 at 3:52 PM shiy.f...@fujitsu.com > wrote: > > > > On Wed, Aug 3, 2022 12:06 PM Masahiko Sawada wrote: > > > > > > I've attached updated patches that incorporated the above comments as > > > well as the comments from Sh

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-08-07 Thread Kyotaro Horiguchi
At Fri, 5 Aug 2022 21:28:16 +, "Imseih (AWS), Sami" wrote in > > Would you mind trying the second attached to abtain detailed log on > > your testing environment? With the patch, the modified TAP test yields > > the log lines like below. > > I applied the logging patch to 13.7 ( attached is

Re: Generalize ereport_startup_progress infrastructure

2022-08-07 Thread Bharath Rupireddy
On Thu, Aug 4, 2022 at 9:57 AM Bharath Rupireddy wrote: > > On Wed, Aug 3, 2022 at 12:11 AM Robert Haas wrote: > > > > On Tue, Aug 2, 2022 at 3:25 AM Bharath Rupireddy > > wrote: > > > ereport_startup_progress infrastructure added by commit 9ce346e [1] > > > will be super-useful for reporting pr

Re: Perform streaming logical transactions by background workers and parallel apply

2022-08-07 Thread Dilip Kumar
On Tue, Aug 2, 2022 at 5:16 PM houzj.f...@fujitsu.com wrote: > > On Wednesday, July 27, 2022 4:22 PM houzj.f...@fujitsu.com wrote: > > > > On Tuesday, July 26, 2022 5:34 PM Dilip Kumar > > wrote: > > > > > 3. > > > Why are we restricting parallel apply workers only for the streamed > > > transact

Re: Handle infinite recursion in logical replication setup

2022-08-07 Thread vignesh C
On Fri, Jul 29, 2022 at 10:51 AM vignesh C wrote: > > On Fri, Jul 29, 2022 at 8:31 AM Peter Smith wrote: > > > > Here are some comments for the patch v40-0001: > > > > == > > > > 1. Commit message > > > > It might be better to always use 'copy_data = true' in favour of > > 'copy_data = on' ju

Re: old_snapshot: add test for coverage

2022-08-07 Thread Tom Lane
Dong Wook Lee writes: > I wrote a test of the old_snapshot extension for coverage. Hmm, does this really provide any meaningful coverage? The test sure looks like it's not doing much. I spent some time a week or so ago trying to graft testing of contrib/old_snapshot into src/test/modules/snapsh

Re: Perform streaming logical transactions by background workers and parallel apply

2022-08-07 Thread Dilip Kumar
On Mon, Aug 8, 2022 at 10:18 AM Dilip Kumar wrote: > > > Based on above, we plan to first introduce the patch to perform streaming > > logical transactions by background workers, and then introduce parallel > > apply > > normal transaction which design is different and need some additional > > h

Re: Patch to address creation of PgStat* contexts with null parent context

2022-08-07 Thread Kyotaro Horiguchi
At Sat, 6 Aug 2022 19:19:39 -0700, Andres Freund wrote in > Hi, > > On 2022-08-05 17:22:38 +0900, Kyotaro Horiguchi wrote: > > I think it a bit different. Previously that memory (but for a bit > > different use, precisely) was required only when stats data is read so > > almost all server proce

Re: logical decoding and replication of sequences

2022-08-07 Thread Thomas Munro
On Mon, Aug 8, 2022 at 9:09 AM Thomas Munro wrote: > Thanks for the repro patch and bisection work. Looking... I don't have the complete explanation yet, but it's something like this. We hit the following branch in xlogrecovery.c... if (StandbyMode && !XLogReaderValidat

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-08-07 Thread Bharath Rupireddy
On Thu, Aug 4, 2022 at 11:59 AM Bharath Rupireddy wrote: > > On Sun, Jul 31, 2022 at 8:36 PM mahendrakar s > wrote: > > > >> On Mon, 25 Jul 2022 at 16:42, Bharath Rupireddy > >> wrote: > >> Here's the v3 patch after rebasing. > >> > >> I just would like to reiterate the issue the patch is tryin

Re: Logical replication - schema change not invalidating the relation cache

2022-08-07 Thread vignesh C
On Sat, Mar 12, 2022 at 1:29 PM vignesh C wrote: > > On Fri, Dec 3, 2021 at 3:21 PM vignesh C wrote: > > > > On Fri, Dec 3, 2021 at 1:13 PM Michael Paquier wrote: > > > > > > On Thu, Aug 26, 2021 at 09:00:39PM +0530, vignesh C wrote: > > > > The previous patch was failing because of the recent t

Re: optimize lookups in snapshot [sub]xip arrays

2022-08-07 Thread John Naylor
On Sun, Aug 7, 2022 at 4:25 AM Nathan Bossart wrote: > > [v8] Okay, I think it's basically in good shape. Since it should be a bit faster than a couple versions ago, would you be up for retesting with the original test having 8 to 512 writers? And also add the const markers we discussed upthread?

Re: automatically generating node support functions

2022-08-07 Thread Amit Kapila
On Sun, Aug 7, 2022 at 8:19 PM Tom Lane wrote: > > Amit Kapila writes: > > On Wed, Aug 3, 2022 at 7:16 PM Tom Lane wrote: > >> More likely, we need to add something explicit to Mkvcbuild.pm > >> for this. I recall that it has stanzas to deal with updating > >> other autogenerated files; I bet w