Re: Sequence's value can be rollback after a crashed recovery.

2021-11-21 Thread Andy Fan
> > The reason is because we never flush the xlog for the nextval_internal > > for the above case. So if > > the system crashes, there is nothing to redo from. It can be fixed > > with the following online change > > code. > > > > @@ -810,6 +810,8 @@ nextval_internal(Oid relid, bool check_permissi

Re: pg_get_publication_tables() output duplicate relid

2021-11-21 Thread Amit Langote
On Fri, Nov 19, 2021 at 2:28 PM Amit Kapila wrote: > On Fri, Nov 19, 2021 at 7:19 AM Amit Langote wrote: > > As in, > > do we know of any replication (initial/streaming) misbehavior caused > > by the duplicate partition OIDs in this case or is the only problem > > that pg_publication_tables outp

Re: dfmgr additional ABI version fields

2021-11-21 Thread Peter Eisentraut
I have committed this patch as posted.

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-21 Thread Laurenz Albe
On Mon, 2021-11-22 at 14:57 +0800, Andy Fan wrote: > Should we guarantee the sequence's nextval should never be rolled back > even in a crashed recovery case? > I can produce the rollback in the following case: > > Session 1: > CREATE SEQUENCE s; > BEGIN; > SELECT nextval('s'); \watch 0.01 > > Se

Re: Keep notnullattrs in RelOptInfo (Was part of UniqueKey patch series)

2021-11-21 Thread Andy Fan
Hi Dmitry: On Wed, Nov 17, 2021 at 11:20 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > > On Wed, Jul 07, 2021 at 01:20:24PM +1200, David Rowley wrote: > > On Wed, 7 Jul 2021 at 13:04, Andy Fan wrote: > > > Looking forward to watching this change closely, thank you both David and > > > Tom!

Re: Sequence's value can be rollback after a crashed recovery.

2021-11-21 Thread David G. Johnston
On Sunday, November 21, 2021, Andy Fan wrote: > > Should we guarantee the sequence's nextval should never be rolled back > even in a crashed recovery case? > I can produce the rollback in the following case: > This seems to be the same observation that was made a little over a year ago. https://

Re: A problem about partitionwise join

2021-11-21 Thread Richard Guo
On Wed, Oct 6, 2021 at 1:19 AM Jaime Casanova wrote: > On Wed, Jul 21, 2021 at 04:44:53PM +0800, Richard Guo wrote: > > On Fri, Nov 27, 2020 at 8:05 PM Ashutosh Bapat < > ashutosh.bapat@gmail.com> > > wrote: > > > > > > > > In the example you gave earlier, the equi join on partition key was >

Sequence's value can be rollback after a crashed recovery.

2021-11-21 Thread Andy Fan
Hi: Should we guarantee the sequence's nextval should never be rolled back even in a crashed recovery case? I can produce the rollback in the following case: Session 1: CREATE SEQUENCE s; BEGIN; SELECT nextval('s'); \watch 0.01 Session 2: kill -9 {sess1.pid} After the restart, the nextval('s')

Re: Optionally automatically disable logical replication subscriptions on error

2021-11-21 Thread vignesh C
On Thu, Nov 18, 2021 at 12:52 PM osumi.takami...@fujitsu.com wrote: > > On Thursday, November 18, 2021 2:08 PM Greg Nancarrow > wrote: > > A minor comment: > Thanks for your comments ! > > > doc/src/sgml/ref/alter_subscription.sgml > > (1) disable_on_err? > > > > + disable_on_err. > > > > T

Re: An obsolete comment of pg_stat_statements

2021-11-21 Thread Kyotaro Horiguchi
At Mon, 22 Nov 2021 15:38:23 +0900 (JST), Kyotaro Horiguchi wrote in > * queryId is supposed to be a valid value, otherwise this function dosen't > * calucate it by its own as before then returns immediately. Mmm. That's bad. This is the correted version. * queryId is supposed to be a valid

An obsolete comment of pg_stat_statements

2021-11-21 Thread Kyotaro Horiguchi
Hello. I noticed obsolete lines in the function comment of pgss_store(). * If queryId is 0 then this is a utility statement for which we couldn't * compute a queryId during parse analysis, and we should compute a suitable * queryId internally. Previously the function actually calculates query

Re: A spot of redundant initialization of brin memtuple

2021-11-21 Thread Richard Guo
On Mon, Nov 22, 2021 at 12:52 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Mon, Nov 22, 2021 at 8:53 AM Richard Guo > wrote: > > > > > > On Sat, Nov 20, 2021 at 12:23 AM Bharath Rupireddy < > bharath.rupireddyforpostg...@gmail.com> wrote: > >> > >> On Fri, Nov 19, 20

Re: xlog.c: removing ReadRecPtr and EndRecPtr

2021-11-21 Thread Amul Sul
On Fri, Nov 19, 2021 at 12:43 AM Robert Haas wrote: > > On Thu, Nov 18, 2021 at 7:30 AM Amul Sul wrote: > > Somehow with and without patch I am getting the same log. > > Try applying the attached 0001-dubious-test-cast.patch for you and see > if that fails. It does for me. If so, then try applyin

Re: Teach pg_receivewal to use lz4 compression

2021-11-21 Thread Jeevan Ladhe
On Fri, Nov 19, 2021 at 7:37 AM Michael Paquier wrote: > On Thu, Nov 18, 2021 at 07:54:37PM +0530, Jeevan Ladhe wrote: > > In dir_open_for_write() I observe that we are writing the header > > and then calling LZ4F_compressEnd() in case there is an error > > while writing the buffer to the file, a

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-11-21 Thread Bharath Rupireddy
On Mon, Nov 22, 2021 at 8:25 AM Fujii Masao wrote: > > On 2021/11/20 1:38, Bharath Rupireddy wrote: > > It reports "remote SQL command: (cancel request)" which isn't a sql > > query, but it looks okay to me as we report (cancel request). The > > pgfdw_get_cleanup_result_v1 patch LGTM. > > BTW, we

Re: issue in pgfdw_report_error()?

2021-11-21 Thread Bharath Rupireddy
On Mon, Nov 22, 2021 at 8:17 AM Fujii Masao wrote: > > Well, in that case, why can't we get rid of "(message_primary != NULL" > > and just have "message_primary[0] != '\0' ? errmsg_internal("%s", > > message_primary) : errmsg("could not obtain message string for remote > > error")" ? > > That's po

Re: [PATCH] Add extra statistics to explain for Nested Loop

2021-11-21 Thread Justin Pryzby
Hi, and sorry to take such a long break from this patch. On Wed, Apr 14, 2021 at 02:27:36PM +0300, e.sokol...@postgrespro.ru wrote: > diff --git a/src/backend/commands/explain.c b/src/backend/commands/explain.c > index b62a76e7e5a..bf8c37baefd 100644 > --- a/src/backend/commands/explain.c > +++ b/

Re: A spot of redundant initialization of brin memtuple

2021-11-21 Thread Bharath Rupireddy
On Mon, Nov 22, 2021 at 8:53 AM Richard Guo wrote: > > > On Sat, Nov 20, 2021 at 12:23 AM Bharath Rupireddy > wrote: >> >> On Fri, Nov 19, 2021 at 1:13 PM Richard Guo wrote: >> > >> > Happened to notice this when reading around the codes. The BrinMemTuple >> > would be initialized in brin_new_m

Re: parallel vacuum comments

2021-11-21 Thread Amit Kapila
On Fri, Nov 19, 2021 at 7:55 AM houzj.f...@fujitsu.com wrote: > > On Tues, Nov 16, 2021 1:53 PM Masahiko Sawada wrote: > > I've incorporated these comments and attached an updated patch. > > > 2) > static void vacuum_error_callback(void *arg); > > I noticed the patch changed the parallel worker'

Re: Failed transaction statistics to measure the logical replication progress

2021-11-21 Thread vignesh C
On Fri, Nov 19, 2021 at 7:41 PM Masahiko Sawada wrote: > > On Thu, Nov 18, 2021 at 12:26 PM Amit Kapila wrote: > > > > On Wed, Nov 17, 2021 at 7:12 PM osumi.takami...@fujitsu.com > > wrote: > > > > > > On Wednesday, November 17, 2021 10:00 PM Amit Kapila > > > wrote: > > > > > > > > Can you pl

Re: A spot of redundant initialization of brin memtuple

2021-11-21 Thread Richard Guo
On Sat, Nov 20, 2021 at 12:23 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Fri, Nov 19, 2021 at 1:13 PM Richard Guo > wrote: > > > > Happened to notice this when reading around the codes. The BrinMemTuple > > would be initialized in brin_new_memtuple(), right after b

Re: Shouldn't postgres_fdw report warning when it gives up getting result from foreign server?

2021-11-21 Thread Fujii Masao
On 2021/11/20 1:38, Bharath Rupireddy wrote: It reports "remote SQL command: (cancel request)" which isn't a sql query, but it looks okay to me as we report (cancel request). The pgfdw_get_cleanup_result_v1 patch LGTM. BTW, we can hide the message "remote SQL command: .." in cancel request c

Re: issue in pgfdw_report_error()?

2021-11-21 Thread Fujii Masao
On 2021/11/20 1:16, Bharath Rupireddy wrote: With the existing code, it emits "" for message_primary[0] == '\0' cases but with the patch it emits "could not obtain message string for remote error". Yes. Well, in that case, why can't we get rid of "(message_primary != NULL" and just have "m

Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-11-21 Thread Peter Geoghegan
Attached WIP patch series significantly simplifies the definition of scanned_pages inside vacuumlazy.c. Apart from making several very tricky things a lot simpler, and moving more complex code outside of the big "blkno" loop inside lazy_scan_heap (building on the Postgres 14 work), this refactoring

Re: row filtering for logical replication

2021-11-21 Thread Greg Nancarrow
On Thu, Nov 18, 2021 at 12:33 PM Peter Smith wrote: > > PSA new set of v40* patches. > Another thing I noticed was in the 0004 patch, list_free_deep() should be used instead of list_free() in the following code block, otherwise the rfnodes themselves (allocated by stringToNode()) are not freed:

Re: wait event and archive_command

2021-11-21 Thread Fujii Masao
On 2021/11/20 0:19, Fujii Masao wrote: On 2021/11/19 16:54, Michael Paquier wrote: On Thu, Nov 18, 2021 at 10:04:57AM +0530, Bharath Rupireddy wrote: Yeah let's not do that. I'm fine with the wait_event_for_archive_command_v2.patch as is. Switched the patch as RfC, then. Thanks! Barrin

Re: using extended statistics to improve join estimates

2021-11-21 Thread Justin Pryzby
Your regression tests include two errors, which appear to be accidental, and fixing the error shows that this case is being estimated poorly. +-- try combining with single-column (and single-expression) statistics +DROP STATISTICS join_test_2; +ERROR: statistics object "join_test_2" does not exis

RE: Improve logging when using Huge Pages

2021-11-21 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Sawada-san, Fujii-san, Thank you for your reviews. In a database with huge_pages = on / off explicitly set, if memory allocation fails, the instance cannot be started, so I think that additional logs are unnecessary. The attached patch outputs the log only when huge_pages = try, and outputs the

Re: Why not try for a HOT update, even when PageIsFull()?

2021-11-21 Thread Peter Geoghegan
On Fri, Nov 19, 2021 at 11:51 AM Alvaro Herrera wrote: > Hmm, I don't have any memory of introducing this; and if you look at the > thread, you'll notice that it got there between the first patch I posted > and the second one, without any mention of the reason. I probably got > that code from the

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Tom Lane
Thomas Munro writes: > On Mon, Nov 22, 2021 at 10:42 AM Tom Lane wrote: >> It'd be useful to check if Lars' patch cures that symptom. > Yeah, it sounds like it might solve at least the server-side problem. > Let's call that weird behaviour #1: RST on process exit. (I wonder if > my keep-the-soc

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Thomas Munro
On Mon, Nov 22, 2021 at 10:42 AM Tom Lane wrote: > Thomas Munro writes: > > Hmm, maybe it's still not enough. Now that I have coffee, I thought > > about the well known failure of idle_in_transaction_timeout to report > > errors on Windows[1]. > > Yeah, I think that may well be a manifestation o

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Tom Lane
Thomas Munro writes: > Hmm, maybe it's still not enough. Now that I have coffee, I thought > about the well known failure of idle_in_transaction_timeout to report > errors on Windows[1]. Yeah, I think that may well be a manifestation of the same problem: once the backend exits, Winsock issues RS

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Tom Lane
Thomas Munro writes: > Hmm. Well, if I understand how this works (and I'm not too familiar > with this Windows code so I maybe I don't), the postmaster duplicates > the socket into the child process (see > {write,read}_inheritable_socket()) and then closes its own handle (see > ServerLoop()'s cal

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Thomas Munro
On Mon, Nov 22, 2021 at 9:24 AM Thomas Munro wrote: > Hmm. Well, if I understand how this works (and I'm not too familiar > with this Windows code so I maybe I don't), the postmaster duplicates > the socket into the child process (see > {write,read}_inheritable_socket()) and then closes its own h

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Thomas Munro
On Mon, Nov 22, 2021 at 8:19 AM Lars Kanis wrote: > The other way around would be to make sure on the client side, that the > last message is retrieved before the RST packet arrives, so that no data > is lost. This works mostly well through the sync API of libpq, but with > the async API the trigg

Re: Windows: Wrong error message at connection termination

2021-11-21 Thread Lars Kanis
Am 18.11.21 um 03:04 schrieb Tom Lane: Thomas Munro writes: I realise now that the experiments we did a while back to try to understand this across a few different operating systems[2] had missed this subtlety, because that Python script had an explicit close() call, whereas PostgreSQL exits.

Re: Improving psql's \password command

2021-11-21 Thread Tom Lane
"Bossart, Nathan" writes: > On 11/20/21, 1:58 PM, "Tom Lane" wrote: >> Meh ... I'm inclined to fix those programs by just moving their pqsignal >> calls down to after their initial GetConnection calls, as attached. >> This'd be simple enough to back-patch, for one thing. > Yeah, I was looking fo

Re: Slow standby snapshot

2021-11-21 Thread Michail Nikolaev
Hello, Alexander. Thanks for your review. > It looks like KnownAssignedXidsNext doesn't have to be > pg_atomic_uint32. I see it only gets read with pg_atomic_read_u32() > and written with pg_atomic_write_u32(). Existing code believes that > read/write of 32-bit values is atomic. So, you can us

Re: Pasword expiration warning

2021-11-21 Thread Gilles Darold
Le 20/11/2021 à 14:48, Andrew Dunstan a écrit : > On 11/19/21 19:17, Bossart, Nathan wrote: >> On 11/19/21, 7:56 AM, "Tom Lane" wrote: >>> That leads me to wonder about server-side solutions. It's easy >>> enough for the server to see that it's used a password with an >>> expiration N days away,