Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 16:47:01 +0900 (JST), Kyotaro Horiguchi wrote in > So, this is the patches for pg12-10. 11 can share the same patch with > 12. 10 has differences in two points. > > 10 has ControlFile->prevCheckPoint. > > The DETAILS of the "recovery restart point at" message is not > cap

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 16:06:31 +0900 (JST), Kyotaro Horiguchi wrote in > At Fri, 25 Feb 2022 15:31:12 +0900 (JST), Kyotaro Horiguchi > wrote in > > While making patches for v12, I see a test failure of pg_rewind for > > uncertain reason. I'm investigating that but I post this for > > discussion

Re: [Proposal] Global temporary tables

2022-02-24 Thread Andres Freund
Hi, This is a huge thread. Realistically reviewers and committers can't reread it. I think there needs to be more of a description of how this works included in the patchset and *why* it works that way. The readme does a bit of that, but not particularly well. On 2022-02-25 14:26:47 +0800, Wenj

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 20:53:08 -0800, Peter Geoghegan wrote: > 0002 makes page-level freezing a first class thing. > heap_prepare_freeze_tuple now has some (limited) knowledge of how this > works. heap_prepare_freeze_tuple's cutoff_xid argument is now always > the VACUUM caller's OldestXmin (not its F

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 15:31:12 +0900 (JST), Kyotaro Horiguchi wrote in > While making patches for v12, I see a test failure of pg_rewind for > uncertain reason. I'm investigating that but I post this for > discussion. Hmm. Too stupid. Somehow I overly removed the latchet condition for minRecover

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-24 Thread Julien Rouhaud
Hi, On Fri, Feb 25, 2022 at 12:23:27AM +0530, Nitin Jadhav wrote: > > I think the change to ImmediateCheckpointRequested() makes no sense. > > Before this patch, that function merely inquires whether there's an > > immediate checkpoint queued. After this patch, it ... changes a > > progress-repor

Re: Typo in pgbench messages.

2022-02-24 Thread Yugo NAGATA
On Thu, 24 Feb 2022 14:44:05 +0100 Daniel Gustafsson wrote: > > On 24 Feb 2022, at 13:58, Fabien COELHO wrote: > > >> One argument against a backpatch is that this would be disruptive with > >> tools that parse and analyze the output generated by pgbench. Fabien, > >> don't you have some tools

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-24 Thread Japin Li
On Fri, 25 Feb 2022 at 03:02, David Christensen wrote: > Greetings, > > This patch adds the ability to specify a RelFileNode and optional BlockNum to > limit output of > pg_waldump records to only those which match the given criteria. This should > be more performant > than `pg_waldump | gre

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-24 Thread Kyotaro Horiguchi
At Tue, 22 Feb 2022 17:44:01 +0900 (JST), Kyotaro Horiguchi wrote in > At Tue, 22 Feb 2022 01:59:45 +0900, Fujii Masao > wrote in > > > > > Of the following, I think we should do (a) and (b) to make future > > > backpatchings easier. > > > a) Use RedoRecPtr and PriorRedoPtr after they are as

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-24 Thread Julien Rouhaud
Hi, On Thu, Feb 24, 2022 at 09:18:26PM -0800, Andres Freund wrote: > > On 2022-02-25 13:01:26 +0800, Julien Rouhaud wrote: > > On Thu, Feb 24, 2022 at 08:44:08PM +, Jacob Champion wrote: > > > > > > Yeah... I was following a similar track with the initial work last > > > year, but I dropped

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Brar Piening
On 24.02.2022 at 17:07, Brar Piening wrote: On 24.02.2022 at 16:46, Alvaro Herrera wrote: Would it be possible to create such anchor links as part of the XSL stylesheets for HTML? I'll investiogate our options and report back. Yes, that would be possible. In fact appending a link and optiona

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Amit Kapila
On Thu, Feb 24, 2022 at 9:20 PM Masahiko Sawada wrote: > > > > > 6. src/backend/postmaster/pgstat.c - function name > > > > +/* -- > > + * pgstat_reset_subscription_counter() - > > + * > > + * Tell the statistics collector to reset a single subscription > > + * counter, or all subscription

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-24 Thread Andres Freund
Hi, On 2022-02-25 13:01:26 +0800, Julien Rouhaud wrote: > On Thu, Feb 24, 2022 at 08:44:08PM +, Jacob Champion wrote: > > > > Yeah... I was following a similar track with the initial work last > > year, but I dropped it when the cost of implementation started to grow > > considerably. At the

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-24 Thread Julien Rouhaud
On Thu, Feb 24, 2022 at 08:44:08PM +, Jacob Champion wrote: > > Yeah... I was following a similar track with the initial work last > year, but I dropped it when the cost of implementation started to grow > considerably. At the time, though, it looked like some overhauls to the > stats framewor

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-02-24 Thread Peter Geoghegan
On Sun, Feb 20, 2022 at 12:27 PM Peter Geoghegan wrote: > You've given me a lot of high quality feedback on all of this, which > I'll work through soon. It's hard to get the balance right here, but > it's made much easier by this kind of feedback. Attached is v9. Lots of changes. Highlights: * M

Re: Separate the result of \watch for each query execution (psql)

2022-02-24 Thread Pavel Stehule
pá 25. 2. 2022 v 5:23 odesílatel Noboru Saito napsal: > Hi, > > Pavel Stehule : > > > I strongly agree. It was a lot of work to find a workable solution for > pspg. Special chars that starting result and maybe other, that ending > result can significantly increase robustness and can reduce code.

Re: BufferAlloc: don't take two simultaneous locks

2022-02-24 Thread Simon Riggs
On Mon, 21 Feb 2022 at 08:06, Yura Sokolov wrote: > > Good day, Kyotaro Horiguchi and hackers. > > В Чт, 17/02/2022 в 14:16 +0900, Kyotaro Horiguchi пишет: > > At Wed, 16 Feb 2022 10:40:56 +0300, Yura Sokolov > > wrote in > > > Hello, all. > > > > > > I thought about patch simplification, and te

Re: [BUG] Panic due to incorrect missingContrecPtr after promotion

2022-02-24 Thread Imseih (AWS), Sami
>Nice catch! However, I'm not sure I like the patch. > * made it through and start writing after the portion that > persisted. > * (It's critical to first write an OVERWRITE_CONTRECORD message, > which > * we'll do as soon as we're open for writing new WA

Re: Separate the result of \watch for each query execution (psql)

2022-02-24 Thread Noboru Saito
Hi, Pavel Stehule : > > I strongly agree. It was a lot of work to find a workable solution for > > pspg. Special chars that starting result and maybe other, that ending > > result can significantly increase robustness and can reduce code. I think > > it can be better to use form feed at the end

Re: Optionally automatically disable logical replication subscriptions on error

2022-02-24 Thread Amit Kapila
On Thu, Feb 24, 2022 at 6:30 PM Masahiko Sawada wrote: > > On Thu, Feb 24, 2022 at 8:08 PM Amit Kapila wrote: > > > > On Thu, Feb 24, 2022 at 1:20 PM Masahiko Sawada > > wrote: > > > > > > Here are some comments: > > > > > > Why do we need SyncTableStartWrapper() and ApplyLoopWrapper()? > > > >

Re: Proposal: Support custom authentication methods using hooks

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 17:02:45 -0800, Jeff Davis wrote: > On Thu, 2022-02-17 at 11:25 -0800, samay sharma wrote: > One caveat is that this only works given information available from > existing authentication methods, because that's all the client > supports. In practice, it seems to only be useful wi

Re: Buffer Manager and Contention

2022-02-24 Thread Simon Riggs
On Fri, 25 Feb 2022 at 01:20, Kyotaro Horiguchi wrote: > Yura Sokolov is proposing a patch to separte the two partition locks. > > https://www.postgresql.org/message-id/1edbb61981fe1d99c3f20e3d56d6c88999f4227c.camel%40postgrespro.ru > > And it seems to me viable for me and a benchmarking in the t

Re: why do hash index builds use smgrextend() for new splitpoint pages

2022-02-24 Thread Amit Kapila
On Fri, Feb 25, 2022 at 8:54 AM Amit Kapila wrote: > > On Fri, Feb 25, 2022 at 4:41 AM Melanie Plageman > wrote: > > > > I'm trying to understand why hash indexes are built primarily in shared > > buffers except when allocating a new splitpoint's worth of bucket pages > > -- which is done with sm

Re: Buffer Manager and Contention

2022-02-24 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 10:20:25 +0900 (JST), Kyotaro Horiguchi wrote in > (I added Yura, as the author of a related patch) > > At Thu, 24 Feb 2022 12:58:23 +, Simon Riggs > wrote in > > Thinking about poor performance in the case where the data fits in > > RAM, but the working set is too bi

Re: pg_stat_get_replication_slot and pg_stat_get_subscription_worker incorrectly marked as proretset

2022-02-24 Thread Amit Kapila
On Thu, Feb 24, 2022 at 12:03 PM Michael Paquier wrote: > > On Wed, Feb 23, 2022 at 09:52:02AM +0530, Amit Kapila wrote: > > Thanks, so you are okay with me pushing that patch just to HEAD. > > Yes, I am fine with that. I am wondering about patching the second > function though, to avoid any risk

Re: why do hash index builds use smgrextend() for new splitpoint pages

2022-02-24 Thread Amit Kapila
On Fri, Feb 25, 2022 at 4:41 AM Melanie Plageman wrote: > > I'm trying to understand why hash indexes are built primarily in shared > buffers except when allocating a new splitpoint's worth of bucket pages > -- which is done with smgrextend() directly in _hash_alloc_buckets(). > > Is this just so

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Maciek Sakrejda
On Thu, Feb 24, 2022, 16:52 Kyotaro Horiguchi wrote: > At Tue, 21 Dec 2021 08:47:27 +0100, Brar Piening wrote in > > On 20.12.2021 at 16:09, Robert Haas wrote: > > > As a data point, this is something I have also wanted to do, from time > > > to time. I am generally of the opinion that any place

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Peter Smith
Below are my review comments for the v3 patch. == 1. Commit message (An earlier review comment [Peter-v2] #2 was only partly fixed) "there are no longer any relation information" --> "there is no longer any relation information" ~~~ 2. doc/src/sgml/monitoring.sgml + pg_stat_subscri

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Andres Freund
Hi, On February 24, 2022 5:44:57 PM PST, Tom Lane wrote: >Tomas Vondra writes: >> I wonder if we could check the index tuple length, and adjust the siglen >> based on that, somehow ... > >In an already-corrupted index, there won't be a unique answer. If we're breaking every ltree gist index, c

Re: Proposal: Support custom authentication methods using hooks

2022-02-24 Thread Tom Lane
Jeff Davis writes: > On Thu, 2022-02-17 at 11:25 -0800, samay sharma wrote: >> To enable this, I've proposed adding a new authentication method >> "custom" which can be specified in pg_hba.conf and takes a mandatory >> argument "provider" specifying which authentication provider to use. > One ca

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tom Lane
Tomas Vondra writes: > I wonder if we could check the index tuple length, and adjust the siglen > based on that, somehow ... In an already-corrupted index, there won't be a unique answer. regards, tom lane

Re: Buffer Manager and Contention

2022-02-24 Thread Kyotaro Horiguchi
(I added Yura, as the author of a related patch) At Thu, 24 Feb 2022 12:58:23 +, Simon Riggs wrote in > Thinking about poor performance in the case where the data fits in > RAM, but the working set is too big for shared_buffers, I notice a > couple of things that seem bad in BufMgr, but don

Re: set TESTDIR from perl rather than Makefile

2022-02-24 Thread Justin Pryzby
On Mon, Feb 21, 2022 at 07:00:54AM -0500, Andrew Dunstan wrote: > On 2/19/22 18:53, Justin Pryzby wrote: > > On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote: > >> I rebased and fixed the check-guc script to work, made it work with vpath > >> builds, and cleaned it up some. > > I also

Re: Proposal: Support custom authentication methods using hooks

2022-02-24 Thread Jeff Davis
On Thu, 2022-02-17 at 11:25 -0800, samay sharma wrote: > To enable this, I've proposed adding a new authentication method > "custom" which can be specified in pg_hba.conf and takes a mandatory > argument "provider" specifying which authentication provider to use. > I've also moved a couple static

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Chapman Flack
On 02/24/22 19:52, Kyotaro Horiguchi wrote: > FWIW in that perspecive, there's no requirement from me that it should > be human-readable. I'm fine with automatically-generated ids. One thing I would be −many on, though, would be automatically-generated ids that are not, somehow, stable. I've been

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Kyotaro Horiguchi
At Tue, 21 Dec 2021 08:47:27 +0100, Brar Piening wrote in > On 20.12.2021 at 16:09, Robert Haas wrote: > > As a data point, this is something I have also wanted to do, from time > > to time. I am generally of the opinion that any place the +1 from me. When I put an URL in the answer for inquiri

Re: Proposal: Support custom authentication methods using hooks

2022-02-24 Thread samay sharma
Hi Aleksander, On Thu, Feb 24, 2022 at 1:17 AM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi Samay, > > > I wanted to submit a patch to expose 2 new hooks (one for the > authentication check and another one for error reporting) in auth.c. These > will allow users to implement their

why do hash index builds use smgrextend() for new splitpoint pages

2022-02-24 Thread Melanie Plageman
I'm trying to understand why hash indexes are built primarily in shared buffers except when allocating a new splitpoint's worth of bucket pages -- which is done with smgrextend() directly in _hash_alloc_buckets(). Is this just so that the value returned by smgrnblocks() includes the new splitpoint

RE: Failed transaction statistics to measure the logical replication progress

2022-02-24 Thread osumi.takami...@fujitsu.com
On Wednesday, February 23, 2022 3:30 PM Amit Kapila > On Tue, Feb 22, 2022 at 6:45 AM tanghy.f...@fujitsu.com > wrote: > > > > I found a problem when using it. When a replication workers exits, the > > transaction stats should be sent to stats collector if they were not > > sent before because it

RE: Failed transaction statistics to measure the logical replication progress

2022-02-24 Thread osumi.takami...@fujitsu.com
On Thursday, February 24, 2022 11:07 AM Masahiko Sawada wrote: > I have some comments on v23 patch: > > @@ -66,6 +66,12 @@ typedef struct LogicalRepWorker > TimestampTz last_recv_time; > XLogRecPtr reply_lsn; > TimestampTz reply_time; > + > + /* > +* Transaction statistics of

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tomas Vondra
On 2/24/22 23:13, Tomas Vondra wrote: > > > On 2/24/22 23:06, Andres Freund wrote: >> Hi, >> >> On 2022-02-24 22:15:20 +0100, Tomas Vondra wrote: >>> After thinking about this I only see two options: >>> >>> 1) Don't apply the patch and tell everyone using ltree_gist they need to >>> rebuild t

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tomas Vondra
On 2/24/22 23:06, Andres Freund wrote: > Hi, > > On 2022-02-24 22:15:20 +0100, Tomas Vondra wrote: >> After thinking about this I only see two options: >> >> 1) Don't apply the patch and tell everyone using ltree_gist they need to >> rebuild the index after pg_upgrade from 12 to 13+. The downsi

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 22:15:20 +0100, Tomas Vondra wrote: > After thinking about this I only see two options: > > 1) Don't apply the patch and tell everyone using ltree_gist they need to > rebuild the index after pg_upgrade from 12 to 13+. The downside of this > is larger indexes (because some tuples

Re: ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tom Lane
Tomas Vondra writes: > After thinking about this I only see two options: > 1) Don't apply the patch and tell everyone using ltree_gist they need to > rebuild the index after pg_upgrade from 12 to 13+. The downside of this > is larger indexes (because some tuples are 20B larger). > 2) Apply the p

Re: remove more archiving overhead

2022-02-24 Thread Nathan Bossart
I tested this again with many more WAL files and a much larger machine (r5d.24xlarge with data directory on an NVMe SSD instance store volume). As before, I am using a bare-bones archive module that does nothing but return true. Without the patch, archiving ~120k WAL files took about 109 seconds.

real/float example for testlibpq3

2022-02-24 Thread Mark Wong
Hi everyone, Would adding additional examples to testlibpq3.c on handling more data types be helpful? I've attached a patch adding how to handle a REAL to current example. Regards, Mark diff --git a/src/test/examples/testlibpq3.c b/src/test/examples/testlibpq3.c index 4f7b791388..0c1739fcbb 1006

ltree_gist indexes broken after pg_upgrade from 12 to 13

2022-02-24 Thread Tomas Vondra
Hi, Victor Yegorov reported a crash related to a GiST index on ltree [1], following a pg_upgrade from 12.x to 14.x, with a data set reproducing this. I spent some time investigating this, and it seems this is a silly bug in commit commit 911e70207703799605f5a0e8aad9f06cff067c63 (HEAD) Author:

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-24 Thread Jacob Champion
On Fri, 2022-02-25 at 01:15 +0800, Julien Rouhaud wrote: > On Thu, Feb 24, 2022 at 04:50:59PM +, Jacob Champion wrote: > > On Thu, 2022-02-24 at 20:39 +0900, Michael Paquier wrote: > > > I don't quite see the additional value that this API brings as > > > MyProcPort is directly accessible, and

Re: Logical insert/update/delete WAL records for custom table AMs

2022-02-24 Thread Simon Riggs
On Mon, 17 Jan 2022 at 09:05, Jeff Davis wrote: > > On Wed, 2022-01-05 at 20:19 +, Simon Riggs wrote: > > I spoke with Jeff in detail about this patch in NYC Dec 21, and I now > > think it is worth pursuing. It seems much more likely that this would > > be acceptable than fully extensible rmgr

Re: reallocing without oom check in pg_regress

2022-02-24 Thread Daniel Gustafsson
> On 23 Feb 2022, at 23:05, Tom Lane wrote: > > Daniel Gustafsson writes: >> In pg_regress we realloc() with the destination and source pointer being >> equal, >> without checking for OOM. While a fairly unlikely source of errors, is >> there a >> reason not to use pg_realloc() there for hygi

Re: document that brin's autosummarize parameter is off by default

2022-02-24 Thread Justin Pryzby
Ten months ago, Jaime Casanova wrote: > Hi everyone, > > Just noted that the default value of autosummarize reloption for brin > indexes is not documented, or at least not well documented. > > I added the default value in create_index.sgml where other options > mention their own defaults, also ma

Re: fix crash with Python 3.11

2022-02-24 Thread Tom Lane
I wrote: > * I'm not satisfied with using static storage for > SaveTransactionCharacteristics/RestoreTransactionCharacteristics. Looking closer at this, I was not too amused to discover that of the three existing SaveTransactionCharacteristics calls, two already conflict with each other: _SPI_comm

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-24 Thread David Christensen
Added to commitfest as: https://commitfest.postgresql.org/37/3565/

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-24 Thread Peter Geoghegan
On Thu, Feb 24, 2022 at 11:06 AM David Christensen wrote: > This patch adds the ability to specify a RelFileNode and optional BlockNum to > limit output of > pg_waldump records to only those which match the given criteria. This should > be more performant > than `pg_waldump | grep` as well as m

[PATCH] add relation and block-level filtering to pg_waldump

2022-02-24 Thread David Christensen
Greetings, This patch adds the ability to specify a RelFileNode and optional BlockNum to limit output of pg_waldump records to only those which match the given criteria. This should be more performant than `pg_waldump | grep` as well as more reliable given specific variations in output style d

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-24 Thread Nitin Jadhav
> I think the change to ImmediateCheckpointRequested() makes no sense. > Before this patch, that function merely inquires whether there's an > immediate checkpoint queued. After this patch, it ... changes a > progress-reporting flag? I think it would make more sense to make the > progress-report

Re: JSON path decimal literal syntax

2022-02-24 Thread Peter Eisentraut
On 18.02.22 11:17, Peter Eisentraut wrote: I noticed that the JSON path lexer does not support the decimal literal syntax forms .1 1. (that is, there are no digits before or after the decimal point).  This is allowed by the relevant ECMAScript standard (https://262.ecma-international.org/5.1

Re: fix crash with Python 3.11

2022-02-24 Thread Tom Lane
I wrote: > * It might be a good idea to add parallel test cases for the other PLs. As I suspected, plperl and pltcl show exactly the same problems when trapping a failure at commit, reinforcing my opinion that this is a SPI bug that needs to be fixed in SPI. (plpgsql is not subject to this proble

Re: remove more archiving overhead

2022-02-24 Thread Nathan Bossart
Thanks for taking a look! On Thu, Feb 24, 2022 at 02:13:49PM +0900, Kyotaro Horiguchi wrote: > https://www.postgresql.org/docs/14/continuous-archiving.html > | The archive command should generally be designed to refuse to > | overwrite any pre-existing archive file. This is an important safety > |

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 17:03:33 +, Jacob Champion wrote: > On Thu, 2022-02-24 at 08:46 -0800, Andres Freund wrote: > > One annoying bit is that our current tap invocation infrastructure for msvc > > won't know how to deal with that. We put the build directory containing t/ > > onto PATH, but that w

Re: libpq async duplicate error results

2022-02-24 Thread Tom Lane
Actually ... it now seems to me that there's live bugs in the interaction between pipeline mode and error-buffer clearing. Namely, that places like PQsendQueryStart will unconditionally clear the buffer even though we may be in the middle of a pipeline sequence, and the buffer might hold an error t

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-24 Thread Julien Rouhaud
Hi, On Thu, Feb 24, 2022 at 04:50:59PM +, Jacob Champion wrote: > On Thu, 2022-02-24 at 20:39 +0900, Michael Paquier wrote: > > I don't quite see the additional value that this API brings as > > MyProcPort is directly accessible, and contrib modules like > > postgres_fdw and sslinfo just use t

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-24 Thread Matthias van de Meent
On Wed, 23 Feb 2022 at 14:28, Nitin Jadhav wrote: > > Sharing the v2 patch. Kindly have a look and share your comments. Thanks for updating. > diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml With the new pg_stat_progress_checkpoint, you should also add a backreference t

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Jacob Champion
On Thu, 2022-02-24 at 08:46 -0800, Andres Freund wrote: > One annoying bit is that our current tap invocation infrastructure for msvc > won't know how to deal with that. We put the build directory containing t/ > onto PATH, but that won't work for test/. But we also don't want to install > test bin

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-24 Thread Jacob Champion
On Thu, 2022-02-24 at 20:39 +0900, Michael Paquier wrote: > I don't quite see the additional value that this API brings as > MyProcPort is directly accessible, and contrib modules like > postgres_fdw and sslinfo just use that to find the data of the current > backend. Right -- I just didn't know i

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 10:17:28 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2022-02-24 13:31:40 +0100, Peter Eisentraut wrote: > >> I think the TAP scripts should be in src/interfaces/libpq/t/, as usual. The > >> supporting code snippets could live in some other directory under > >> src/inte

Re: Time to drop plpython2?

2022-02-24 Thread Mark Wong
On Wed, Feb 23, 2022 at 07:59:01AM -0800, Mark Wong wrote: > On Tue, Feb 22, 2022 at 08:50:07PM -0500, Tom Lane wrote: > > Mark Wong writes: > > > Take 3. :) > > > > > I've upgraded everyone to the v14 buildfarm scripts and made sure the > > > --test passed on HEAD on each one. So I hopefully di

Re: Extract epoch from Interval weird behavior

2022-02-24 Thread Tom Lane
Joseph Koshakow writes: > I do want to briefly mention, if I'm understanding the history of > EXTRACT correctly, that the previous behavior > actually was to multiply by 365.25, not 365. However The commit that > changed the return type from numeric [1] > changed that behavior. Looking through the

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Brar Piening
On 24.02.2022 at 16:46, Alvaro Herrera wrote: On 2022-Feb-24, Dagfinn Ilmari Mannsåker wrote: Peter Eisentraut writes: Is there a way to obtain those URLs other than going into the HTML sources and checking if there is an anchor near where you want go? I use the jump-to-anchor extension: htt

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Masahiko Sawada
Hi, Thank you for the comments! On Thu, Feb 24, 2022 at 4:20 PM tanghy.f...@fujitsu.com wrote: > > On Thu, Feb 24, 2022 9:33 AM Masahiko Sawada wrote: > > > > Thank you for the comments! I've attached the latest version patch > > that incorporated all comments I got so far. The primary change f

Re: create_index test fails when synchronous_commit = off @ master

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 07:33:39 -0800, Andres Freund wrote: > I added the SELECT relpages, reltuples, relallvisible FROM pg_class WHERE oid > = 'tenk1'::regclass; > just after the > VACUUM ANALYZE tenk1; > > synchronous_commit=on > + relpages | reltuples | relallvisible > +--+---+-

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Alvaro Herrera
On 2022-Feb-24, Dagfinn Ilmari Mannsåker wrote: > Peter Eisentraut writes: > > Is there a way to obtain those URLs other than going into the HTML > > sources and checking if there is an anchor near where you want go? > > I use the jump-to-anchor extension: https://github.com/brettz9/jump-to-anc

Re: create_index test fails when synchronous_commit = off @ master

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 16:47:25 +0300, Aleksander Alekseev wrote: > - QUERY PLAN > > - Index Only Scan using tenk1_thous_tenthous on tenk1 > - Index Cond: (thousand < 2) > - Filter: (tenthous = ANY ('{1001,3000}'::integer

Re: Extract epoch from Interval weird behavior

2022-02-24 Thread Joseph Koshakow
On Thu, Feb 24, 2022 at 4:47 AM Aleksander Alekseev wrote: > Extracting an epoch from an interval is quite a strange case since intervals > are not connected to any specific dates. I agree, I think it's a weird use case and that it's probably not worth fixing. Though it was fun for me to try. >

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Tom Lane
Andres Freund writes: > On 2022-02-24 13:31:40 +0100, Peter Eisentraut wrote: >> I think the TAP scripts should be in src/interfaces/libpq/t/, as usual. The >> supporting code snippets could live in some other directory under >> src/interfaces/libpq/, which might be called "test" or something else

Re: Frontend error logging style

2022-02-24 Thread Tom Lane
Peter Eisentraut writes: > This patch already illustrates a couple of things that are wrong with > this approach: All of these objections are a bit moot with my followup proposal, no? > - It doesn't allow any other way of exiting. For example, in pg_dump, > you have removed a few exit_nicely(

Re: Readd use of TAP subtests

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 11:16:03 +0100, Peter Eisentraut wrote: > Now that we have switched everything to done_testing(), the subtests feature > isn't that relevant anymore, but it might still be useful to get better > output when running with PROVE_FLAGS=--verbose. I've incidentally played with subtes

Re: Frontend error logging style

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 14:06:18 +0100, Peter Eisentraut wrote: > My suggestion is to just get rid of pg_log_fatal() and replace them all with > pg_log_error(). -1. This ignores that already several places came up with their slightly different versions of fatal exit handlers. We don't gain anything by

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 13:31:40 +0100, Peter Eisentraut wrote: > I think the TAP scripts should be in src/interfaces/libpq/t/, as usual. The > supporting code snippets could live in some other directory under > src/interfaces/libpq/, which might be called "test" or something else, not > that important.

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Andres Freund
Hi, On 2022-02-24 13:23:55 +0100, Peter Eisentraut wrote: > On 24.02.22 12:46, Masahiko Sawada wrote: > > > We have a view called pg_stat_activity, which is very well known. From > > > that perspective, "activity" means what is happening right now or what > > > has happened most recently. The re

Re: GiST index build missing smgrimmedsync()?

2022-02-24 Thread Heikki Linnakangas
On 23/02/2022 23:30, Melanie Plageman wrote: I brought this up in [1] but wanted to start a dedicated thread. Since 16fa9b2b30a357 GiST indexes are not built in shared buffers. However, smgrimmedsync() is not done anywhere and skipFsync=true is always passed to smgrwrite() and smgrextend(). So,

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > On 18.12.21 00:53, Brar Piening wrote: >> The purpose is that you can directly link to the id in the public html >> docs which still gets generated (e. g. >> https://www.postgresql.org/docs/14/protocol-replication.html#PROTOCOL-REPLICATION-BASE-BACKUP). >> >> Essentia

Re: Patch a potential memory leak in describeOneTableDetails()

2022-02-24 Thread Daniel Gustafsson
> On 24 Feb 2022, at 07:32, Kyotaro Horiguchi wrote: > > At Thu, 24 Feb 2022 14:44:56 +0900 (JST), Kyotaro Horiguchi > wrote in >> At Tue, 22 Feb 2022 09:59:09 +0100, Daniel Gustafsson >> wrote in >>> The section in question was added to our docs 22 years ago, to make it >>> reflect >>> th

create_index test fails when synchronous_commit = off @ master

2022-02-24 Thread Aleksander Alekseev
Hi hackers, I noticed that create_index test (make installcheck) fails on my laptop because different query plans are chosen: - QUERY PLAN - Index Only Scan using tenk1_unique1 on tenk1 - Index Cond: (unique1 = ANY ('

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-24 Thread Daniel Gustafsson
> On 24 Feb 2022, at 14:43, Gunnar Nick Bluth wrote: > That looks just as good to me, and it already has tests, so I'll happily step > down! Actually, I think this looks like a saner approach. Putting a config setting in two place (postgresql.conf and on the commandline for pg_rewind) is a rec

Re: Typo in pgbench messages.

2022-02-24 Thread Daniel Gustafsson
> On 24 Feb 2022, at 13:58, Fabien COELHO wrote: >> One argument against a backpatch is that this would be disruptive with >> tools that parse and analyze the output generated by pgbench. Fabien, >> don't you have some tools and/or wrappers doing exactly that? > > Yep, I like to "| cut -d' ' -f

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-24 Thread Gunnar "Nick" Bluth
Am 24.02.2022 um 14:08 schrieb Daniel Gustafsson: On 24 Feb 2022, at 10:27, Alexander Kukushkin wrote: Hello Gunnar, On Thu, 24 Feb 2022 at 10:03, Aleksander Alekseev wrote: wants to use the "-c" option on a typical Debian/Ubuntu installation (where the config resides below /etc/postgres

Re: Readd use of TAP subtests

2022-02-24 Thread Daniel Gustafsson
> On 24 Feb 2022, at 11:16, Peter Eisentraut > wrote: > I think for deeply nested tests and test routines defined in other modules, > this helps structure the test output more like the test source code is laid > out, so it makes following the tests and locating failing test code easier. I don't

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-24 Thread Daniel Gustafsson
> On 24 Feb 2022, at 10:27, Alexander Kukushkin wrote: > > Hello Gunnar, > > On Thu, 24 Feb 2022 at 10:03, Aleksander Alekseev > wrote: > > > wants to use the "-c" option on a typical Debian/Ubuntu installation > > (where the config resides below /etc/postgresql/), pg_rewind needs a way > >

Re: Frontend error logging style

2022-02-24 Thread Peter Eisentraut
On 23.02.22 00:23, Tom Lane wrote: This conversation seems to have tailed off without full resolution, but I observe that pretty much everyone except Peter is on board with defining pg_log_fatal as pg_log_error + exit(1). So I think we should just do that, unless Peter wants to produce a finishe

Re: Optionally automatically disable logical replication subscriptions on error

2022-02-24 Thread Masahiko Sawada
On Thu, Feb 24, 2022 at 8:08 PM Amit Kapila wrote: > > On Thu, Feb 24, 2022 at 1:20 PM Masahiko Sawada wrote: > > > > Here are some comments: > > > > Why do we need SyncTableStartWrapper() and ApplyLoopWrapper()? > > > > I have given this comment to move the related code to separate > functions t

Re: Typo in pgbench messages.

2022-02-24 Thread Fabien COELHO
Bonjour Michaël, I think it's better to back-patch this to stable branches if there's no objection. Thought? That's only cosmetic, so I would just bother about HEAD if I were to change something like that (I would not bother at all, personally). One argument against a backpatch is that this

Buffer Manager and Contention

2022-02-24 Thread Simon Riggs
Thinking about poor performance in the case where the data fits in RAM, but the working set is too big for shared_buffers, I notice a couple of things that seem bad in BufMgr, but don't understand why they are like that. 1. If we need to allocate a buffer to a new block we do this in one step, whi

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Masahiko Sawada
On Thu, Feb 24, 2022 at 9:23 PM Peter Eisentraut wrote: > > > On 24.02.22 12:46, Masahiko Sawada wrote: > >> We have a view called pg_stat_activity, which is very well known. From > >> that perspective, "activity" means what is happening right now or what > >> has happened most recently. The rew

Re: Proposal: Support custom authentication methods using hooks

2022-02-24 Thread Andrew Dunstan
On 2/24/22 04:16, Aleksander Alekseev wrote: > Hi Samay, > >> I wanted to submit a patch to expose 2 new hooks (one for the authentication >> check and another one for error reporting) in auth.c. These will allow users >> to implement their own authentication methods for Postgres or add custom

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Peter Eisentraut
On 24.02.22 02:52, Tom Lane wrote: Peter Eisentraut writes: On 23.02.22 23:58, Tom Lane wrote: Peter Eisentraut writes: libpq TAP tests should be in src/interfaces/libpq/t/. That's failing to account for the fact that a libpq test can't really be a pure-perl TAP test; you need some C code

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Peter Eisentraut
On 24.02.22 12:46, Masahiko Sawada wrote: We have a view called pg_stat_activity, which is very well known. From that perspective, "activity" means what is happening right now or what has happened most recently. The reworked view in this patch does not contain that (we already have pg_stat_su

Re: convert libpq uri-regress tests to tap test

2022-02-24 Thread Andrew Dunstan
On 2/23/22 20:52, Tom Lane wrote: > Peter Eisentraut writes: >> On 23.02.22 23:58, Tom Lane wrote: >>> Peter Eisentraut writes: libpq TAP tests should be in src/interfaces/libpq/t/. >>> That's failing to account for the fact that a libpq test can't >>> really be a pure-perl TAP test; you n

Re: Add id's to various elements in protocol.sgml

2022-02-24 Thread Peter Eisentraut
On 18.12.21 00:53, Brar Piening wrote: The purpose is that you can directly link to the id in the public html docs which still gets generated (e. g. https://www.postgresql.org/docs/14/protocol-replication.html#PROTOCOL-REPLICATION-BASE-BACKUP). Essentially it gives people discussing the prot

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-24 Thread Masahiko Sawada
On Thu, Feb 24, 2022 at 9:05 PM Amit Kapila wrote: > > On Thu, Feb 24, 2022 at 2:24 PM Peter Smith wrote: > > > > 10. src/backend/replication/logical/worker.c > > > > (from my previous [Peter-v1] #9) > > > > >> + /* Report the worker failed during table synchronization */ > > >> + pgstat_report_s

  1   2   >