Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Andres Freund
Hi, On 2022-02-21 11:06:49 +0300, Yura Sokolov wrote: > From 04b07d0627ec65ba3327dc8338d59dbd15c405d8 Mon Sep 17 00:00:00 2001 > From: Yura Sokolov > Date: Mon, 21 Feb 2022 08:49:03 +0300 > Subject: [PATCH v3] [PGPRO-5616] bufmgr: do not acquire two partition locks. > > Acquiring two partition l

Re: Add checkpoint and redo LSN to LogCheckpointEnd log message

2022-02-25 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 16:52:30 +0900 (JST), Kyotaro Horiguchi wrote in > Ugh! Wait for a moment. Something's wrong. Sorry, what is wrong was my working directory. It was broken by my bogus operation. All the files apply corresponding versions correctly. regards. -- Kyotaro Horiguchi NTT Open

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Kyotaro Horiguchi
At Fri, 25 Feb 2022 00:04:55 -0800, Andres Freund wrote in > Why don't you just use LockBufHdr/UnlockBufHdr? FWIW, v2 looked fine to me in regards to this point. regards. -- Kyotaro Horiguchi NTT Open Source Software Center

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Yura Sokolov
Hello, Simon. В Пт, 25/02/2022 в 04:35 +, Simon Riggs пишет: > On Mon, 21 Feb 2022 at 08:06, Yura Sokolov wrote: > > Good day, Kyotaro Horiguchi and hackers. > > > > В Чт, 17/02/2022 в 14:16 +0900, Kyotaro Horiguchi пишет: > > > At Wed, 16 Feb 2022 10:40:56 +0300, Yura Sokolov > > > wrote

Re: PATCH: add "--config-file=" option to pg_rewind

2022-02-25 Thread Gunnar "Nick" Bluth
Am 24.02.22 um 14:46 schrieb Daniel Gustafsson: On 24 Feb 2022, at 14:43, Gunnar Nick Bluth wrote: That looks just as good to me, and it already has tests, so I'll happily step down! Actually, I think this looks like a saner approach. Putting a config setting in two place (postgresql.conf

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Simon Riggs
On Fri, 25 Feb 2022 at 09:24, Yura Sokolov wrote: > > This approach is cleaner than v1, but should also perform better > > because there will be a 1:1 relationship between a buffer and its > > dynahash entry, most of the time. > > Thank you for suggestion. Yes, it is much clearer than my initial

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Yura Sokolov
Hello, Andres В Пт, 25/02/2022 в 00:04 -0800, Andres Freund пишет: > Hi, > > On 2022-02-21 11:06:49 +0300, Yura Sokolov wrote: > > From 04b07d0627ec65ba3327dc8338d59dbd15c405d8 Mon Sep 17 00:00:00 2001 > > From: Yura Sokolov > > Date: Mon, 21 Feb 2022 08:49:03 +0300 > > Subject: [PATCH v3] [PGPR

Re: logical replication empty transactions

2022-02-25 Thread Peter Smith
Hi. Here are my review comments for the v19 patch. == 1. Commit message The current logical replication behavior is to send every transaction to subscriber even though the transaction is empty (because it does not contain changes from the selected publications). SUGGESTION "to subscriber ev

Commitfest manager for 2022-03

2022-02-25 Thread Julien Rouhaud
Hi, The final commitfest for pg15 will start in a few days, and I didn't see any discussion on it or anyone volunteering to be a CFM. I thought it would be a good idea to send this reminder now and avoid the same situation as the last commitfest, to avoid unnecessary pain for the CFM(s). Is ther

Re: PROXY protocol support

2022-02-25 Thread Magnus Hagander
On Tue, Nov 16, 2021 at 12:03 AM Jacob Champion wrote: > > On Thu, 2021-11-04 at 12:03 +0100, Magnus Hagander wrote: > > Thanks for the pointer, PFA a rebase. > > I think the Unix socket handling needs the same "success" fix that you > applied to the TCP socket handling above it: > > > @@ -1328,9

Re: Some optimisations for numeric division

2022-02-25 Thread Dean Rasheed
On Wed, 23 Feb 2022 at 22:55, Tom Lane wrote: > > Dean Rasheed writes: > > > One thought that occurred to me was that it's a bit silly that > > exp_var() and ln_var() have to use a NumericVar for what could just be > > an int, if we had a div_var_int() function that could divide by an > > int. Th

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-02-25 Thread Bharath Rupireddy
On Wed, Feb 16, 2022 at 9:04 AM Ashutosh Sharma wrote: > I don't think that's the use case of this patch. Unless there is some > other valid reason, I would suggest you remove it. Removed the function pg_verify_raw_wal_record. Robert and Greg also voted for removal upthread. > > > Should we add

Re: logical replication empty transactions

2022-02-25 Thread Ajin Cherian
On Fri, Feb 18, 2022 at 9:27 PM Amit Kapila wrote: > > > Yeah, I think there could be multiple ways (a) We can send such a keep > alive in WalSndUpdateProgress() itself by using ctx->write_location. > For this, we need to modify WalSndKeepalive() to take sentPtr as > input. (b) set some flag in Wa

Add LZ4 compression in pg_dump

2022-02-25 Thread Georgios
Hi, please find attached a patchset which adds lz4 compression in pg_dump. The first commit does the heavy lifting required for additional compression methods. It expands testing coverage for the already supported gzip compression. Commit bf9aa490db introduced cfp in compress_io.{c,h} with the i

Re: Typo in pgbench messages.

2022-02-25 Thread Fabien COELHO
Hello Daniel, I think that the break of typographical rules is intentional to allow such simplistic low-level stream handling through pipes or scripts. I'd prefer that the format is not changed. Maybe a comment could be added to explain the reason behind it. That doesn't sound like an over

Re: Some optimisations for numeric division

2022-02-25 Thread Dean Rasheed
On Fri, 25 Feb 2022 at 10:45, Dean Rasheed wrote: > > Attached is the updated patch series (0001 and 0002 unchanged). > And another update following feedback from the cfbot. Regards, Dean From 41732ad9a44dcd12e52d823fb59cb23cce4fe217 Mon Sep 17 00:00:00 2001 From: Dean Rasheed Date: Sun, 20 Feb

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-25 Thread osumi.takami...@fujitsu.com
On Friday, February 25, 2022 12:57 PM Amit Kapila wrote: > On Thu, Feb 24, 2022 at 6:30 PM Masahiko Sawada > wrote: > > > > On Thu, Feb 24, 2022 at 8:08 PM Amit Kapila > wrote: > > > > > > On Thu, Feb 24, 2022 at 1:20 PM Masahiko Sawada > wrote: > > > > > > > > Here are some comments: > > > >

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-25 Thread osumi.takami...@fujitsu.com
On Thursday, February 24, 2022 8:09 PM Amit Kapila > On Thu, Feb 24, 2022 at 1:20 PM Masahiko Sawada > wrote: > > + /* > > +* Log the error that caused DisableSubscriptionOnError to be > called. We > > +* do this immediately so that it won't be lost if some other > > intern

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
> Cool. I think we can report an error instead of reading wal files, > if the tablespace, database, or relation is invalid. Does there any > WAL record that has invalid tablespace, database, or relation OID? The only sort of validity check we could do here is range checking for the underlying d

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-25 Thread osumi.takami...@fujitsu.com
On Thursday, February 24, 2022 4:50 PM Masahiko Sawada wrote: > On Tue, Feb 22, 2022 at 3:03 PM Peter Smith > wrote: > > > > ~~~ > > > > 1. worker.c - comment > > > > + subform = (Form_pg_subscription) GETSTRUCT(tup); > > + > > + /* > > + * We would not be here unless this subscription's disable

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-25 Thread osumi.takami...@fujitsu.com
On Wednesday, February 23, 2022 6:52 PM Tang, Haiying/唐 海英 wrote: > I have a comment on v21 patch. > > I wonder if we really need subscription s2 in 028_disable_on_error.pl. I > think for > subscription s2, we only tested some normal cases(which could be tested with > s1), and didn't test any e

RE: Optionally automatically disable logical replication subscriptions on error

2022-02-25 Thread osumi.takami...@fujitsu.com
On Tuesday, February 22, 2022 3:03 PM Peter Smith wrote: > Here are a couple more review comments for v21. > > ~~~ > > 1. worker.c - comment > > + subform = (Form_pg_subscription) GETSTRUCT(tup); > + > + /* > + * We would not be here unless this subscription's disableonerr field > + was > + * t

Re: should vacuum's first heap pass be read-only?

2022-02-25 Thread Dilip Kumar
On Mon, Feb 7, 2022 at 10:06 PM Robert Haas wrote: > > On Fri, Feb 4, 2022 at 4:12 PM Peter Geoghegan wrote: > > I had imagined that we'd > > want to do heap vacuuming in the same way as today with the dead TID > > conveyor belt stuff -- it just might take several VACUUM operations > > until we a

Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Magnus Hagander
Here's a patch to add the sum of timings for JIT counters to pg_stat_statements, as a way to follow-up on if JIT is doing a good or a bad job in a configuration. I decided to only store the total time for the timings, since there are 4 different timings and storing max/min/etc for each one of them

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread Japin Li
On Fri, 25 Feb 2022 at 20:48, David Christensen wrote: >> Cool. I think we can report an error instead of reading wal files, >> if the tablespace, database, or relation is invalid. Does there any >> WAL record that has invalid tablespace, database, or relation OID? > > The only sort of validi

Re: Add id's to various elements in protocol.sgml

2022-02-25 Thread Peter Eisentraut
On 25.02.22 06:36, Brar Piening wrote: Yes, that would be possible. In fact appending a link and optionally adding a tiny bit of CSS like I show below does the trick. The major problem in that regard would probably be my lack of XSLT/docbook skills but if no one can jump in here, I can see if I

Re: convert libpq uri-regress tests to tap test

2022-02-25 Thread Peter Eisentraut
On 24.02.22 16:17, Tom Lane wrote: I think that having t/ directories contain only Perl test scripts is a good convention that we should stick to. Peter's proposal of a separate test/ subdirectory for C test scaffolding is probably fine. I wonder if there are any conventions in the Perl commun

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Julien Rouhaud
Hi, On Fri, Feb 25, 2022 at 02:06:29PM +0100, Magnus Hagander wrote: > Here's a patch to add the sum of timings for JIT counters to > pg_stat_statements, as a way to follow-up on if JIT is doing a good or > a bad job in a configuration. +1, it seems like something quite useful. > I decided to on

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread Bharath Rupireddy
On Fri, Feb 25, 2022 at 12:36 AM David Christensen wrote: > > Greetings, > > This patch adds the ability to specify a RelFileNode and optional BlockNum to > limit output of > pg_waldump records to only those which match the given criteria. This should > be more performant > than `pg_waldump | g

Re: Readd use of TAP subtests

2022-02-25 Thread Peter Eisentraut
On 24.02.22 16:00, Andres Freund wrote: I've incidentally played with subtests yesterdays, when porting src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it seems that subtests aren't actually specified in the tap format, and that different libraries generate different output form

Re: set TESTDIR from perl rather than Makefile

2022-02-25 Thread Andrew Dunstan
On 2/24/22 20:17, Justin Pryzby wrote: > On Mon, Feb 21, 2022 at 07:00:54AM -0500, Andrew Dunstan wrote: >> On 2/19/22 18:53, Justin Pryzby wrote: >>> On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote: I rebased and fixed the check-guc script to work, made it work with vpath

Re: Readd use of TAP subtests

2022-02-25 Thread Andrew Dunstan
On 2/25/22 08:39, Peter Eisentraut wrote: > On 24.02.22 16:00, Andres Freund wrote: >> I've incidentally played with subtests yesterdays, when porting >> src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it >> seems >> that subtests aren't actually specified in the tap format, and

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-25 Thread Greg Stark
On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote: > > Hi, > > I was looking the shared memory stats patch again. Can you point me to this thread? I looked for it but couldn't find it. -- greg

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
> Thank you Alvaro and Matthias for your views. I understand your point > of not updating the progress-report flag here as it just checks > whether the CHECKPOINT_IMMEDIATE is set or not and takes an action > based on that but it doesn't change the checkpoint flags. I will > modify the code but I a

Size functions inconsistent results

2022-02-25 Thread Fabrízio de Royes Mello
Hi all, While doing some work using our functions [1] for calculate relations size I noticed an inconsistency between pg_total_relation_size and calculate everything separately, have a look in this example: fabrizio=# create table test_size (id bigserial primary key, toast_column text); CREATE TA

Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-25 Thread Bharath Rupireddy
On Thu, Jan 6, 2022 at 1:29 PM SATYANARAYANA NARLAPURAM wrote: > > Consider a cluster formation where we have a Primary(P), Sync Replica(S1), > and multiple async replicas for disaster recovery and read scaling (within > the region and outside the region). In this setup, S1 is the preferred > f

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
> + if ((ckpt_flags & > +(CHECKPOINT_IS_SHUTDOWN | CHECKPOINT_END_OF_RECOVERY)) == 0) > + { > > This code (present at multiple places) looks a little ugly to me, what > we can do instead is add a macro probably named IsShutdownCheckpoint() > which does the above check an

Fix typo in logicalfuncs.c - :%s/private date/Private data

2022-02-25 Thread Bharath Rupireddy
Hi, Here's a tiny patch to do $subject. Regards, Bharath Rupireddy. v1-0001-Fix-typo-in-logicalfuncs.c.patch Description: Binary data

Re: Size functions inconsistent results

2022-02-25 Thread Japin Li
On Fri, 25 Feb 2022 at 22:58, Fabrízio de Royes Mello wrote: > Hi all, > > While doing some work using our functions [1] for calculate relations size > I noticed an inconsistency between pg_total_relation_size and calculate > everything separately, have a look in this example: > > fabrizio=# cr

Add parameter jit_warn_above_fraction

2022-02-25 Thread Magnus Hagander
This patch adds a configuration parameter jit_warn_above_fraction that will cause a warning to be logged if the fraction of time spent on doing JIT is bigger than the specified one. For example, this can be used to track down those cases where JIT ends up taking 90% of the query runtime because of

Re: Size functions inconsistent results

2022-02-25 Thread Fabrízio de Royes Mello
On Fri, Feb 25, 2022 at 12:10 PM Japin Li wrote: > > > I think, you forget the index size of toast table. > > with relations as ( > select schemaname, relname, relid > from pg_stat_user_tables > where relname = 'test_size' > ), > sizes as ( > select > schemaname, > r.relname, > >

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Magnus Hagander
On Fri, Feb 25, 2022 at 2:33 PM Julien Rouhaud wrote: > > Hi, > > On Fri, Feb 25, 2022 at 02:06:29PM +0100, Magnus Hagander wrote: > > Here's a patch to add the sum of timings for JIT counters to > > pg_stat_statements, as a way to follow-up on if JIT is doing a good or > > a bad job in a configur

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Nitin Jadhav
> > Thank you Alvaro and Matthias for your views. I understand your point > > of not updating the progress-report flag here as it just checks > > whether the CHECKPOINT_IMMEDIATE is set or not and takes an action > > based on that but it doesn't change the checkpoint flags. I will > > modify the co

Re: Fix overflow in justify_interval related functions

2022-02-25 Thread Joseph Koshakow
Just checking because I'm not very familiar with the process, are there any outstanding items that I need to do for this patch? - Joe Koshakow

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
On Fri, Feb 25, 2022 at 7:33 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Thanks for the patch. This is not adding something that users can't do > right now, but definitely improves the usability of the pg_waldump as > it avoids external filterings. Also, it can give the

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
On Fri, Feb 25, 2022 at 7:08 AM Japin Li wrote: > > On Fri, 25 Feb 2022 at 20:48, David Christensen < > david.christen...@crunchydata.com> wrote: > >> Cool. I think we can report an error instead of reading wal files, > >> if the tablespace, database, or relation is invalid. Does there any > >>

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Julien Rouhaud
On Fri, Feb 25, 2022 at 04:19:27PM +0100, Magnus Hagander wrote: > > So just to be clear, you're basically thinking: > > jit_count = count of entries where jit_functions>0 > jit_functions = > jit_optimizatinos = count of entries where time spent on jit_optimizations > 0 > > etc? Yes exactly, s

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Magnus Hagander
On Fri, Feb 25, 2022 at 4:40 PM Julien Rouhaud wrote: > > On Fri, Feb 25, 2022 at 04:19:27PM +0100, Magnus Hagander wrote: > > > > So just to be clear, you're basically thinking: > > > > jit_count = count of entries where jit_functions>0 > > jit_functions = > > jit_optimizatinos = count of entrie

Re: Add parameter jit_warn_above_fraction

2022-02-25 Thread Julien Rouhaud
Hi, On Fri, Feb 25, 2022 at 04:16:01PM +0100, Magnus Hagander wrote: > This patch adds a configuration parameter jit_warn_above_fraction that > will cause a warning to be logged if the fraction of time spent on > doing JIT is bigger than the specified one. For example, this can be > used to track

Re: Add parameter jit_warn_above_fraction

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 16:16:01 +0100, Magnus Hagander wrote: > This patch adds a configuration parameter jit_warn_above_fraction that > will cause a warning to be logged if the fraction of time spent on > doing JIT is bigger than the specified one. For example, this can be > used to track down those c

Re: Add parameter jit_warn_above_fraction

2022-02-25 Thread Justin Pryzby
On Fri, Feb 25, 2022 at 04:16:01PM +0100, Magnus Hagander wrote: > + { > + {"jit_warn_above_fraction", PGC_SUSET, LOGGING_WHEN, > + gettext_noop("Sets the fraction of query time spent on > JIT before writing" > + "a w

Re: Readd use of TAP subtests

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 14:39:15 +0100, Peter Eisentraut wrote: > On 24.02.22 16:00, Andres Freund wrote: > > I've incidentally played with subtests yesterdays, when porting > > src/interfaces/libpq/test/regress.pl to a tap test. Unfortunately it seems > > that subtests aren't actually specified in the

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-25 Thread Jacob Champion
On Thu, 2022-02-24 at 20:44 +, Jacob Champion wrote: > That simplifies things. PFA a smaller v2; if pgaudit can't make use of > libpq-be.h for some reason, then I guess we can tailor a fix to that > use case. Ha, opr_sanity caught my use of cstring. I'll work on a fix later today. --Jacob

Re: Add parameter jit_warn_above_fraction

2022-02-25 Thread Magnus Hagander
On Fri, Feb 25, 2022 at 5:20 PM Andres Freund wrote: > > Hi, > > On 2022-02-25 16:16:01 +0100, Magnus Hagander wrote: > > This patch adds a configuration parameter jit_warn_above_fraction that > > will cause a warning to be logged if the fraction of time spent on > > doing JIT is bigger than the s

Re: Readd use of TAP subtests

2022-02-25 Thread Jacob Champion
On Fri, 2022-02-25 at 09:43 -0500, Andrew Dunstan wrote: > AIUI TAP consumers are supposed to ignore lines they don't understand. It's undefined behavior [1]: > Any output that is not a version, a plan, a test line, a YAML block, > a diagnostic or a bail out is incorrect. How a harness handles th

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Julien Rouhaud
On Fri, Feb 25, 2022 at 08:53:50PM +0530, Nitin Jadhav wrote: > > > > I'm not sure what Matthias meant, but as far as I know there's no > > fundamental > > difference between checkpoint with and without the CHECKPOINT_IMMEDIATE > > flag, > > and there's also no scheduling for multiple checkpoints

Re: Readd use of TAP subtests

2022-02-25 Thread Jacob Champion
On Fri, 2022-02-25 at 16:35 +, Jacob Champion wrote: > On Fri, 2022-02-25 at 09:43 -0500, Andrew Dunstan wrote: > > AIUI TAP consumers are supposed to ignore lines they don't understand. > > It's undefined behavior [1]: And of course the minute I send this I notice that I've linked the v13 sp

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Magnus Hagander
On Fri, Feb 25, 2022 at 4:41 PM Magnus Hagander wrote: > > On Fri, Feb 25, 2022 at 4:40 PM Julien Rouhaud wrote: > > > > On Fri, Feb 25, 2022 at 04:19:27PM +0100, Magnus Hagander wrote: > > > > > > So just to be clear, you're basically thinking: > > > > > > jit_count = count of entries where jit_

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Dmitry Dolgov
> On Fri, Feb 25, 2022 at 04:19:27PM +0100, Magnus Hagander wrote: > On Fri, Feb 25, 2022 at 2:33 PM Julien Rouhaud wrote: > > > > Hi, > > > > On Fri, Feb 25, 2022 at 02:06:29PM +0100, Magnus Hagander wrote: > > > Here's a patch to add the sum of timings for JIT counters to > > > pg_stat_statement

Re: Readd use of TAP subtests

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 09:43:20 -0500, Andrew Dunstan wrote: > AIUI TAP consumers are supposed to ignore lines they don't understand. Are they? In http://testanything.org/tap-version-13-specification.html there's: "Lines written to standard output matching /^(not )?ok\b/ must be interpreted as test

Re: Expose JIT counters/timing in pg_stat_statements

2022-02-25 Thread Julien Rouhaud
On Fri, Feb 25, 2022 at 05:38:45PM +0100, Magnus Hagander wrote: > > Per some off-list discussion with Julien, we have clearly been talking > in slightly different terms. So let's summarize the options into what > theÿ́d actually be: > > Option 0: what is int he patch now > > Option 1: > jit_count

Mingw task for Cirrus CI

2022-02-25 Thread Melih Mutlu
Hi All, I've been working on adding Windows+MinGW environment into cirrus-ci tasks (discussion about ci is here [1]). It uses MSYS2 to set the environment. UCRT is chosen as C standard library, instead of MSVCRT. The task configures postgres with features that are available in MSYS2 (see available

Re: Add parameter jit_warn_above_fraction

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 17:28:41 +0100, Magnus Hagander wrote: > On Fri, Feb 25, 2022 at 5:20 PM Andres Freund wrote: > > On 2022-02-25 16:16:01 +0100, Magnus Hagander wrote: > > > This patch adds a configuration parameter jit_warn_above_fraction that > > > will cause a warning to be logged if the frac

Re: BufferAlloc: don't take two simultaneous locks

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 12:51:22 +0300, Yura Sokolov wrote: > > > + * The usage_count starts out at 1 so that the buffer can survive one > > > + * clock-sweep pass. > > > + * > > > + * We use direct atomic OR instead of Lock+Unlock since no other backend > > > + * could be interested in the buffer.

Re: should vacuum's first heap pass be read-only?

2022-02-25 Thread Peter Geoghegan
On Fri, Feb 25, 2022 at 5:06 AM Dilip Kumar wrote: > Based on this discussion, IIUC, we are saying that now we will do the > lazy_scan_heap every time like we are doing now. And we will > conditionally skip the index vacuum for all or some of the indexes and > then based on how much index vacuum

Re: Readd use of TAP subtests

2022-02-25 Thread Andrew Dunstan
On 2/25/22 11:41, Andres Freund wrote: > Hi, > > On 2022-02-25 09:43:20 -0500, Andrew Dunstan wrote: >> AIUI TAP consumers are supposed to ignore lines they don't understand. > Are they? > > In http://testanything.org/tap-version-13-specification.html there's: > > "Lines written to standard outpu

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Jeff Davis
On Thu, 2022-02-24 at 19:47 -0800, Andres Freund wrote: > Why is it restricted to that? You could do sasl negotiation as well > from what > I can see? And that'd theoretically also allow to negotiate whether > the client > supports different ways of doing auth? Not saying that that's easy, > but I

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Jeff Davis
On Thu, 2022-02-24 at 20:47 -0500, Tom Lane wrote: > ... and, since we can't readily enforce that the client only sends > those cleartext passwords over suitably-encrypted connections, this > could easily be a net negative for security. Not sure that I think > it's a good idea. I don't understand

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Tom Lane
Jeff Davis writes: > On Thu, 2022-02-24 at 20:47 -0500, Tom Lane wrote: >> ... and, since we can't readily enforce that the client only sends >> those cleartext passwords over suitably-encrypted connections, this >> could easily be a net negative for security. Not sure that I think >> it's a good

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 09:33:45 -0800, Jeff Davis wrote: > On Thu, 2022-02-24 at 20:47 -0500, Tom Lane wrote: > > ... and, since we can't readily enforce that the client only sends > > those cleartext passwords over suitably-encrypted connections, this > > could easily be a net negative for security.

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Matthias van de Meent
On Fri, 25 Feb 2022 at 17:35, Julien Rouhaud wrote: > > On Fri, Feb 25, 2022 at 08:53:50PM +0530, Nitin Jadhav wrote: > > > > > > I'm not sure what Matthias meant, but as far as I know there's no > > > fundamental > > > difference between checkpoint with and without the CHECKPOINT_IMMEDIATE > >

Re: convert libpq uri-regress tests to tap test

2022-02-25 Thread Andres Freund
Hi, On 2022-02-24 08:46:23 -0800, Andres Freund wrote: > I'm mildly inclined to only do 0001 and 0002 for now. We'd not loose msvc > coverage, because it already doesn't build the test. Once we've ironed that > stuff out, we could do 0003? >From what I can see in the buildfarm client, we'd not lo

Re: [PATCH] Add reloption for views to enable RLS

2022-02-25 Thread Dean Rasheed
On Fri, 18 Feb 2022 at 14:57, Laurenz Albe wrote: > > Here is a new version, with improved documentation and the option renamed > to "check_permissions_owner". I just prefer the shorter form. > Re-reading this thread, I think I preferred the name "security_invoker". The main objection seemed to

Re: Some optimisations for numeric division

2022-02-25 Thread Tom Lane
Dean Rasheed writes: > And another update following feedback from the cfbot. This patchset LGTM. On my machine there doesn't seem to be any measurable performance change for the numeric regression test, but numeric_big gets about 15% faster. The only additional thought I have, based on your com

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Julien Rouhaud
On Fri, Feb 25, 2022 at 06:49:42PM +0100, Matthias van de Meent wrote: > > The point I was trying to make was "If cps->ckpt_flags is > CHECKPOINT_IMMEDIATE, we hurry up to start the new checkpoint that is > actually immediate". That doesn't mean that this checkpoint was > created with IMMEDIATE or

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-25 Thread Julien Rouhaud
On Sat, Feb 26, 2022 at 02:30:36AM +0800, Julien Rouhaud wrote: > On Fri, Feb 25, 2022 at 06:49:42PM +0100, Matthias van de Meent wrote: > > > > The point I was trying to make was "If cps->ckpt_flags is > > CHECKPOINT_IMMEDIATE, we hurry up to start the new checkpoint that is > > actually immediate

Re: Commitfest manager for 2022-03

2022-02-25 Thread Greg Stark
I would like to volunteer. On Fri, 25 Feb 2022 at 05:31, Julien Rouhaud wrote: > > Hi, > > The final commitfest for pg15 will start in a few days, and I didn't see any > discussion on it or anyone volunteering to be a CFM. > > I thought it would be a good idea to send this reminder now and avoid

Re: trigger example for plsample

2022-02-25 Thread Chapman Flack
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested This patch is straightforward, does what it says, and passes the test

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Jonathan S. Katz
On 2/25/22 12:39 PM, Tom Lane wrote: Jeff Davis writes: On Thu, 2022-02-24 at 20:47 -0500, Tom Lane wrote: ... and, since we can't readily enforce that the client only sends those cleartext passwords over suitably-encrypted connections, this could easily be a net negative for security. Not su

Re: C++ Trigger Framework

2022-02-25 Thread Bruce Momjian
On Tue, Feb 22, 2022 at 11:33:05PM +0200, Shmuel Kamensky wrote: > Hi Nathan, > > Thanks for the reply. I did indeed read that document and it's a great place > to > get started. But it doesn't quite answer my questions. Do you personally have > experience writing software in C++ that interacts w

Re: Using operators to do query hints

2022-02-25 Thread Bruce Momjian
On Tue, Feb 22, 2022 at 04:12:15PM -0500, Greg Stark wrote: > I've been playing with an idea I had a while back. Basically that it > would be useful to have some "noop" operators that are used purely to > influence the planner. > > For context I've suggested in the past that there are two categori

Re: Two noncritical bugs of pg_waldump

2022-02-25 Thread Andres Freund
Hi, On 2022-02-14 18:18:47 +0900, Kyotaro Horiguchi wrote: > > If I give an empty file to the tool it complains as the follows. > > > > > pg_waldump: fatal: could not read file "hoge": No such file or directory > > > > No, the file exists. The cause is it reads uninitialized errno to > > detect

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Jeff Davis
On Fri, 2022-02-25 at 12:39 -0500, Tom Lane wrote: > My point is that sending cleartext passwords over the wire is an > insecure-by-definition protocol that we shouldn't be encouraging > more use of. We can require custom auth entries in pg_hba.conf to also specify local, hostssl or hostgssenc. I

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-02-25 Thread David Christensen
Bharath Rupireddy writes: > On Fri, Feb 25, 2022 at 12:36 AM David Christensen > wrote: >> >> Greetings, >> >> This patch adds the ability to specify a RelFileNode and optional BlockNum >> to limit output of >> pg_waldump records to only those which match the given criteria. This >> should be

Re: Fix overflow in justify_interval related functions

2022-02-25 Thread Nathan Bossart
On Fri, Feb 25, 2022 at 10:30:57AM -0500, Joseph Koshakow wrote: > Just checking because I'm not very familiar with the process, > are there any outstanding items that I need to do for this patch? Unless someone has additional feedback, I don't think so. -- Nathan Bossart Amazon Web Services: ht

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Tom Lane
Jeff Davis writes: > On Fri, 2022-02-25 at 12:39 -0500, Tom Lane wrote: >> My point is that sending cleartext passwords over the wire is an >> insecure-by-definition protocol that we shouldn't be encouraging >> more use of. > We can require custom auth entries in pg_hba.conf to also specify > loc

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-25 Thread Euler Taveira
On Fri, Feb 25, 2022, at 11:52 AM, Greg Stark wrote: > On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote: > > > > Hi, > > > > I was looking the shared memory stats patch again. > > Can you point me to this thread? I looked for it but couldn't find it. https://postgr.es/m/20180629.173418.190173462

Re: Race conditions in 019_replslot_limit.pl

2022-02-25 Thread Andres Freund
Hi, On 2022-02-22 18:06:24 -0800, Andres Freund wrote: > On 2022-02-18 15:14:15 -0800, Andres Freund wrote: > > I'm running out of ideas for how to try to reproduce this. I think we might > > need some additional debugging information to get more information from the > > buildfarm. > > > I'm thin

Re: Design of pg_stat_subscription_workers vs pgstats

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 16:25:01 -0300, Euler Taveira wrote: > On Fri, Feb 25, 2022, at 11:52 AM, Greg Stark wrote: > > On Tue, 25 Jan 2022 at 01:32, Andres Freund wrote: > > > > > > Hi, > > > > > > I was looking the shared memory stats patch again. > > > > Can you point me to this thread? I looked fo

Re: Allow async standbys wait for sync replication (was: Disallow quorum uncommitted (with synchronous standbys) txns in logical replication subscribers)

2022-02-25 Thread Nathan Bossart
On Fri, Feb 25, 2022 at 08:31:37PM +0530, Bharath Rupireddy wrote: > Thanks Satya and others for the inputs. Here's the v1 patch that > basically allows async wal senders to wait until the sync standbys > report their flush lsn back to the primary. Please let me know your > thoughts. I haven't had

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Jacob Champion
On Fri, 2022-02-25 at 13:40 -0500, Jonathan S. Katz wrote: > I'm generally in favor of being able to support additional > authentication methods, the first one coming to mind is supporting OIDC. > Having a pluggable auth infrastructure could possibly make such efforts > easier. I'm definitely in

Re: Proposal: Support custom authentication methods using hooks

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 14:10:39 -0500, Tom Lane wrote: > I'm happy to add support for custom auth methods if they can use > a protocol that's safer than cleartext-password. But if that's the > only feasible option, then we're just encouraging people to use > insecure methods. It looks like scram can

Re: Add index scan progress to pg_stat_progress_vacuum

2022-02-25 Thread Nathan Bossart
On Wed, Feb 23, 2022 at 10:41:36AM -0800, Peter Geoghegan wrote: > On Wed, Feb 23, 2022 at 10:02 AM Imseih (AWS), Sami > wrote: >> If the failsafe kicks in midway through a vacuum, the number indexes_total >> will not be reset to 0. If INDEX_CLEANUP is turned off, then the value will >> be 0 at

Re: Commitfest manager for 2022-03

2022-02-25 Thread David Steele
Hi Greg, On 2/25/22 12:39, Greg Stark wrote: I would like to volunteer. On Fri, 25 Feb 2022 at 05:31, Julien Rouhaud wrote: Is there any volunteer? Since I've done the March CF for the last seven years, I thought it would be good if I chimed in. I've been agonizing a bit because I have t

Re: Race conditions in 019_replslot_limit.pl

2022-02-25 Thread Tom Lane
Andres Freund writes: > Seems to suggest something is holding a problematic lock in a way that I do > not understand yet: > https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2022-02-23%2013%3A47%3A20&stg=recovery-check > 2022-02-23 09:09:52.299 EST [2022-02-23 09:09:52 EST 1

Re: Race conditions in 019_replslot_limit.pl

2022-02-25 Thread Andres Freund
Hi, On 2022-02-25 15:07:01 -0500, Tom Lane wrote: > Andres Freund writes: > > Seems to suggest something is holding a problematic lock in a way that I do > > not understand yet: > > > https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2022-02-23%2013%3A47%3A20&stg=recovery-c

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-25 Thread Jacob Champion
On Fri, 2022-02-25 at 16:28 +, Jacob Champion wrote: > Ha, opr_sanity caught my use of cstring. I'll work on a fix later > today. Fixed in v3. --Jacob From 2fde60a6bc3739f1894c8c264120e4fa0f04df64 Mon Sep 17 00:00:00 2001 From: Jacob Champion Date: Mon, 14 Feb 2022 08:10:53 -0800 Subject: [P

Re: Is it correct to update db state in control file as "shutting down" during end-of-recovery checkpoint?

2022-02-25 Thread Nathan Bossart
On Mon, Jan 31, 2022 at 04:25:21PM +0900, Michael Paquier wrote: > At the end of the day, it may be better to just let this stuff be. > Another argument for doing nothing is that this could cause > hard-to-spot conflicts when it comes to back-patch something. This one has been quiet for a while.

Re: [PATCH] pg_permissions

2022-02-25 Thread Chapman Flack
I would be happy to review this patch, but a look through the email leaves me thinking it may still be waiting on a C implementation of pg_get_acl(). Is that right? And perhaps a view rename to pg_privileges, following Peter's comment?

Re: [PATCH] Expose port->authn_id to extensions and triggers

2022-02-25 Thread Andres Freund
On 2022-02-25 20:19:24 +, Jacob Champion wrote: > From 2fde60a6bc3739f1894c8c264120e4fa0f04df64 Mon Sep 17 00:00:00 2001 > From: Jacob Champion > Date: Mon, 14 Feb 2022 08:10:53 -0800 > Subject: [PATCH v3] Add API to retrieve authn_id from SQL > The authn_id field in MyProcPort is currently o

  1   2   >