Re: Experimental tool to explore commitfest patches

2025-02-25 Thread Jacob Brazeal
I wanted to provide a quick update on the app [0]. Here are the main issues I've seen flagged so far: 1. The ranking system needs improvement. Ideally it should promote *relevant, important, ready-for-review* patches. 2. We should display contributor names as they appear in the commitfest app (thi

Re: per backend WAL statistics

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 03:00:35PM +, Bertrand Drouvot wrote: > That makes fully sense. Done in 0004 attached. Somehow related to that, I've > a patch in progress to address some of Rahila's comments ([1]) (the one > related > to the AuxiliaryPidGetProc() call is relevant specially since a051e

Re: Log connection establishment timings

2025-02-25 Thread Bertrand Drouvot
Hi, On Wed, Feb 26, 2025 at 01:46:19PM +0900, Fujii Masao wrote: > > > On 2025/02/26 6:36, Melanie Plageman wrote: > > On Tue, Feb 25, 2025 at 3:23 PM Melanie Plageman > > wrote: > > > > > > Thanks for doing this! I have implemented your suggestion in attached v3. Thanks for the new patch ver

Re: Doc fix of aggressive vacuum threshold for multixact members storage

2025-02-25 Thread John Naylor
On Wed, Feb 5, 2025 at 2:23 AM Sami Imseih wrote: > > I confirmed the 20GB value as is described here [1]. > 8k page can hold 409 member groups and each > member group can hold 4 members, thus > (2^32/(409 *4))*8192 = 20GB. > > I also fixed whitespace issues in v3. It seems at a minimum this one-

Re: support virtual generated column not null constraint

2025-02-25 Thread ego alter
Hi, I’ve had a chance to review the patch. As I am still getting familiar with executor part, questions and feedbacks could be relatively trivial. There are two minor issues i want to discuss: 1. The way of caching constraint-checking expr for virtual generated not null The existing array for cac

Re: Spinlock can be released twice in procsignal.c

2025-02-25 Thread Maxim Orlov
On Wed, 26 Feb 2025 at 07:56, Michael Paquier wrote: > > This fix is not correct. No system function calls (well basically > most of them) or even more no PostgreSQL-specific calls should happen > while holding a spinlock. elog() is a good example of what not to do. > One example: imagine a pal

Re: [BUG]: the walsender does not update its IO statistics until it exits

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 01:42:08PM +, Bertrand Drouvot wrote: > Now we can see that the numbers increased for the relation object and that we > get non zeros numbers for the wal object too (which makes fully sense). > > With the attached patch applied, we would get the same numbers already in

Enhances pg_createsubscriber documentation for the -d option.

2025-02-25 Thread vignesh C
Hi, Currently, pg_createsubscriber converts streaming replication to logical replication using the specified database name. If the database name is not provided, it is obtained from the --publisher-server option. If the database name is not specified in either the --database or --publisher-server

Re: Postmaster crashed during start

2025-02-25 Thread Srinath Reddy
On Wed, Feb 26, 2025 at 9:50 AM Srinath Reddy wrote: > > > On Wed, Feb 26, 2025 at 9:23 AM Tom Lane wrote: > >> Srinath Reddy writes: >> > when we kill postmaster using kill -9 and start immediately it crashes >> with >> >> FATAL: pre-existing shared memory block (key 2495405, ID 360501) is >>

Re: Fix logging for invalid recovery timeline

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 04:25:40PM +, David Steele wrote: > On 2/25/25 04:07, Benoit Lobréau wrote: >> Thank you Michael and David. >> I never paid attention to thoses... Don't worry about that. There is a lot of Postgres-ism in this code base, and we all keep learning stuff :D > This looks

Re: Improve CRC32C performance on SSE4.2

2025-02-25 Thread John Naylor
On Wed, Feb 26, 2025 at 7:21 AM Devulapalli, Raghuveer wrote: > > > I agree it would be preferable to make a centralized check work. > > Here is my first stab at it. v9 is same as v8 + a commit to move all cpuid > checks into one single place including the AVX512 popcount code. Any new > feature

Re: Statistics Import and Export

2025-02-25 Thread Tom Lane
Corey Huinker writes: > Just to confirm, we ARE able to assume dense packing of attributes in an > index, and thus we can infer the attnum from the position of the attname in > the aggregated array, and there's no need to do a parallel array_agg of > attnums, yes? Yes, absolutely, there are no dr

Re: Statistics Import and Export

2025-02-25 Thread Corey Huinker
On Tue, Feb 25, 2025 at 11:36 PM Tom Lane wrote: > Corey Huinker writes: > > My solution so far is to take allo the v11+ (SELECT array_agg...) > functions > > and put them into a LATERAL, two of them filtered by attstattarget > 0 > and > > a new one aggregating attnames with no filter. > > > An

Re: Spinlock can be released twice in procsignal.c

2025-02-25 Thread Michael Paquier
On Wed, Feb 26, 2025 at 09:08:53AM +0500, Andrey Borodin wrote: > Looks like the oversight in 9d9b9d4. IMO the fix is correct. if (pg_atomic_read_u32(&slot->pss_pid) != 0) { - SpinLockRelease(&slot->pss_mutex); elog(LOG, "process %d taking over ProcSig

Re: Log connection establishment timings

2025-02-25 Thread Fujii Masao
On 2025/02/26 6:36, Melanie Plageman wrote: On Tue, Feb 25, 2025 at 3:23 PM Melanie Plageman wrote: Thanks for doing this! I have implemented your suggestion in attached v3. I missed an include in the EXEC_BACKEND not defined case. attached v4 is fixed up. Thanks for updating the patch!

Re: Statistics Import and Export

2025-02-25 Thread Tom Lane
Corey Huinker writes: > My solution so far is to take allo the v11+ (SELECT array_agg...) functions > and put them into a LATERAL, two of them filtered by attstattarget > 0 and > a new one aggregating attnames with no filter. > An alternative would be a new subselect for array_agg(attname) WHERE

Re: Spinlock can be released twice in procsignal.c

2025-02-25 Thread Andrey Borodin
> On 26 Feb 2025, at 00:34, Maksim.Melnikov wrote: > > In applied patch I removed spinlock release in if clause. Looks like the oversight in 9d9b9d4. IMO the fix is correct. Best regards, Andrey Borodin.

Re: Extend postgres_fdw_get_connections to return remote backend pid

2025-02-25 Thread Sagar Shedge
> However, if PGconn is NULL, it seems like postgres_fdw_get_connections() > wouldn't include that connection in the result. If the connection status > is not CONNECTION_OK, it looks like the connection would be closed by > pgfdw_reset_xact_state() before the local backend processes the query > cal

Re: Statistics Import and Export

2025-02-25 Thread Corey Huinker
> > To my mind the next task is to get the buildfarm green again by > fixing the expression-index-stats problem. I can have a go at > that once Jeff pushes these patches, unless one of you are already > on it? > Already on it, but I can step aside if you've got a clearer vision of how to solve it

Re: Statistics Import and Export

2025-02-25 Thread Jeff Davis
On Tue, 2025-02-25 at 22:40 -0500, Tom Lane wrote: > To my mind the next task is to get the buildfarm green again by > fixing the expression-index-stats problem.  I can have a go at > that once Jeff pushes these patches, unless one of you are already > on it? I just committed them. Regards,

Re: Postmaster crashed during start

2025-02-25 Thread Tom Lane
Srinath Reddy writes: > when we kill postmaster using kill -9 and start immediately it crashes with >> FATAL: pre-existing shared memory block (key 2495405, ID 360501) is still >> in use "Doctor, it hurts when I do this!" "So don't do that!" This is not a supported way of shutting down the pos

Re: long-standing data loss bug in initial sync of logical replication

2025-02-25 Thread Shlok Kyal
On Wed, 11 Dec 2024 at 12:37, Masahiko Sawada wrote: > > On Tue, Oct 8, 2024 at 2:51 AM Shlok Kyal wrote: > > > > On Wed, 31 Jul 2024 at 03:27, Masahiko Sawada wrote: > > > > > > On Wed, Jul 24, 2024 at 9:53 PM Amit Kapila > > > wrote: > > > > > > > > On Wed, Jul 17, 2024 at 5:25 PM vignesh C

Re: Statistics Import and Export

2025-02-25 Thread Tom Lane
Corey Huinker writes: > We can still convert the "EXECUTE getAttributeStats" call to a Params call, > but that involves creating an ExecuteSqlQueryParams(), which starts to > snowball in the changes required. Yeah, let's leave that for some other day. It's not really apparent that it'd buy us mu

Re: Restrict copying of invalidated replication slots

2025-02-25 Thread Amit Kapila
On Tue, Feb 25, 2025 at 11:21 PM Masahiko Sawada wrote: > > On Tue, Feb 25, 2025 at 2:36 AM Amit Kapila wrote: > > > > > > > > Scenario-3: the source gets invalidated after creating the copied slot > > > (i.e. after create_logical/physical_replication_slot()). In this case, > > > since the newly

Re: bug in stored generated column over domain with constraints.

2025-02-25 Thread jian he
hi. comments refined and minor aesthetic adjustments made. From 22ef7ce384de3098fc19ae0bb9bc9777b269b8ec Mon Sep 17 00:00:00 2001 From: jian he Date: Wed, 26 Feb 2025 11:29:04 +0800 Subject: [PATCH v2 1/1] fix default insertion for stored generated column with domain over constraints. create do

Postmaster crashed during start

2025-02-25 Thread Srinath Reddy
Hi, when we kill postmaster using kill -9 and start immediately it crashes with > FATAL: pre-existing shared memory block (key 2495405, ID 360501) is still > in use HINT: Terminate any old server processes associated with data directory *We can reproduce this* > kill -9 $(head -n 1 $PGDATA/po

Re: Statistics Import and Export

2025-02-25 Thread Corey Huinker
On Tue, Feb 25, 2025 at 9:00 PM Jeff Davis wrote: > On Mon, 2025-02-24 at 09:54 -0500, Andres Freund wrote: > > Have you compared performance of with/without stats after these > > optimizations? > > On unoptimized build with asserts enabled, dumping the regression > database: > > --no-statistic

Re: Statistics Import and Export

2025-02-25 Thread Jeff Davis
On Mon, 2025-02-24 at 09:54 -0500, Andres Freund wrote: > Have you compared performance of with/without stats after these > optimizations? On unoptimized build with asserts enabled, dumping the regression database: --no-statistics: 1.0s master: 3.6s v3j-0001:3.0s v3j-0002

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-02-25 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 25 Feb 2025 17:14:43 -0800, Masahiko Sawada wrote: > I've attached updated patches. Thanks. I found one more missing last ".": 0002: > --- a/src/backend/commands/copyfrom.c > +++ b/src/backend/comm

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-02-25 Thread Masahiko Sawada
On Tue, Feb 25, 2025 at 3:52 PM Sutou Kouhei wrote: > > Thank you for reviewing the patches. I've addressed comments except for the following comment: > > --- a/src/backend/commands/copyfromparse.c > > +++ b/src/backend/commands/copyfromparse.c > > > @@ -1087,7 +1132,7 @@ NextCopyFrom(CopyFromSt

Re: Get rid of WALBufMappingLock

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 05:19:29PM +0200, Alexander Korotkov wrote: > It seems that I managed to reproduce the issue on my Raspberry PI 4. > After running our test suite in a loop for 2 days I found one timeout. Hmm. It's surprising to not see a higher occurence. My buildfarm host has caught tha

Re: Remove wal_[sync|write][_time] from pg_stat_wal and track_wal_io_timing

2025-02-25 Thread Michael Paquier
On Tue, Feb 25, 2025 at 08:12:53AM +, Bertrand Drouvot wrote: > The idea was to not let track_io_timing alone enable the timing in the WAL > code path. My reasoning was: if you want to see timing in pg_stat_io then you > need to enable track_io_timing. But that's not enough if you also want to

Re: Parallel heap vacuum

2025-02-25 Thread Masahiko Sawada
On Tue, Feb 25, 2025 at 2:44 PM Melanie Plageman wrote: > > On Tue, Feb 25, 2025 at 5:14 PM Masahiko Sawada wrote: > > > > Given that we have only about one month until the feature freeze, I > > find that it's realistic to introduce either one parallelism for PG18 > > and at least we might want t

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-02-25 Thread Masahiko Sawada
On Thu, Feb 20, 2025 at 6:48 PM Sutou Kouhei wrote: > > Hi, > > In > "Re: Make COPY format extendable: Extract COPY TO format implementations" > on Thu, 20 Feb 2025 15:28:26 -0800, > Masahiko Sawada wrote: > > > Looking at the 0001 patch again, I have a question: we have > > CopyToTextLikeO

Re: psql \dh: List High-Level (Root) Tables and Indexes

2025-02-25 Thread Sadeq Dousti
Dear Greg, Thank you so much for the kind and prompt review! Please find the patches attached. The second patch (0002) is where I applied the requested changes. Best regards, Sadeq Dousti On Wed, Feb 26, 2025 at 1:01 AM Greg Sabino Mullane wrote: > The patch applies cleanly, and works as adv

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-25 Thread Masahiko Sawada
On Tue, Feb 25, 2025 at 3:03 PM Dagfinn Ilmari Mannsåker wrote: > > Hi, > > While working on another round of the long option and fat comma style > cleanup, I noticed that the test for pg_upgrade --set-char-signedess > doesn't test what it's supposed to: > > Masahiko Sawada writes: > > > diff --g

RE: Improve CRC32C performance on SSE4.2

2025-02-25 Thread Devulapalli, Raghuveer
> I agree it would be preferable to make a centralized check work. Here is my first stab at it. v9 is same as v8 + a commit to move all cpuid checks into one single place including the AVX512 popcount code. Any new feature that requires CPUID information can access that information with pg_cpu_

Re: pgbench client-side performance issue on large scripts

2025-02-25 Thread Tom Lane
I wrote: > I got nerd-sniped by this question and spent some time looking into > it. On second look, I'd failed to absorb your point about how the main loop of ParseScript doesn't need the line number at all; only if it's a backslash command are we going to use that. So we can move the calculatio

Re: Statistics Import and Export

2025-02-25 Thread Jeff Davis
On Tue, 2025-02-25 at 15:31 -0500, Corey Huinker wrote: > Documentation: > > +         The currently-supported relation statistics are > +         relpages with a value of type > +         integer, reltuples with a > value of > +         type real, and > relallvisible with a > +         value of t

Re: psql \dh: List High-Level (Root) Tables and Indexes

2025-02-25 Thread Greg Sabino Mullane
The patch applies cleanly, and works as advertised. Nice work! Quick notes: * doc/src/sgml/ref/psql-ref.sgml In the varlistentry section, the order should be the same as the other places (N after m) Line 1644 has an extra comma Line 1651, maybe the example is simpler as \dNt to keep the wordin

Re: Make COPY format extendable: Extract COPY TO format implementations

2025-02-25 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Tue, 25 Feb 2025 14:05:28 -0800, Masahiko Sawada wrote: > The first two patches are refactoring patches (+ small performance > improvements). I've reviewed these patches again and attached the > updated pat

Re: psql \dh: List High-Level (Root) Tables and Indexes

2025-02-25 Thread Sadeq Dousti
Thanks Greg and others for the feedback! Please find attached the patch for implementing \dN (including \dNt, \dNi, \dNit). Best Regards, Sadeq Dousti From d8fde4b05eee95089548384c07b59304f2fecc1c Mon Sep 17 00:00:00 2001 From: Sadeq Dousti <3616518+msdou...@users.noreply.github.com> Date: Tue, 2

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-25 Thread Dagfinn Ilmari Mannsåker
Hi, While working on another round of the long option and fat comma style cleanup, I noticed that the test for pg_upgrade --set-char-signedess doesn't test what it's supposed to: Masahiko Sawada writes: > diff --git a/src/bin/pg_upgrade/t/005_char_signedness.pl > b/src/bin/pg_upgrade/t/005_cha

Re: Parallel heap vacuum

2025-02-25 Thread Melanie Plageman
On Tue, Feb 25, 2025 at 5:14 PM Masahiko Sawada wrote: > > Given that we have only about one month until the feature freeze, I > find that it's realistic to introduce either one parallelism for PG18 > and at least we might want to implement the one first that is more > beneficial and helpful for u

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Nathan Bossart
On Tue, Feb 25, 2025 at 05:19:30PM -0500, Melanie Plageman wrote: > Yes, so one thing you haven't said yet is if you are +1 on going > forward with these patches in general. Sorry, yes, I'm +1 in general. It conceptually makes sense to me that we should disregard frozen pages when deciding whethe

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Melanie Plageman
On Tue, Feb 25, 2025 at 1:52 PM Robert Haas wrote: > > On Tue, Feb 25, 2025 at 11:03 AM Melanie Plageman > wrote: > > This does however leave me with the question of how to handle the > > original question of whether or not to cap the proposed relallfrozen > > to the value of relallvisible when u

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Melanie Plageman
On Tue, Feb 25, 2025 at 3:05 PM Nathan Bossart wrote: > > On Tue, Feb 25, 2025 at 01:52:28PM -0500, Robert Haas wrote: > > Given that users could manually update the catalog, we have to be able > > to tolerate bad data in the catalogs without the world ending. If that > > code has to exist anyway,

Re: Parallel heap vacuum

2025-02-25 Thread Masahiko Sawada
On Tue, Feb 25, 2025 at 9:59 AM Melanie Plageman wrote: > > On Mon, Feb 24, 2025 at 8:15 PM Masahiko Sawada wrote: > > > > What I can see from these results was that we might not benefit much > > from parallelizing phase III, unfortunately. Although in the best case > > the phase III got about 2x

Re: pgbench client-side performance issue on large scripts

2025-02-25 Thread Tom Lane
"Daniel Verite" writes: > For the moment I'll stay with my quick fix, then l'll try > to come up with something to replace expr_scanner_get_lineno() . I got nerd-sniped by this question and spent some time looking into it. ParseScript has got worse problems than just being slow: it's actively bu

Re: Log connection establishment timings

2025-02-25 Thread Melanie Plageman
On Tue, Feb 25, 2025 at 3:23 PM Melanie Plageman wrote: > > Thanks for doing this! I have implemented your suggestion in attached v3. I missed an include in the EXEC_BACKEND not defined case. attached v4 is fixed up. - Melanie From c493f0be7243bfe965d3493f07982ea1072d89b7 Mon Sep 17 00:00:00 200

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Melanie Plageman
On Tue, Feb 25, 2025 at 11:36 AM Greg Sabino Mullane wrote: > > On Tue, Feb 25, 2025 at 11:03 AM Melanie Plageman > wrote: >> >> Because users can now manually update these values in pg_class, there >> wouldn't be a way to detect the difference >> between a bogus relallfrozen value due to VM co

Spinlock can be released twice in procsignal.c

2025-02-25 Thread Maksim.Melnikov
Hi, it seems we can release spinlock twice in /src/backend/storage/ipc/procsignal.c file, method ProcSignalInit. void ProcSignalInit(bool cancel_key_valid, int32 cancel_key) {     ProcSignalSlot *slot;     uint64    barrier_generation; ...

Re: Psql meta-command conninfo+

2025-02-25 Thread Alvaro Herrera
On 2025-Feb-22, Alvaro Herrera wrote: > Also, there's a bunch of "(char *)" casts that are 100% due to > printTableAddCell() taking a char * instead of const char * for the cell > value. That seems a bit silly, we should change that. Ah, but the problem is that most of the input cells there come

Relaxing constraints on BitmapAnd eligibility?

2025-02-25 Thread Dmytro Astapov
Hi! I've been investigating why postgres does not do BitmapAnd of two well-suited indexes, and reading indxpath.c In my case, there is a table (d date, col1 int, col2 int) -- types not really important -- and there are two indices on (d,col1) and (d, col2). For queries that do WHERE d>=X AND col

Re: Securing PostgreSQL for rootless containers

2025-02-25 Thread Yogesh Sharma
On 2/24/25 14:51, Yogesh Sharma wrote: This patch has effect on current use of socket unless systemd socket are used. Code is also guarded when postgres is not compiled with systemd flag. I meant to say This patch has **no** effect on current use of socket unless systemd socket are used.

Re: Statistics Import and Export

2025-02-25 Thread Corey Huinker
On Tue, Feb 25, 2025 at 1:22 PM Jeff Davis wrote: > On Mon, 2025-02-24 at 12:50 -0500, Tom Lane wrote: > > Also, while working on the attached, I couldn't help forming the > > opinion that we'd be better off to nuke pg_set_attribute_stats() > > from orbit and require people to use pg_restore_attr

Re: RFC: Additional Directory for Extensions

2025-02-25 Thread Matheus Alcantara
Thanks for reviewing! On Tue, Feb 25, 2025 at 9:45 AM Andrew Dunstan wrote: > I think your additions generally look good. We should be able to > simplify this: > > > +system_dir = psprintf("%s/extension", sharepath); > +ecp = system_dir; > + > +if (strlen(Extension_control_path) == 0)

Re: Log connection establishment timings

2025-02-25 Thread Melanie Plageman
Thanks for taking a look! On Mon, Jan 20, 2025 at 12:53 PM Jacob Champion wrote: > > On Mon, Jan 20, 2025 at 7:01 AM Bertrand Drouvot > wrote: > > Though time changes are "rare", given the fact that those metrics could > > provide > > "inaccurate" measurements during that particular moment (tim

Re: Log connection establishment timings

2025-02-25 Thread Melanie Plageman
Thanks for taking a look! On Mon, Jan 20, 2025 at 10:01 AM Bertrand Drouvot wrote: > > The patch needed a rebase due to 34486b6092e. I did it in v2 attached (it's > a minor rebase due to the AmRegularBackendProcess() introduction in > miscadmin.h). > > v2 could rely on AmRegularBackendProcess()

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Nathan Bossart
On Tue, Feb 25, 2025 at 01:52:28PM -0500, Robert Haas wrote: > Given that users could manually update the catalog, we have to be able > to tolerate bad data in the catalogs without the world ending. If that > code has to exist anyway, then it's not mandatory to cap. On the other > hand, there's no

Proposal: Limitations of palloc inside checkpointer

2025-02-25 Thread Ekaterina Sokolova
Hi, hackers! Historically, the checkpointer process use palloc() into AbsorbSyncRequests() function. Therefore, the checkpointer does not expect to receive a request larger than 1 GB. We encountered a case where the database went into recovery state, after applying all wal, the checkpointer

Re: Restrict copying of invalidated replication slots

2025-02-25 Thread Masahiko Sawada
On Mon, Feb 24, 2025 at 10:09 PM Shlok Kyal wrote: > > On Tue, 25 Feb 2025 at 01:03, Masahiko Sawada wrote: > > > > I've checked if this issue exists also on v15 or older, but IIUC it > > doesn't exist, fortunately. Here is the summary: > > > > Scenario-1: the source gets invalidated before the c

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Robert Haas
On Tue, Feb 25, 2025 at 11:03 AM Melanie Plageman wrote: > This does however leave me with the question of how to handle the > original question of whether or not to cap the proposed relallfrozen > to the value of relallvisible when updating stats at the end of > vacuum. The current code in heap_v

Re: Statistics Import and Export

2025-02-25 Thread Jeff Davis
On Mon, 2025-02-24 at 12:50 -0500, Tom Lane wrote: > Also, while working on the attached, I couldn't help forming the > opinion that we'd be better off to nuke pg_set_attribute_stats() > from orbit and require people to use pg_restore_attribute_stats(). Attached a patch to do so. The docs and test

Re: Log connection establishment timings

2025-02-25 Thread Melanie Plageman
On Mon, Dec 16, 2024 at 6:26 PM Jelte Fennema-Nio wrote: > > Two thoughts: > 1. Would it make sense to also expose these timings in some pg_stat_xyz view? So, right now there isn't an obvious place this would fit. There are dynamic statistics views that are per connection -- like pg_stat_ssl/gssa

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Nathan Bossart
On Tue, Feb 25, 2025 at 12:36:40PM -0500, Robert Treat wrote: > On Tue, Feb 25, 2025 at 11:32 AM Nathan Bossart > wrote: >> I'm currently leaning towards the "remove all caps" idea. But I'm not sure >> I totally understand the need for a WARNING. What do we expect users to do >> with that inform

Re: Adjusting hash join memory limit to handle batch explosion

2025-02-25 Thread James Hunter
On Tue, Feb 25, 2025 at 9:39 AM Tomas Vondra wrote: > > On 2/25/25 17:30, James Hunter wrote: > > On Wed, Feb 19, 2025 at 12:22 PM Tomas Vondra wrote: > > -- OK, but the customer *didn't* set their workmem to 32 MB. (If they > > had, we wouldn't need this patch -- but we *do* need this patch, wh

Re: Restrict copying of invalidated replication slots

2025-02-25 Thread Masahiko Sawada
On Tue, Feb 25, 2025 at 2:36 AM Amit Kapila wrote: > > On Tue, Feb 25, 2025 at 1:03 AM Masahiko Sawada wrote: > > > > I've checked if this issue exists also on v15 or older, but IIUC it > > doesn't exist, fortunately. Here is the summary: > > > > Scenario-1: the source gets invalidated before the

Re: Parallel heap vacuum

2025-02-25 Thread Melanie Plageman
On Mon, Feb 24, 2025 at 8:15 PM Masahiko Sawada wrote: > > What I can see from these results was that we might not benefit much > from parallelizing phase III, unfortunately. Although in the best case > the phase III got about 2x speedup, as for the total duration it's > about only 10% speedup. My

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Robert Treat
On Tue, Feb 25, 2025 at 11:32 AM Nathan Bossart wrote: > On Tue, Feb 25, 2025 at 11:02:40AM -0500, Melanie Plageman wrote: > > This does however leave me with the question of how to handle the > > original question of whether or not to cap the proposed relallfrozen > > to the value of relallvisibl

Re: Redact user password on pg_stat_statements

2025-02-25 Thread Sami Imseih
> > Well sure, but best effort is better than no effort at all. Preventing > CREATE/ALTER will catch 99% of items, and as I advocated, there really is no > reason for them to be in pg_stat_statements in the first place. > >> >> The clients that set passwords could simply turn off track_utility on

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-25 Thread Jacob Champion
On Mon, Feb 24, 2025 at 2:02 PM Jacob Champion wrote: > Fair enough. I'll work on a patch to disallow it; best case, no one > ever complains, and we've pruned an entire configuration from the list > of things to worry about. Here goes: - 0001 fails configuration if the AsynchDNS feature is not b

Re: Update docs for UUID data type

2025-02-25 Thread Laurenz Albe
On Mon, 2025-02-24 at 21:04 -0500, Andy Alsup wrote: > Please find the attached patch, which only addresses the UUID functions > (in table format). I appreciate the comments related to the UUID datatype. > If you feel like the additional content didn't add clarity, I certainly won't > argue. Your

Re: Adjusting hash join memory limit to handle batch explosion

2025-02-25 Thread Tomas Vondra
On 2/25/25 17:30, James Hunter wrote: > On Wed, Feb 19, 2025 at 12:22 PM Tomas Vondra wrote: >> >> I've pushed the first (and main) part of the patch series, after some >> more cleanup and comment polishing. > > Two comments on your merged patch -- > > First, it's easier to see what's going on i

A small correction to doc and comment of FSM for indexes

2025-02-25 Thread Alex Friedman
Hi, This patch fixes a couple of small inaccuracies in the doc and the comment for FSM about index handling. 1. In the doc for pg_freespacemap, it currently says: For indexes, what is tracked is entirely-unused pages, rather than free space within pages. Therefore, the values are not meaning

Re: Proposal - Reduce lock during first phase of VACUUM TRUNCATE from ACCESS EXCLUSIVE to EXCLUSIVE

2025-02-25 Thread Ramanathan
> Except that mid-transaction lock upgrades increase the risk of deadlock failures. Thanks for the feedback. However, wouldn’t that risk already exist in the current vacuum truncate process? As it stands, VACUUM TRUNCATE performs a lock upgrade—from ShareUpdateExclusiveLock to AccessExclusiveLock—

bug in stored generated column over domain with constraints.

2025-02-25 Thread jian he
hi. create domain d1 as int not null; create domain d2 as int check (value > 1); create domain d3 as int check (value is not null); create table t0(b int, a d3 GENERATED ALWAYS as (b + 11) stored); insert into t0 values (1, default); ERROR: value for domain d3 violates check constraint "d3_check

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-02-25 Thread Shubham Khanna
On Tue, Feb 25, 2025 at 4:50 PM Hayato Kuroda (Fujitsu) wrote: > > Dear Shubham, > > Thanks for updating the patch. Few comments. > > 01. CreateSubscriberOptions > ``` > + boolcleanup_existing_publications; /* drop all > publications */ > ``` > > My previous comment [1] #1 did

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Greg Sabino Mullane
On Tue, Feb 25, 2025 at 11:03 AM Melanie Plageman wrote: > Because users can now manually update these values in pg_class, there > wouldn't be a way to detect the difference > between a bogus relallfrozen value due to VM corruption or a bogus value > due to manual statistics intervention. Er..y

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Nathan Bossart
On Tue, Feb 25, 2025 at 11:02:40AM -0500, Melanie Plageman wrote: > This does however leave me with the question of how to handle the > original question of whether or not to cap the proposed relallfrozen > to the value of relallvisible when updating stats at the end of > vacuum. The current code i

Re: Adjusting hash join memory limit to handle batch explosion

2025-02-25 Thread James Hunter
On Wed, Feb 19, 2025 at 12:22 PM Tomas Vondra wrote: > > I've pushed the first (and main) part of the patch series, after some > more cleanup and comment polishing. Two comments on your merged patch -- First, it's easier to see what's going on if we overlook the logic to round to nearest power o

Re: Redact user password on pg_stat_statements

2025-02-25 Thread Greg Sabino Mullane
On Tue, Feb 25, 2025 at 10:12 AM Sami Imseih wrote: > > What about a more general solution, such as a flag to turn off logging > of ALTER ROLE statements completely? > > IMO, flags for a specific type of utility statement seems way too much for > pg_stat_statements, and this will also not complet

Re: Fix logging for invalid recovery timeline

2025-02-25 Thread David Steele
On 2/25/25 04:07, Benoit Lobréau wrote: On 2/24/25 11:33 PM, Michael Paquier wrote: On Mon, Feb 24, 2025 at 05:30:35PM +, David Steele wrote: +    /* translator: %s is a backup_label or pg_control file */ See for example PostmasterMain() with the "/* translator

Commit fest 2025-03

2025-02-25 Thread vignesh C
Hi Everyone, The next PostgreSQL CommitFest is starting in a few days, and I wanted to remind everyone to register their patches to ensure they receive the necessary attention and feedback. I've noticed that a few patches are currently unregistered. If you're still evaluating a POC or finalizing t

Re: PATCH: warn about, and deprecate, clear text passwords

2025-02-25 Thread Greg Sabino Mullane
On Tue, Feb 25, 2025 at 10:34 AM Nathan Bossart wrote: > I noticed a nearby thread [0] in which there appears to be some budding > support for a GUC that disables sending passwords to the server in > clear-text, at least for CREATE/ALTER ROLE. Yep, that was the thread that inspired this patch!

Re: Adding extension default version to \dx

2025-02-25 Thread Nathan Bossart
On Tue, Feb 25, 2025 at 04:42:37PM +0100, Magnus Hagander wrote: > Thanks goes to both you and the previous responders - I did manage to mute > this thread away and missed the early replies, but got Jeltes the other day > which brought it back up on my list to get to within the Not Too Long > Futu

Re: Redact user password on pg_stat_statements

2025-02-25 Thread Tom Lane
Sami Imseih writes: >> What about a more general solution, such as a flag to turn off logging of >> ALTER ROLE statements completely? > IMO, flags for a specific type of utility statement seems way too much > for pg_stat_statements, and this will also not completely prevent leaking > plain text

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Nathan Bossart
On Tue, Feb 25, 2025 at 10:38:48AM -0500, Robert Haas wrote: > On Mon, Feb 24, 2025 at 6:35 PM Melanie Plageman > wrote: >> I'm on the fence about adding a PANIC. We do PANIC in other places >> where we notice corruption (like PageAddItemExtended()). But, in most >> of the cases, it seems like we

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Melanie Plageman
On Tue, Feb 25, 2025 at 10:39 AM Robert Haas wrote: > > On Mon, Feb 24, 2025 at 6:35 PM Melanie Plageman > wrote: > > I'm on the fence about adding a PANIC. We do PANIC in other places > > where we notice corruption (like PageAddItemExtended()). But, in most > > of the cases, it seems like we ar

Re: Adding extension default version to \dx

2025-02-25 Thread Magnus Hagander
On Tue, Feb 25, 2025 at 4:40 PM Nathan Bossart wrote: > On Sun, Feb 23, 2025 at 04:05:34PM +0100, Jelte Fennema-Nio wrote: > > On Sun Feb 23, 2025 at 3:51 PM CET, Jelte Fennema-Nio wrote: > >> Attached is an updated patch that fixes the tests and changes the > >> whitespace as discussed (includin

Re: Adding extension default version to \dx

2025-02-25 Thread Nathan Bossart
On Sun, Feb 23, 2025 at 04:05:34PM +0100, Jelte Fennema-Nio wrote: > On Sun Feb 23, 2025 at 3:51 PM CET, Jelte Fennema-Nio wrote: >> Attached is an updated patch that fixes the tests and changes the >> whitespace as discussed (including removing a spurious second space >> before the ON, which I not

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Robert Haas
On Mon, Feb 24, 2025 at 6:35 PM Melanie Plageman wrote: > I'm on the fence about adding a PANIC. We do PANIC in other places > where we notice corruption (like PageAddItemExtended()). But, in most > of the cases, it seems like we are PANICing because there isn't a > reasonable way to accomplish t

Re: Get rid of WALBufMappingLock

2025-02-25 Thread Alexander Korotkov
On Tue, Feb 18, 2025 at 2:29 AM Alexander Korotkov wrote: > > On Tue, Feb 18, 2025 at 2:21 AM Michael Paquier wrote: > > > > On Mon, Feb 17, 2025 at 11:25:05AM -0500, Tom Lane wrote: > > > This timeout failure on hachi looks suspicious as well: > > > > > > https://buildfarm.postgresql.org/cgi-bin

Re: PATCH: warn about, and deprecate, clear text passwords

2025-02-25 Thread Nathan Bossart
On Mon, Feb 24, 2025 at 04:20:44PM -0500, Greg Sabino Mullane wrote: > On Mon, Feb 24, 2025 at 4:18 PM Nathan Bossart > wrote: >> Well, the discussion upthread suggests "disallowing plain text passwords >> completely" > > Yeah, that's more of a long-term dream than a real plan. It would certainly

Re: Trigger more frequent autovacuums of heavy insert tables

2025-02-25 Thread Melanie Plageman
On Mon, Feb 24, 2025 at 10:30 PM wenhui qiu wrote: > > Hi Melanie > > relallvisible. It seems like we should make it consistent. Perhaps we > > should just remove it from heap_vacuum_rel(). Then add an assert in > > all these places to at least protect development mistakes. > I think there's some

Re: Redact user password on pg_stat_statements

2025-02-25 Thread Sami Imseih
> What about a more general solution, such as a flag to turn off logging of > ALTER ROLE statements completely? IMO, flags for a specific type of utility statement seems way too much for pg_stat_statements, and this will also not completely prevent leaking plain text passwords from all ways that

Re: per backend WAL statistics

2025-02-25 Thread Bertrand Drouvot
Hi, On Tue, Feb 25, 2025 at 03:50:38PM +0900, Michael Paquier wrote: > On Mon, Feb 24, 2025 at 09:07:39AM +, Bertrand Drouvot wrote: > > Now that 2421e9a51d2 is in, let's resume working in this thread. PFA a > > rebase to > > make the CF bot happy. Nothing has changed since V7, V8 only remove

Lowering temp_buffers minimum

2025-02-25 Thread Andres Freund
Hi, I am working on writing some tests for temporary tables. One of the tests is that we correctly handle running out of buffer pins. That's a bit more annoying than it needs to because the minimum for temp_buffers is 100. It seems rather odd that our minimum for temp_buffers is 100 while the min

Re: Make query cancellation keys longer

2025-02-25 Thread Jelte Fennema-Nio
On Mon, 24 Feb 2025 at 18:24, Robert Haas wrote: > > On Sat, Feb 22, 2025 at 9:58 AM Jelte Fennema-Nio wrote: > > My patchset in the other protocol thread needed a rebase. So I took > > that as an opportunity to rebase this patchset on top of it, because > > this seems to be the protocol change t

Re: GetRelationPath() vs critical sections

2025-02-25 Thread Andres Freund
Hi, On 2025-02-24 10:50:21 -0500, Andres Freund wrote: > Since I've not heard anything on that point, I'm currently thinking with going > with return-by-value. Attached are two very mildly updated patches. Most of > the changes is added commit messages. And pushed. Greetings, Andres Freund

  1   2   >