Re: Avoid circular header file dependency

2025-04-26 Thread Michael Paquier
On Sat, Apr 26, 2025 at 01:20:56AM -0400, Tom Lane wrote: > Bertrand Drouvot writes: >> While working on wait events I faced some compilation issues due to circular >> header file dependency (introduced in fa88928470b5) between wait_event.h and >> wait_event_types.h. > > Ugh. I still carry the s

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-04-24 Thread Michael Paquier
On Wed, Apr 23, 2025 at 11:59:26AM -0400, Robert Haas wrote: > That's nice to know, but I think the key question is not so much what > the feature costs when it is used but what it costs when it isn't > used. If we implement a system where we don't let > dictionary-compressed zstd datums leak out o

Re: Improve verification of recovery_target_timeline GUC.

2025-04-24 Thread Michael Paquier
On Thu, Apr 24, 2025 at 03:34:29PM +, David Steele wrote: > Done. This means there are no commas in the upper bound but I don't think > it's a big deal and it more closely matches other GUC messages. One thing that I have double-checked is if we have similar messages for GUCs that are defined

Re: Add Pipelining support in psql

2025-04-23 Thread Michael Paquier
On Wed, Apr 23, 2025 at 04:13:14PM +0900, Michael Paquier wrote: > The tweak for psql_fails_like() was kind of independent of the rest of > the fix, so I have applied that as a commit of its own. I am not > convinced about the addition of a 4th test where we use the queries > wit

Re: Avoid core dump in pgstat_read_statsfile()

2025-04-23 Thread Michael Paquier
On Wed, Apr 23, 2025 at 08:01:40AM +, Bertrand Drouvot wrote: > I think that a check on pgstat_get_kind_info() is missing for this scenario, > the > patch adds it. Such a check already exists for PGSTAT_FILE_ENTRY_FIXED and > for stats entry identified by name on disk, but not for > PGSTAT_FI

Re: Add Pipelining support in psql

2025-04-23 Thread Michael Paquier
have applied that as a commit of its own. I am not convinced about the addition of a 4th test where we use the queries with semicolons without a \getresults between the sync and the end. -- Michael From 2641356ffacc0c69779a6252a8072f41521c0f3f Mon Sep 17 00:00:00 2001 From: Michael Paquier Date:

Re: Recent pg_rewind test failures in buildfarm

2025-04-22 Thread Michael Paquier
On Tue, Apr 22, 2025 at 06:22:51AM +, Bertrand Drouvot wrote: > Yeah, unless that might come from fc415edf8ca but I don't think that's the > case. I've been eyeing this whole area of the code for a few hours to convince myself on HEAD, and I cannot really find a defect directly related to it.

Re: Add pg_get_injection_points() for information of injection points

2025-04-21 Thread Michael Paquier
ttached if the build uses USE_INJECTION_POINTS. -- Michael From 53447da561a03afa7e162c2bd478f3b76f984209 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Tue, 22 Apr 2025 12:59:30 +0900 Subject: [PATCH v4 1/2] Add InjectionPointList() to retrieve list of injection points This hides the inter

Re: doc patch: clarify the naming rule for injection_points

2025-04-21 Thread Michael Paquier
On Mon, Apr 21, 2025 at 12:10:51PM +, Hayato Kuroda (Fujitsu) wrote: > Thanks for suggesting them. ISTM, you are correct. PSA updated version. Thanks, I've applied some slight tweaks, and applied the result down to v17, leaving the heap_update point alone. -- Michael signature.asc Descriptio

Re: Add Pipelining support in psql

2025-04-21 Thread Michael Paquier
On Mon, Apr 21, 2025 at 03:22:28PM +0900, Michael Paquier wrote: > Anyway, I don't think that there is much we can do under a > PGRES_FATAL_ERROR in this code path when discarding the pipe results. > As far as I can tell, the server has failed the query suddenly and the > whole

Re: doc patch: clarify the naming rule for injection_points

2025-04-21 Thread Michael Paquier
On Mon, Apr 14, 2025 at 12:36:20PM +, Hayato Kuroda (Fujitsu) wrote: > I also feel just converting lower case is not good. The point seems to locate > in > the end-of-transaction callback and it accepts invalidation messages. Based > on the > fact, how about "inval-process-invalidation-messag

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-04-21 Thread Michael Paquier
On Fri, Apr 18, 2025 at 12:22:18PM -0400, Robert Haas wrote: > I think we could add plain-old zstd compression without really > tackling this issue, but if we are going to add dictionaries then I > think we might need to revisit the idea of preventing things from > leaking out of tables. What I can

Re: Add Pipelining support in psql

2025-04-20 Thread Michael Paquier
On Wed, Apr 16, 2025 at 09:18:01AM -0700, Michael Paquier wrote: > On Wed, Apr 16, 2025 at 09:31:59PM +0700, a.kozhemyakin wrote: >> After commit 2cce0fe on master >> >> ERROR:  unexpected message type 0x50 during COPY from stdin >> CONTEXT:  COPY psql_pipeline,

Re: pipelining in psql, commit 41625ab

2025-04-20 Thread Michael Paquier
On Wed, Apr 16, 2025 at 04:13:10PM -0700, Noah Misch wrote: > Works for me. I have fixed the three defects you have reported with 5ee7bd944ee8, 5743d122fcf3 and 78231baaf967 for respectively the error code of \getresults, the description of \flush[request] and the output of --help=commands with th

Re: Recent pg_rewind test failures in buildfarm

2025-04-20 Thread Michael Paquier
dded in v15. Thoughts? -- Michael From 7010fb6e02cd5a6923b75829807f9f406356b57b Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 21 Apr 2025 10:13:56 +0900 Subject: [PATCH] Remove assertion in pgstat.c WAL senders can flush stats, making this assertions triggerable. --- src/backend/utils

Re: Typos in the code and README

2025-04-20 Thread Michael Paquier
On Mon, Apr 21, 2025 at 10:42:19AM +1200, David Rowley wrote: > On Sun, 20 Apr 2025 at 13:32, David Rowley wrote: >> Here are a few more fixes of a similar ilk. All new in 2025, >> predominantly from the last few days before feature freeze. > > I've pushed those ones. Thanks for sharing the comm

Re: Rename injection point names in test_aio

2025-04-19 Thread Michael Paquier
On Wed, Apr 16, 2025 at 03:25:19PM -0700, Michael Paquier wrote: > If anybody objects about the points getting renamed here, please let > me know. Applied this one as 114f7fa81c72. -- Michael signature.asc Description: PGP signature

Re: Typos in the code and README

2025-04-19 Thread Michael Paquier
On Sat, Apr 19, 2025 at 11:00:00AM +0300, Alexander Lakhin wrote: > I've gathered the following collection of typos and inconsistencies > appeared since 2025-01-01: Good finds, Alexander, particularly for the set of function and variable names used in the comments that do not match with the realit

Re: pg_dump: Fix dangling pointer in EndCompressorZstd()

2025-04-16 Thread Michael Paquier
On Wed, Apr 16, 2025 at 04:19:02PM +0200, Daniel Gustafsson wrote: > Agreed, while it's perfectly safe today the end method should not make > assumptions about the use of the private_data pointer upon return and should > leave it set to NULL. Indeed. I was just looking at applying what Alexander

Re: Rename injection point names in test_aio

2025-04-16 Thread Michael Paquier
On Mon, Apr 14, 2025 at 11:14:59AM +, Hayato Kuroda (Fujitsu) wrote: >> Anyway, would you like to propose a patch for the documentation? > > Sure, I did. Please see [1]. > > [1]: > https://www.postgresql.org/message-id/OSCPR01MB14966E14C1378DEE51FB7B7C5F5B32%40OSCPR01MB14966.jpnprd01.prod.ou

Re: Support for runtime parameters in injection points, for AIO tests

2025-04-16 Thread Michael Paquier
On Mon, Apr 14, 2025 at 04:05:40AM -0400, Andres Freund wrote: > On 2025-04-14 16:46:22 +0900, Michael Paquier wrote: >> We are still in early April, and I'd like to propose a cleanup of the >> AIO code on HEAD, even if we are post-feature freeze, to not release >> t

Re: pipelining in psql, commit 41625ab

2025-04-16 Thread Michael Paquier
On Tue, Apr 15, 2025 at 02:34:50PM -0700, Noah Misch wrote: > On Fri, Feb 21, 2025 at 11:33:41AM +0900, Michael Paquier wrote: > commit 41625ab wrote: >> * \syncpipeline queues a synchronisation request, without flushing the >> commands to the server, equivalent of

Re: Add Pipelining support in psql

2025-04-16 Thread Michael Paquier
On Wed, Apr 16, 2025 at 09:31:59PM +0700, a.kozhemyakin wrote: > After commit 2cce0fe on master > > ERROR:  unexpected message type 0x50 during COPY from stdin > CONTEXT:  COPY psql_pipeline, line 1 > Pipeline aborted, command did not run > psql: common.c:1510: discardAbortedPipelineResults: Asser

Re: Add pg_get_injection_points() for information of injection points

2025-04-15 Thread Michael Paquier
On Mon, Apr 14, 2025 at 05:26:14PM +0500, Kirill Reshke wrote: > Hi! I noticed you do not bump catalog version here, so i was a little > confused with > ``` > reshke=# SELECT name, library, function FROM pg_get_injection_points(); > ERROR: function pg_get_injection_points() does not exist > ``` >

Re: Recent pg_rewind test failures in buildfarm

2025-04-14 Thread Michael Paquier
On Mon, Apr 14, 2025 at 10:58:28PM -0400, Tom Lane wrote: > That assert appears to be several years old, and the > 008_min_recovery_point.pl test script that's triggering it hasn't > changed very recently either, so I'm baffled where to start digging. > It has the odor of a timing problem, so maybe

Re: Add pg_get_injection_points() for information of injection points

2025-04-14 Thread Michael Paquier
first patch anyway. -- Michael From d738943616f1af8fc2ce1d309a3b7a0b77f29a22 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Tue, 15 Apr 2025 14:40:30 +0900 Subject: [PATCH v3 1/2] Add InjectionPointList() to retrieve list of injection points This hides the internals of the shmem array lookup

Re: Add pg_get_injection_points() for information of injection points

2025-04-14 Thread Michael Paquier
On Mon, Apr 14, 2025 at 04:29:37PM +0300, Aleksander Alekseev wrote: > If I didn't miss anything, all SQL functions dealing with injection > points are gathered in injection_points extension so IMO > pg_get_injection_points() belongs there. It would be awkward to have > it in the core considering t

Re: Add pg_get_injection_points() for information of injection points

2025-04-14 Thread Michael Paquier
hould not use "idx" > for > the result array - PSA the fix. Oops. That's a brain fart from my side. Thanks. -- Michael From ee033a0f883d52638cc9c90b9c0eff29b9dde5a5 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Tue, 15 Apr 2025 08:39:36 +0900 Subject: [PATCH v2] Ad

Re: Rename injection point names in test_aio

2025-04-14 Thread Michael Paquier
On Mon, Apr 14, 2025 at 08:13:44AM +, Hayato Kuroda (Fujitsu) wrote: > I have no objections for the patch, but I feel there are no concrete naming > rules > (I confused while creating patches). There has been a sort of implied rule in the code to use lower characters, with terms separated by

Support for runtime parameters in injection points, for AIO tests

2025-04-14 Thread Michael Paquier
0001. The CI is passing. Thoughts and comments are welcome. -- Michael From 7e8453987d127144477a8e7e9fa7ae61e48091be Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 14 Apr 2025 16:24:18 +0900 Subject: [PATCH 1/3] Add support for runtime arguments in injection points The mac

Re: Adding facility for injection points (or probe points?) for more advanced tests

2025-04-14 Thread Michael Paquier
On Mon, Apr 14, 2025 at 01:24:30PM +0900, Michael Paquier wrote: > IMO, we ought to clean up the way the AIO code does its tests with > injection point with something like the patch of this thread. And > perhaps even do it in v18 to have the code in a cleaner state at > release. I&#x

Rename injection point names in test_aio

2025-04-13 Thread Michael Paquier
Hi all, (well, Andres) 93bc3d75d8e1 has introduced a couple of new injection points, but these don't follow the convention in-place where points are named more-or-less-like-that. Please find attached a patch to make all these more consistent. Thoughts or comments? -- Michael diff --git a/src/bac

Re: Adding facility for injection points (or probe points?) for more advanced tests

2025-04-13 Thread Michael Paquier
On Wed, Feb 05, 2025 at 09:19:22AM +0900, Michael Paquier wrote: > On Mon, Feb 03, 2025 at 09:30:33PM -0800, Jeff Davis wrote: >> One extra benefit of supporting arguments is that it would be a more >> flexible way to change the local state around the injection point. >> Righ

Add pg_get_injection_points() for information of injection points

2025-04-13 Thread Michael Paquier
, but I am sending an independent patch as this stuff is useful on its own. The patch includes a couple of tests and some documentation. Thanks, -- Michael From 8865e2ca3b1b6291e94d2a5476926095245a4e2f Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Mon, 14 Apr 2025 09:26:52 +0900 Subject: [

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-13 Thread Michael Paquier
On Sun, Apr 13, 2025 at 11:51:57AM -0400, Tom Lane wrote: > Noah Misch writes: > > Tom and Michael, do you still object to the test addition, or not? If there > > are no new or renewed objections by 2025-04-20, I'll proceed to add the > > test. > > While I still don't love it, I don't have a be

Re: stats.sql fails during installcheck on mac

2025-04-11 Thread Michael Paquier
On Fri, Apr 11, 2025 at 07:37:49PM -0500, Sami Imseih wrote: > Yes, there should be some tests running for these stats, > so if it's possible to enable fsync on one or a few animals, that > will be better than nothing. I have just done that on batta that only tests HEAD, that's a start. -- Michael

Re: stats.sql fails during installcheck on mac

2025-04-11 Thread Michael Paquier
On Fri, Apr 11, 2025 at 10:44:59AM -0500, Sami Imseih wrote: > I actually originally had it this way, but for some reason > felt it would be better to be explicit about the methods we want to test > rather > than not test. I can't think of a very compelling reason to go either way, so > v2 > LGTM

Re: New committer: Jacob Champion

2025-04-11 Thread Michael Paquier
On Fri, Apr 11, 2025 at 01:26:04PM -0700, Jonathan S. Katz wrote: > Please join us in wishing Jacob much success and few reverts! Congratulations and welcome, Jacob! May your path lead to a peaceful buildfarm and few reverts. -- Michael signature.asc Description: PGP signature

Re: Silence resource leaks alerts

2025-04-10 Thread Michael Paquier
On Thu, Apr 10, 2025 at 03:10:02PM -0300, Ranier Vilela wrote: > While it is arguable that this is a false warning, there is a benefit in > moving the initialization of the string buffer, silencing the warnings that > are presented in this case. > > 1. pg_overexplain.c > 2. ruleutils.c These code

Re: merge file_exists_in_directory and _fileExistsInDirectory functions and move into common file dumputils.c

2025-04-10 Thread Michael Paquier
On Thu, Apr 10, 2025 at 10:41:33PM +0530, Mahendra Singh Thalor wrote: > We have file_exists_in_directory function in pg_restore.c and same > code we are using in _fileExistsInDirectory function in pg_backup_archiver.c > also. > Here, I am attaching a patch to move these duplicate functions into >

Re: stats.sql fails during installcheck on mac

2025-04-10 Thread Michael Paquier
On Thu, Apr 10, 2025 at 04:39:45PM -0500, Sami Imseih wrote: > Also, The documentation for pg_stat_wal already makes this point clear > in [0] that "wal_sync is only > incremented when the wal_sync_method is either fdatasync, fsync or > fsync_writethrough". > > Perhaps, the same clarification will

Re: stats.sql fails during installcheck on mac

2025-04-10 Thread Michael Paquier
On Thu, Apr 10, 2025 at 06:45:36PM -0500, Sami Imseih wrote: > IIUC, this is only an issue for wal syncing Yes, good catch. I have missed this effect of issue_xlog_fsync(), which has two callers. The first one in XLogWrite() never happens if wal_sync_method is open_sync or open_datasync. The se

Re: sync_standbys_defined read/write race on startup

2025-04-10 Thread Michael Paquier
On Thu, Apr 10, 2025 at 12:55:54PM +0300, Maksim.Melnikov wrote: > On 10.04.2025 12:15, Michael Paquier wrote: >> Hmm, yeah. Instead of last, it would be better to put it in second >> place perhaps, for clarity? That would be the same at the end, but we >> would be slightly

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-10 Thread Michael Paquier
On Tue, Apr 08, 2025 at 08:47:53AM +0530, Ashutosh Bapat wrote: > Thanks for addressing the comment. > > In PG code, we start a multiline comment with just /* on the first > line and end with just */ on the last line. All the lines in-between > start with * . Please check other comments in the fil

Re: Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-04-10 Thread Michael Paquier
On Mon, Apr 07, 2025 at 07:13:20AM +, Bertrand Drouvot wrote: > As we now have 2 code paths I think we "really" need 2 tests. The attached > (to apply on top of v7) seems to do the job. Confirmed. I am sold on this extra location on HEAD, and it does not impact the run time of the test as the

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-10 Thread Michael Paquier
On Tue, Apr 08, 2025 at 01:26:59AM -0400, Tom Lane wrote: > +1. When I saw the patch I was mainly afraid that pgindent would > make a hash of it. Yes. I was actually surprised to see that it did not mess up with the code generated. -- Michael signature.asc Description: PGP signature

Re: sync_standbys_defined read/write race on startup

2025-04-10 Thread Michael Paquier
u think? Hmm, yeah. Instead of last, it would be better to put it in second place perhaps, for clarity? That would be the same at the end, but we would be slightly more consistent with the past logic regarding the ordering. Does that look OK to you? -- Michael From b3b43a7eff2fd83a118cab13b99acd6cc8

Re: Consistently use macro HeapTupleIsValid to check the validity of tuples in tablecmds.c

2025-04-09 Thread Michael Paquier
On Wed, Apr 09, 2025 at 05:43:24PM +0300, Heikki Linnakangas wrote: > Agreed. I use both, depending on which mood I'm in. Same here, extended to OidIsValid(), HeapTupleIsValid(), XLogRecPtr, etc., and I tend to prefer such macros, except if consistency of the surroundings matter most. FWIW, I thi

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-09 Thread Michael Paquier
On Wed, Apr 09, 2025 at 12:07:31PM +0530, Amit Kapila wrote: > On Wed, Apr 9, 2025 at 11:24 AM Bertrand Drouvot > wrote: > I can't think of a good reason to have this DEBUG1 as there is no > predictability of it getting generated even with tests using an > injection point. OTOH, I don't have any o

Re: sync_standbys_defined read/write race on startup

2025-04-09 Thread Michael Paquier
s been the origin of some spurious failures in the buildfarm. I cannot point my finger to a particular one now, but it would not go unnoticed especially on the slow animals.. What do you think? -- Michael From f60a004116bc21440b8769c75f11126ac4ef667f Mon Sep 17 00:00:00 2001 From: Michael Paquier Da

Re: sync_standbys_defined read/write race on startup

2025-04-09 Thread Michael Paquier
On Mon, Apr 07, 2025 at 11:39:23PM +0300, Maksim.Melnikov wrote: > Hi everyone, thanks for your comments. > I've just wanted to share little cosmetic > fixes for previous patch. Note: I would suggest to use a version number for the patches, rather than a 0003 which usually implies that this is thi

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-08 Thread Michael Paquier
On Tue, Apr 08, 2025 at 06:42:53AM +, Bertrand Drouvot wrote: > Fully agree. Will need to find another way to prevent a process to wait > between the > wakeup and the detach. I'll open a dedicated thread. By the way, there is a small thing that's itching me a bit about the change done in LogS

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-08 Thread Michael Paquier
On Tue, Apr 08, 2025 at 11:21:31PM -0300, Euler Taveira wrote: > The logical replication creates origin names as pg_SUBOID_RELID or pg_SUBOID. > It means the maximum origin name is 24. This limited origin name also applies > to pglogical that limits the name to 54 IIRC. I think that covers the majo

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-08 Thread Michael Paquier
On Tue, Apr 08, 2025 at 06:19:02AM +, Bertrand Drouvot wrote: > - A new injection_points_wakeup_detach() function that is holding the spinlock > during the whole duration to ensure that no process can wait in between the > wakeup and the detach. That would not a correct spinlock use. injectio

Re: Fix 035_standby_logical_decoding.pl race conditions

2025-04-07 Thread Michael Paquier
On Tue, Apr 08, 2025 at 02:00:35AM +, Hayato Kuroda (Fujitsu) wrote: > Your patch looks good to me and it could pass on my env. PSA patches for PG16. > Patch for PG17 is not changed, just renamed. @@ -1287,6 +1288,17 @@ LogStandbySnapshot(void) Assert(XLogStandbyInfoActive()); +#ifdef

Re: Horribly slow pg_upgrade performance with many Large Objects

2025-04-07 Thread Michael Paquier
On Mon, Apr 07, 2025 at 05:25:32PM -0400, Tom Lane wrote: > What version are you testing? We did some work in that area in the > v17 cycle (a45c78e32). I am puzzled by the target version used here, as well. If there is more that can be improved, v19 would be the version to consider for future im

Re: Proposal - Allow extensions to set a Plan Identifier

2025-04-07 Thread Michael Paquier
On Mon, Apr 07, 2025 at 06:11:06PM -0700, Lukas Fittl wrote: > However, since we're basically at feature freeze (and it seems unlikely > this gets into 18), I have a quick alternate proposal: > > What if, for 18, we just make DoJumble exported to be used by plugins? Yes, I am not planning to rush

Re: prevent 006_transfer_modes.pl from leaving files behind

2025-04-07 Thread Michael Paquier
On Mon, Apr 07, 2025 at 04:45:52PM -0500, Nathan Bossart wrote: > The other pg_upgrade tests chdir to tmp_check prior to running pg_upgrade > to avoid leaving behind delete_old_cluster.{sh,bat}. 006_transfer_modes.pl > should, too. However, this test is a little different because it loops > over

Re: Correct mismatched verb in a message

2025-04-06 Thread Michael Paquier
On Mon, Apr 07, 2025 at 03:13:59PM +0900, Kyotaro Horiguchi wrote: > The recent commit 1495eff7bdb introduced a message using a plural verb > for a singular subject. > > > pg_log_info("number of restored databases are %d", num_db_restore); > ~~~ >

Re: Fwd: [BUG]: the walsender does not update its IO statistics until it exits

2025-04-06 Thread Michael Paquier
On Fri, Apr 04, 2025 at 09:33:46PM +0530, vignesh C wrote: > The new test added currently passes even without the patch. It would > be ideal to have a test that fails without the patch and passes once > the patch is applied. Right. The subscription test and logical WAL senders passes without the

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-04-06 Thread Michael Paquier
On Thu, Apr 03, 2025 at 12:36:59AM +0900, Ryo Kanbayashi wrote: > I'll reflect your notice and suggestion to the patch current I'm > working on :) Thanks for that. And I have forgotten to add you as a reviewer of what has been committed as 2c7bd2ba507e. Sorry for that :/ -- Michael signature.a

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-04-06 Thread Michael Paquier
On Mon, Mar 31, 2025 at 04:26:27PM -0500, Andrew Jackson wrote: > I am working on a feature adjacent to the connection service functionality > and noticed some issues with the tests introduced in this thread. Basically > they incorrectly invoke the append perl function by passing multiple > strings

Re: per backend WAL statistics

2025-04-06 Thread Michael Paquier
On Mon, Mar 31, 2025 at 07:42:19AM +, Bertrand Drouvot wrote: > I think we can simply move the pgstat_fetch_stat_backend() call at the end > of pgstat_fetch_stat_backend_by_pid(), like in the attached. With this in > place > the issue is fixed on my side. > > Thoughts? Confirmed. I agree th

Re: Logging which local address was connected to in log_line_prefix

2025-04-06 Thread Michael Paquier
On Sun, Apr 06, 2025 at 06:01:01PM -0400, Tom Lane wrote: > I don't really care for #3 or #4, but I'm ambivalent between #1 and > #2. I think the worry about confusion originated when the patch > would print "[local]" for either a Unix socket or a background > process, and that certainly was confu

Re: [PATCH] clarify palloc comment on quote_literal_cstr

2025-04-06 Thread Michael Paquier
On Sun, Apr 06, 2025 at 12:37:24PM -0500, Steve Chavez wrote: > I found the numbers in `quote_literal_cstr` palloc quite magical. So I've > added a comment clarifying what they mean. The change is small: > > /* We make a worst-case result area; wasting a little space is OK */ > - res

Re: Typo in comment for pgstat_database_flush_cb()

2025-04-06 Thread Michael Paquier
On Sun, Apr 06, 2025 at 01:31:50PM +0200, Etsuro Fujita wrote: > +1 for both suggestions. So I modified the comment as such in each > file with such a flush_cb function. I will push the patch. Thanks for the fix. Could you also backpatch that down to v15? It would be good to keep this level of

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-05 Thread Michael Paquier
On Sat, Apr 05, 2025 at 07:14:27PM +0900, Michael Paquier wrote: > On Wed, Apr 02, 2025 at 05:29:00PM -0700, Noah Misch wrote: >> Your back-patches are correct. Thanks. > > Thanks for double-checking. I'll move on with what I have after a > second look as it's been a

Re: rename pg_log_standby_snapshot

2025-04-05 Thread Michael Paquier
On Sat, Apr 05, 2025 at 10:32:40PM -0400, Andres Freund wrote: > I think this would all be a nice argument to have when introducing a new > function. But I don't think it's a wart sufficiently big to justify breaking > compatibility. Yeah, I would side as well with the compatibility argument on th

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-05 Thread Michael Paquier
On Sat, Apr 05, 2025 at 12:13:39PM -0700, Noah Misch wrote: > Since the 2025-02 releases made non-toy-size archive recoveries fail easily, > that's not enough. If the proposed 3-second test is the wrong thing, what > instead? I don't have a good idea about that in ~16, TBH, but I am sure to not b

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-04-05 Thread Michael Paquier
On Fri, Mar 21, 2025 at 02:45:09PM +0900, Michael Paquier wrote: > Potential ideas about optimizing the branches seem worth > investigating, I am not sure that I would be able to dig into that > until the feature freeze gong, unfortunately, and it would be nice to > fix this issue for

Re: Proposal - Allow extensions to set a Plan Identifier

2025-04-05 Thread Michael Paquier
On Thu, Mar 20, 2025 at 09:46:55PM -0400, Sami Imseih wrote: > * For the same reasons as the query identifiers (see above), > > but, I went ahead and commented it similar to how we document > pgstat_report_query_id and pgstat_get_my_query_id routines. > attached is v2-0001 Looks mostly OK from he

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-04-05 Thread Michael Paquier
On Tue, Mar 18, 2025 at 05:24:42PM +1300, David Rowley wrote: > I had a look at this and it seems the main difference will be that > this patch will protect against the case that a given node is non-null > but has a custom jumble function which selects to not jumble anything > in some cases. Since

Re: Test to dump and restore objects left behind by regression

2025-04-05 Thread Michael Paquier
On Thu, Mar 27, 2025 at 06:15:06PM +0100, Alvaro Herrera wrote: > BTW another idea to shorten this tests's runtime might be to try and > identify which of parallel_schedule tests leave objects behind and > create a shorter schedule with only those (a possible implementation > might keep a list of t

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-05 Thread Michael Paquier
On Wed, Apr 02, 2025 at 05:29:00PM -0700, Noah Misch wrote: > Your back-patches are correct. Thanks. Thanks for double-checking. I'll move on with what I have after a second look as it's been a few weeks since I've looked at all these conflicts. I am also planning to add a few notes in the comm

Re: Fix infinite loop from setting scram_iterations to INT_MAX

2025-04-04 Thread Michael Paquier
On Wed, Mar 26, 2025 at 03:41:10PM +0900, Richard Guo wrote: > I plan to go ahead and push Kevin's fix, barring any objections. Thanks, Richard. -- Michael signature.asc Description: PGP signature

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-04-04 Thread Michael Paquier
On Sat, Mar 22, 2025 at 10:43:00AM +0100, Christoph Berg wrote: > Are we at the point where the patch is already Ready for Committer? I'll think a bit more about how to document all that. Anyway, yes, I'm OK with the per-field custom_query_jumble, so let's move on with that, so I will do somethin

Re: A small correction to doc and comment of FSM for indexes

2025-04-04 Thread Michael Paquier
On Tue, Feb 25, 2025 at 07:24:14PM +0200, Alex Friedman wrote: > However, as what is tracked is entirely-unused pages, the values > mean whether a page is "in-use or empty", rather than "full or > empty". Ah, right, you are obviously right here. Applied and backpatched doen to 13 as the docs of p

Re: Small memory fixes for pg_createsubcriber

2025-04-01 Thread Michael Paquier
On Apr 2, 2025, at 5:14, Noah Misch wrote:Yes, pg_upgrade should not do that.  I was trying to explain the differencebetween NULL and 'NULL'.  I didn't mean pg_upgrade should emit "IS NULL".(Apologies for the probably-weirdly-formatted message)This issue has been already mentioned around here, wit

Re: per backend WAL statistics

2025-03-31 Thread Michael Paquier
> On Mar 31, 2025, at 16:42, Bertrand Drouvot > wrote: > I think we can simply move the pgstat_fetch_stat_backend() call at the end > of pgstat_fetch_stat_backend_by_pid(), like in the attached. With this in > place > the issue is fixed on my side. Thanks for the patch, I’ll check all that next

Re: per backend WAL statistics

2025-03-28 Thread Michael Paquier
On Fri, Mar 28, 2025 at 09:00:00PM +0200, Alexander Lakhin wrote: > Please try the following query: > BEGIN; > SET LOCAL stats_fetch_consistency = snapshot; > SELECT * FROM pg_stat_get_backend_wal(pg_backend_pid()); > > with sanitizers (or under Valgrind). When I run it, I get: > 2025-03-28 18:38:

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-27 Thread Michael Paquier
On Thu, Mar 27, 2025 at 06:31:14PM +0900, Michael Paquier wrote: > With all that in mind and more documentation added to the test, I've > applied 0001, so let's see what the buildfarm has to say. The CI was > stable, so it's a start. The buildfarm (particularly the Wind

Re: pg_stat_database.checksum_failures vs shared relations

2025-03-27 Thread Michael Paquier
On Thu, Mar 27, 2025 at 12:06:45PM -0400, Robert Haas wrote: > On Thu, Mar 27, 2025 at 11:58 AM Andres Freund wrote: > > So, today we have the weird situation that *some* checksum errors on shared > > relations get attributed to the current database (if they happen in a > > backend > > normally a

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-27 Thread Michael Paquier
On Thu, Mar 27, 2025 at 06:31:14PM +0900, Michael Paquier wrote: > I am not sure that I'll have the time to look at 0002 for this release > cycle, could it be possible to get a rebase for it? Here is a simple rebase that I have been able to assemble this morning. I won't have the

Re: [PATCH] PGSERVICEFILE as part of a normal connection string

2025-03-27 Thread Michael Paquier
On Sun, Mar 23, 2025 at 12:32:03PM +0900, Ryo Kanbayashi wrote: > Additional test scripts have been merged to a single script ^^ b I have spent quite a bit of time on the review 0001 with the new tests to get something in for this release, and there was quite a bit going on there: - The script sho

Re: Remove useless casts to (char *)

2025-03-26 Thread Michael Paquier
On Wed, Mar 26, 2025 at 10:01:47PM +0700, Vladlen Popolitov wrote: > I mentioned this patch in my message > https://www.postgresql.org/message-id/f28f3b45ec84bf9dc99fe129023a2d6b%40postgrespro.ru > Starting from it queries with Parallel Seq Scan (probably with other > parallel executor nodes) > ha

Re: making EXPLAIN extensible

2025-03-26 Thread Michael Paquier
On Wed, Mar 26, 2025 at 02:09:32PM -0400, Robert Haas wrote: > On Sat, Mar 22, 2025 at 12:10 PM Robert Haas wrote: > > I'm not going > > to insist on shipping this if I'm the ONLY one who would ever get any > > use out of it, but I doubt that's the case. > > Hearing no other votes against this, I

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-26 Thread Michael Paquier
On Thu, Mar 27, 2025 at 12:09:35PM +1300, David Rowley wrote: > I think it's better it's two commits. They're for different purposes. Fine for me. > Ok, I won't protest. I just feel the bug is a relic of the old design > and I don't believe the tests are testing anything useful in the new > desig

Re: Use CLOCK_MONOTONIC_COARSE for instr_time when available

2025-03-26 Thread Michael Paquier
On Wed, Mar 26, 2025 at 11:14:47AM -0700, 杨江华 wrote: > This patch modifies the instr_time.h header to prefer CLOCK_MONOTONIC_COARSE > when available. By using CLOCK_MONOTONIC_COARSE, we can leverage a lower > resolution(4ms) but faster alternative for timing operations, which reduces > the overhead

Re: Doc: clarify possibility of ephemeral discrepancies between state and wait_event in pg_stat_activity

2025-03-26 Thread Michael Paquier
On Mon, Mar 03, 2025 at 11:35:15AM -0600, Sami Imseih wrote: > Thanks for the update. This LGTM! and I will mark as RFC. Yes, agreed that there is no specific need to be precise about the attributes that can become inconsistent, as this would also depend on the addition of more states, or event mo

Re: sync_standbys_defined read/write race on startup

2025-03-26 Thread Michael Paquier
On Wed, Mar 26, 2025 at 05:17:17PM +0300, Maksim.Melnikov wrote: > Test check visibility of prepared transactions in standby after a restart > while primary is down. > Failed assert check that changes, commited on transaction on primary node, > were synchronously replicated to standby node. > In te

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-25 Thread Michael Paquier
On Tue, Mar 25, 2025 at 07:56:29PM -0500, Sami Imseih wrote: > FWIW, the pg_stat_statements docs in a few places refer to > queries that may look different but have the same meaning > as “semantically equivalent”, this is why I used the same > terminology here. But, I have no issue with the sim

Re: Proposal - Allow extensions to set a Plan Identifier

2025-03-25 Thread Michael Paquier
On Tue, Mar 25, 2025 at 04:23:15PM -0500, Sami Imseih wrote: > On 3/25/25 00:47, Sami Imseih wrote: >> 1. Check out the upstream Postgres source for the given version I'm >> generating jumble code for >> 2. Modify the headers as needed to have the node attributes I want >> 3. Call the gen_node_sup

Re: [PATCH] pg_stat_activity: make slow/hanging authentication more visible

2025-03-25 Thread Michael Paquier
On Mon, Mar 17, 2025 at 10:22:47AM -0700, Jacob Champion wrote: > I looked into switching over to pgstat_report_activity(), but that > wasn't designed to be called in the middle of backend initialization. > It would take more work to make those calls safe/sane when `st_state > == STATE_STARTING`. I

Re: Query ID Calculation Fix for DISTINCT / ORDER BY and LIMIT / OFFSET

2025-03-25 Thread Michael Paquier
On Wed, Mar 26, 2025 at 11:56:50AM +1300, David Rowley wrote: > Here is the v8 version with the bug fix and performance stuff > separated out. Why not. I assume that you would merge these together? > I also added the Assert code to ensure we always add > something to the jumble buffer when jumbl

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-25 Thread Michael Paquier
ar > queries"; I think writing "semantically" will just confuse people. If I get the difference right, semantically would apply to concepts related to linguistics, but that's not what we have here, so you are using a more general term. Thanks for the suggestions. -- Michae

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-25 Thread Michael Paquier
ed. I've also included your doc suggestions. I feel that we could do better here, but that's a common statement anyway when it comes to the documentation. -- Michael From 8ea61bb0d7d6c4dbbb40dbaedb5e751027163cfe Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Wed, 26 Mar 2025

Re: Proposal - Allow extensions to set a Plan Identifier

2025-03-25 Thread Michael Paquier
On Mon, Mar 24, 2025 at 06:47:59PM -0500, Sami Imseih wrote: > I know it was mentioned above by both Michael and Andrei that > AppendJumble should not be exposed. I am not sure I agree with > that. I think it should be exposed, along with > JUMBLE_FIELD, JUMBLE_FIELD_SINGLE and JUMBLE_STRING > and

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-25 Thread Michael Paquier
, and the fixes for the issues pointed out by Christoph. I'll double-check all that again tomorrow. Please find an updated version attached for now. -- Michael From 68d363fbee484b40308a00a85329364ff0901e9b Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Tue, 25 Mar 2025 15:40:10 +0900 S

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-24 Thread Michael Paquier
On Mon, Mar 24, 2025 at 10:30:59PM -0500, Sami Imseih wrote: >> Sami Imseih writes: >>> I agree that some may want stats to merge for the same tables, >>> and others may not. I will suggest this with some hesitation, but why not >>> make this behavior configurable via a GUC? >>> We recently introd

Re: query_id: jumble names of temp tables for better pg_stat_statement UX

2025-03-24 Thread Michael Paquier
On Mon, Mar 24, 2025 at 09:38:35PM -0500, Sami Imseih wrote: > > select * from foo s1; > > select * from foo s2; > > ah, thanks for pointing this out. Not as good of a workaround as > a comment since one must change aliases, but at least there is > a workaround... Exactly. Like Tom I'm not reall

  1   2   3   4   5   6   7   8   9   10   >