Re: Issues with 2PC at recovery: CLOG lookups and GlobalTransactionData

2025-06-06 Thread Noah Misch
On Thu, Jun 05, 2025 at 04:22:48PM +0900, Michael Paquier wrote: > On Mon, Jun 02, 2025 at 06:48:46PM -0700, Noah Misch wrote: > > The wasShutdown case reaches consistency from the beginning, so I don't see > > that as an example of a time we benefit from reading pg_twophase

Re: Wrong security context for deferred triggers?

2025-06-05 Thread Noah Misch
On Thu, Jun 05, 2025 at 11:10:04AM -0400, Tom Lane wrote: > Noah Misch writes: > > Phrase "the role that executed the statement" doesn't match what happens if > > the role changes mid-statement. Example of a statement that does so: > > > select set_conf

Re: PG 18 release notes draft committed

2025-06-04 Thread Noah Misch
On Wed, Jun 04, 2025 at 06:10:38PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I think we have to keep the non-SECURITY-DEFINER designation to keep the > > text accurate, I don't see it that way, because there's currently no experiment the user can perform to distinguish between the follow

Re: PG 18 release notes draft committed

2025-06-04 Thread Noah Misch
On Wed, Jun 04, 2025 at 04:45:18PM -0400, Bruce Momjian wrote: > On Tue, Jun 3, 2025 at 10:21:23AM -0700, Noah Misch wrote: > > When a commit changes the user that runs a function in existing queries, I > > think that almost always needs a release notes entry. It would follow

Re: PG 18 release notes draft committed

2025-06-03 Thread Noah Misch
On Thu, May 01, 2025 at 10:44:50PM -0400, Bruce Momjian wrote: > I have committd the first draft of the PG 18 release notes. When a commit changes the user that runs a function in existing queries, I think that almost always needs a release notes entry. It would follow that commit 01463e1 needs a

Re: Issues with 2PC at recovery: CLOG lookups and GlobalTransactionData

2025-06-02 Thread Noah Misch
On Fri, May 09, 2025 at 02:08:26PM +0900, Michael Paquier wrote: > On Tue, Feb 18, 2025 at 04:57:47PM -0800, Noah Misch wrote: > > As I wrote in [1], "By the time we reach consistency, every file in > > pg_twophase will be applicable (not committed or aborted)." If we fin

Re: Inval reliability, especially for inplace updates

2025-05-10 Thread Noah Misch
On Sun, Nov 03, 2024 at 10:29:25AM -0800, Noah Misch wrote: > On Fri, Nov 01, 2024 at 04:38:29PM -0700, Noah Misch wrote: > > This was a near miss to having a worst-in-years regression in a minor > > release, > > so I'm proposing this sequence: > > > > - R

Re: AIO v2.5

2025-05-08 Thread Noah Misch
On Thu, May 08, 2025 at 09:06:18PM -0400, Andres Freund wrote: > On 2025-05-02 20:05:11 -0700, Noah Misch wrote: > > On Wed, Apr 30, 2025 at 04:00:35PM -0400, Andres Freund wrote: > We do need to hold interrupts in a few other places, I think - with some debug > infrastructu

Re: AIO v2.5

2025-05-02 Thread Noah Misch
On Wed, Apr 30, 2025 at 04:00:35PM -0400, Andres Freund wrote: > pgaio_io_wait_for_free() does what it says on the tin. For that, after a bunch > of other things, finds the oldest in-flight IO and waits for it. > > PgAioHandle *ioh = dclist_head_element(PgAioHandle, node, >

Re: dispchar for oauth_client_secret

2025-04-28 Thread Noah Misch
On Mon, Apr 21, 2025 at 08:18:58AM -0700, Jacob Champion wrote: > Attached is my proposed fix. 0001 disables use of the new oauth_* > options in our FDWs. 0002 changes dispchar. > Subject: [PATCH 1/2] oauth: Disallow OAuth connections via postgres_fdw/dblink > Subject: [PATCH 2/2] oauth: Classify

Re: type cache cleanup improvements

2025-04-28 Thread Noah Misch
On Mon, Apr 21, 2025 at 04:54:08AM +0300, Alexander Korotkov wrote: > On Sat, Apr 12, 2025 at 12:43 AM Alexander Korotkov > wrote: > > On Fri, Apr 11, 2025 at 11:32 PM Noah Misch wrote: > > > On Tue, Oct 22, 2024 at 08:33:24PM +0300, Alexander Korotkov wrote: > > &g

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-26 Thread Noah Misch
On Fri, Apr 25, 2025 at 03:35:06PM -0400, Andres Freund wrote: > On 2025-04-20 14:53:39 -0700, Noah Misch wrote: > > The checkpoints and WAL creation took 30s, but archiving was only 20% done > > (based on file name 0001006D) at the 360s PGCTLTIMEOUT. >

Re: pg_upgrade-breaking release

2025-04-24 Thread Noah Misch
On Thu, Apr 24, 2025 at 08:37:56AM -0400, Bruce Momjian wrote: > On Thu, Apr 24, 2025 at 08:35:10AM -0400, Greg Sabino Mullane wrote: > > On Thu, Apr 24, 2025 at 8:12 AM Bruce Momjian wrote: > > > > Do we think most people are _not_ going to use pg_upgrade now that we > > are defaulting t

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-20 Thread Noah Misch
On Sun, Apr 20, 2025 at 02:53:39PM -0700, Noah Misch wrote: > On Mon, Apr 14, 2025 at 09:19:35AM +0900, Michael Paquier wrote: > > On Sun, Apr 13, 2025 at 11:51:57AM -0400, Tom Lane wrote: > > > Noah Misch writes: > > > > Tom and Michael, do you still object to

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-20 Thread Noah Misch
On Mon, Apr 14, 2025 at 09:19:35AM +0900, Michael Paquier wrote: > On Sun, Apr 13, 2025 at 11:51:57AM -0400, Tom Lane wrote: > > Noah Misch writes: > > > Tom and Michael, do you still object to the test addition, or not? If > > > there > > > are no new or re

Re: Unicode full case mapping: PG_UNICODE_FAST, and standard-compliant UCS_BASIC

2025-04-20 Thread Noah Misch
On Sat, Apr 19, 2025 at 12:30:57PM -0700, Jeff Davis wrote: > On Thu, 2025-04-17 at 06:58 -0700, Noah Misch wrote: > > Should initcap_wbnext() pass in a locale-dependent "bool posix" > > argument like > > the others calls the commit changed? > > Yes, I bel

Re: Unicode full case mapping: PG_UNICODE_FAST, and standard-compliant UCS_BASIC

2025-04-17 Thread Noah Misch
On Fri, Jan 17, 2025 at 04:06:20PM -0800, Jeff Davis wrote: > Committed 0001 and 0002. > Upon reviewing the discussion threads, I removed the Unicode "adjust to > Cased" behavior when titlecasing. As Peter pointed out[1], it doesn't > match the documentation or expectations for INITCAP(). While c

Re: pipelining in psql, commit 41625ab

2025-04-16 Thread Noah Misch
On Wed, Apr 16, 2025 at 09:46:42AM -0700, Michael Paquier wrote: > On Tue, Apr 15, 2025 at 02:34:50PM -0700, Noah Misch wrote: > > On Fri, Feb 21, 2025 at 11:33:41AM +0900, Michael Paquier wrote: > > commit 41625ab wrote: > >> --- a/src/bin/psql/help.c > &

Re: [18] Unintentional behavior change in commit e9931bfb75

2025-04-15 Thread Noah Misch
On Tue, Apr 15, 2025 at 04:08:55PM -0700, Jeff Davis wrote: > On Mon, 2025-04-14 at 13:44 -0700, Noah Misch wrote: > > However, I think v17's > > concept of separate PG_REGEX_ symbols for the default-locale case is > > still the > > right thing for v18.  In other w

pipelining in psql, commit 41625ab

2025-04-15 Thread Noah Misch
On Fri, Feb 21, 2025 at 11:33:41AM +0900, Michael Paquier wrote: > So let's take one step here, I have applied the main patch. commit 41625ab wrote: > * \syncpipeline queues a synchronisation request, without flushing the > commands to the server, equivalent of PQsendPipelineSync(). libpq

Re: dispchar for oauth_client_secret

2025-04-15 Thread Noah Misch
On Tue, Apr 15, 2025 at 01:16:12PM -0700, Jacob Champion wrote: > On Tue, Apr 15, 2025 at 12:14 PM Noah Misch wrote: > > I suspect this should use .dispchar="*" to encourage UIs to display > > oauth_client_secret like a password field. Thoughts? > > Hmm, from

dispchar for oauth_client_secret

2025-04-15 Thread Noah Misch
commit b3f0be7 wrote: + {"oauth_scope", NULL, NULL, NULL, + "OAuth-Scope", "", 15, + offsetof(struct pg_conn, oauth_scope)}, The field containing "" is documented as follows: char *dispchar; /* Indicates how to display this field in a

Re: [18] Unintentional behavior change in commit e9931bfb75

2025-04-15 Thread Noah Misch
On Mon, Apr 14, 2025 at 12:59:50PM -0700, Jeff Davis wrote: > On Sat, 2025-04-12 at 05:34 -0700, Noah Misch wrote: > > I think the code for (2) and for "I/i in Turkish" haven't returned.  > > Given > > commit e3fa2b0 restored the v17 "I/i in Turkish"

Re: Wrong security context for deferred triggers?

2025-04-15 Thread Noah Misch
On Thu, Jan 23, 2025 at 07:28:19PM +0100, Laurenz Albe wrote: > On Thu, 2025-01-23 at 12:30 -0500, Tom Lane wrote: > > Pushed with some cosmetic adjustments > > Thank you! commit 01463e1 wrote: > +NOTICE: I am regress_groot Let's not incur trivially-avoidable trademark risks (https://google.com

Re: Call for Posters: PGConf.dev 2025

2025-04-14 Thread Noah Misch
Thanks for running this. Four logistical questions: 1. By what timestamp should one submit? 2. Where does one send the submission? On Sun, Apr 13, 2025 at 04:01:56PM +0500, Andrey Borodin wrote: > The goal of the poster session is to visually present your patch or project > on an A2-sized poste

Re: datfrozenxid > relfrozenxid w/ crash before XLOG_HEAP_INPLACE

2025-04-13 Thread Noah Misch
On Sun, Apr 06, 2025 at 11:00:54AM -0700, Noah Misch wrote: > I pushed that as commit 8e7e672 (2024-10-25). I now think DELAY_CHKPT_START > is superfluous here, per this proc.h comment: > > * (In the > * extremely common case where the data being modified is in shared buff

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-13 Thread Noah Misch
On Sat, Apr 05, 2025 at 07:09:58PM -0700, Noah Misch wrote: > On Sun, Apr 06, 2025 at 07:42:02AM +0900, Michael Paquier wrote: > > On Sat, Apr 05, 2025 at 12:13:39PM -0700, Noah Misch wrote: > > > Since the 2025-02 releases made non-toy-size archive recoveries fail > > &g

Re: Buildfarm: Enabling injection points on basilisk/dogfish (Alpine / musl)

2025-04-12 Thread Noah Misch
On Sat, Apr 12, 2025 at 07:51:06PM +0200, Wolfgang Walther wrote: > With injection points enabled, I get the following errors in test_aio: > > > [15:14:45.408](0.000s) not ok 187 - worker: first hard IO error is reported: > expected stderr > [15:14:45.409](0.000s) > [15:14:45.409](0.000s) #   Fai

Re: [18] Unintentional behavior change in commit e9931bfb75

2025-04-12 Thread Noah Misch
On Mon, Dec 02, 2024 at 10:24:07PM -0800, Jeff Davis wrote: > On Mon, 2024-12-02 at 17:25 -0500, Tom Lane wrote: > > > Should I put the special case back? > > > > I think so. > > Done. I put the special case back in (commit e3fa2b037c) because the > earlier commit wasn't intended to be a behavio

Re: type cache cleanup improvements

2025-04-11 Thread Noah Misch
On Tue, Oct 22, 2024 at 08:33:24PM +0300, Alexander Korotkov wrote: > On Tue, Oct 22, 2024 at 6:10 PM Pavel Borisov wrote: > > On Tue, 22 Oct 2024 at 11:34, Alexander Korotkov > > wrote: > >> I'm going to push this if no objections. (This became commit b85a9d0.) > + /* Call delete_rel_type

catcache search while BUFFER_LOCK_EXCLUSIVE on catalog

2025-04-10 Thread Noah Misch
4 #35 0x0093fe47 in postmaster_child_launch (child_type=B_BACKEND, child_slot=20, startup_data=0x7fff47fe66b0, startup_data_len=24, client_sock=0x7fff47fe6700) at launch_backend.c:290 #36 0x00945d55 in BackendStartup (client_sock=0x7fff47fe6700) at postmaster.c:3580 #37 0x000

Re: datfrozenxid > relfrozenxid w/ crash before XLOG_HEAP_INPLACE

2025-04-06 Thread Noah Misch
On Wed, Jun 19, 2024 at 06:29:08PM -0700, Noah Misch wrote: > https://postgr.es/m/20240512232923.aa.nmi...@google.com wrote: > > Separable, nontrivial things not fixed in the attached patch stack: > > > - Trouble is possible, I bet, if the system crashes between the

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-05 Thread Noah Misch
On Sun, Apr 06, 2025 at 07:42:02AM +0900, Michael Paquier wrote: > On Sat, Apr 05, 2025 at 12:13:39PM -0700, Noah Misch wrote: > > Since the 2025-02 releases made non-toy-size archive recoveries fail easily, > > that's not enough. If the proposed 3-second test is the wrong thi

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-05 Thread Noah Misch
On Sat, Apr 05, 2025 at 11:07:13AM -0400, Tom Lane wrote: > Michael Paquier writes: > > On Wed, Apr 02, 2025 at 05:29:00PM -0700, Noah Misch wrote: > >> Here it is. Making it fail three times took looping 1383s, 5841s, and > >> 2594s. > >> Hence, it couldn&#

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-05 Thread Noah Misch
On Tue, Mar 11, 2025 at 06:23:15PM -0700, Noah Misch wrote: > On Wed, Mar 12, 2025 at 09:46:27AM +0900, Michael Paquier wrote: > > On Tue, Mar 11, 2025 at 01:57:49PM -0700, Noah Misch wrote: > > > Thanks for crafting back-branch versions. I've queued a task to confirm >

Re: AIO v2.5

2025-04-05 Thread Noah Misch
On Sun, Mar 23, 2025 at 11:57:48AM -0400, Andres Freund wrote: > On 2025-03-22 19:09:55 -0700, Noah Misch wrote: > > On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > > > Attached v2.11 > > > > > Subject: [PATCH v2.11 05/27] aio: Add io_method=io_uri

Re: AIO v2.5

2025-04-05 Thread Noah Misch
On Mon, Mar 24, 2025 at 11:43:47AM -0400, Andres Freund wrote: > On 2025-03-23 17:29:39 -0700, Noah Misch wrote: > > commit 247ce06b wrote: > > > + pgaio_io_reopen(ioh); > > > + > > > + /* > > > + *

Re: AIO v2.5

2025-04-05 Thread Noah Misch
On Thu, Mar 20, 2025 at 02:54:14PM -0400, Andres Freund wrote: > On 2025-03-19 18:11:18 -0700, Noah Misch wrote: > > On Wed, Mar 19, 2025 at 06:17:37PM -0400, Andres Freund wrote: > > > On 2025-03-19 14:25:30 -0700, Noah Misch wrote: > > > > I see this relies on md_r

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-04-05 Thread Noah Misch
On Mon, Mar 17, 2025 at 07:52:02PM -0400, Andres Freund wrote: > Here's a proposed patch for this. It turns out that the bug might already be > reachable, even without defining FDDEBUG. There's a debug ereport() in > register_dirty_segment() - but it's hard to reach in practice. > > I don't really

Re: AIO v2.5

2025-04-05 Thread Noah Misch
On Fri, Apr 04, 2025 at 11:53:13PM -0400, Andres Freund wrote: > On 2025-04-04 14:18:02 -0700, Noah Misch wrote: > > On Fri, Apr 04, 2025 at 03:16:18PM -0400, Andres Freund wrote: > > > > - Make DEFINED in completor before verifying page. It might be > > > > cl

Re: AIO v2.5

2025-04-04 Thread Noah Misch
On Fri, Apr 04, 2025 at 03:16:18PM -0400, Andres Freund wrote: > On 2025-04-03 12:40:23 -0700, Noah Misch wrote: > > On Thu, Apr 03, 2025 at 02:19:43PM -0400, Andres Freund wrote: > > In the general case, we could want client requests as follows: > > > > - If com

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-04-04 Thread Noah Misch
ning against backpatching, but I'm not sure that's not just > laziness. It's also some risk reduction. One of these smgr APIs might have a useful interruptibility that we're now blocking. (I'm not aware of one.) > On 2025-03-19 17:45:14 -0700, Noah Misch wrote: &

Re: dblink query interruptibility

2025-04-03 Thread Noah Misch
On Wed, Mar 12, 2025 at 11:03:41AM +0100, Andreas Karlsson wrote: > On 3/12/25 12:48 AM, Noah Misch wrote: > > Overall, in the absence of objections, I will queue a task to back-patch the > > non-postgres_fdw portion of commit d3c5f37 to v13-v16. Pushed (e.g. v16 has commit 82a8

Re: AIO v2.5

2025-04-03 Thread Noah Misch
On Thu, Apr 03, 2025 at 02:19:43PM -0400, Andres Freund wrote: > 4b) > > That's not all though, after getting past this failure, I see uninitialized > memory errors for reads into temporary buffers: > > ==3334031== VALGRINDERROR-BEGIN > ==3334031== Conditional jump or move depends on uninitialise

Re: AIO v2.5

2025-04-01 Thread Noah Misch
On Tue, Apr 01, 2025 at 06:25:28PM -0400, Andres Freund wrote: > On 2025-04-01 17:47:51 -0400, Andres Freund wrote: > > 3) Some subtests fail if RELCACHE_FORCE_RELEASE and CATCACHE_FORCE_RELEASE > > are defined: > > > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2025-04-01%2

Re: TEMP_CONFIG vs test_aio

2025-04-01 Thread Noah Misch
On Tue, Apr 01, 2025 at 03:42:44PM -0400, Andres Freund wrote: > The reason for the failure is simple, the buildfarm animal specifies > io_method=io_uring (thanks to "cookt" for setting that up so quickly, whoever > you are :)) and the test is assuming that the -c io_method=... it passes to > initd

Re: Small memory fixes for pg_createsubcriber

2025-04-01 Thread Noah Misch
On Tue, Apr 01, 2025 at 04:28:34PM -0300, Ranier Vilela wrote: > Em ter., 1 de abr. de 2025 às 15:39, Noah Misch > escreveu: > > > On Thu, Feb 27, 2025 at 10:23:31AM -0300, Ranier Vilela wrote: > > > Em qui., 27 de fev. de 2025 às 02:51, Michael Paquier < > > mi

Re: Small memory fixes for pg_createsubcriber

2025-04-01 Thread Noah Misch
On Thu, Feb 27, 2025 at 10:23:31AM -0300, Ranier Vilela wrote: > Em qui., 27 de fev. de 2025 às 02:51, Michael Paquier > escreveu: > > > On Tue, Feb 25, 2025 at 08:54:31AM -0300, Ranier Vilela wrote: > > > @@ -455,7 +455,9 @@ set_locale_and_encoding(void) > > > locale->db_locale, > > > strlen

Re: AIO v2.5

2025-04-01 Thread Noah Misch
On Tue, Apr 01, 2025 at 11:55:20AM -0400, Andres Freund wrote: > On 2025-04-01 08:11:59 -0700, Noah Misch wrote: > > On Mon, Mar 31, 2025 at 08:41:39PM -0400, Andres Freund wrote: > I haven't yet pushed the changes, but will work on that in the afternoon. > > I plan to

Re: AIO v2.5

2025-04-01 Thread Noah Misch
On Mon, Mar 31, 2025 at 08:41:39PM -0400, Andres Freund wrote: > updated version All non-write patches (1-7) are ready for commit, though I have some cosmetic recommendations below. I've marked the commitfest entry Ready for Committer. > + # Check a page validity error in another blo

Re: AIO v2.5

2025-03-29 Thread Noah Misch
On Sat, Mar 29, 2025 at 08:39:54PM -0400, Andres Freund wrote: > On 2025-03-29 14:29:29 -0700, Noah Misch wrote: > > On Wed, Mar 26, 2025 at 09:07:40PM -0400, Andres Freund wrote: > > The choice between LOG and LOG_SERVER_ONLY doesn't matter much for $SUBJECT. > > If a

Re: AIO v2.5

2025-03-29 Thread Noah Misch
ges - but > that's not right, the GUC hopefully is only set on a per-session basis Good catch. I've now audited the complete_shared callbacks for other variable references and actions not acceptable there. I found nothing beyond what you found by v2.14. On Sat, Mar 29, 2025 at 1

Re: AIO v2.5

2025-03-29 Thread Noah Misch
On Fri, Mar 28, 2025 at 11:35:23PM -0400, Andres Freund wrote: > The number of combinations is annoyingly large. It's e.g. plausible to use > ignore_checksum_failure=on and zero_damaged_pages=on at the same time for > recovery. That's intricate indeed. > But I finally got to a point where the cod

Re: AIO v2.5

2025-03-27 Thread Noah Misch
On Thu, Mar 27, 2025 at 04:58:11PM -0400, Andres Freund wrote: > I now wrote some tests. And I both regret doing so (because it found problems, > which would have been apparent long ago, if the feature had come with *any* > tests, if I had gone the same way I could have just pushed stuff) and am gl

Re: AIO v2.5

2025-03-27 Thread Noah Misch
On Wed, Mar 12, 2025 at 01:06:03PM -0400, Andres Freund wrote: > On 2025-03-11 20:57:43 -0700, Noah Misch wrote: > > - Like you say, "redefine max_files_per_process to be about the number of > > files each *backend* will additionally open". It will become normal that &g

Re: AIO v2.5

2025-03-26 Thread Noah Misch
On Wed, Mar 26, 2025 at 04:33:49PM -0400, Andres Freund wrote: > On 2025-03-25 17:19:15 -0700, Noah Misch wrote: > > On Mon, Mar 24, 2025 at 09:18:06PM -0400, Andres Freund wrote: > > Second, the aio_internal.h comment changes discussed in > > postgr.es/m/20250325155808.f7.

Re: AIO v2.5

2025-03-26 Thread Noah Misch
I reviewed everything up to and including "[PATCH v2.12 17/28] aio, bufmgr: Comment fixes", the last patch before write support. postgr.es/m/20250326001915.bc.nmi...@google.com covered patches 1-9, and this email covers patches 10-17. All remaining review comments are minor, so I've marked the com

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Tue, Mar 25, 2025 at 08:17:17PM -0400, Andres Freund wrote: > On 2025-03-25 09:15:43 -0700, Noah Misch wrote: > > On Tue, Mar 25, 2025 at 11:57:58AM -0400, Andres Freund wrote: > > > FWIW, I prototyped this, it's not hard. > > > > > > But it can't

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Mon, Mar 24, 2025 at 09:18:06PM -0400, Andres Freund wrote: > Attached v2.12, with the following changes: > TODO: > Wonder if it's worth adding some coverage for when checksums are disabled? > Probably not necessary? Probably not necessary, agreed. Orthogonal to AIO, it's likely worth a

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Tue, Mar 25, 2025 at 04:56:53PM -0400, Andres Freund wrote: > The repeated-iteration approach taken in pgaio_closing_fd() isn't the > prettiest, but it's hard to to imagine that ever being a noticeable. Yep. I've reviewed the fixup code, and it looks all good. > This survives a testrun where

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Tue, Mar 25, 2025 at 04:07:35PM -0400, Andres Freund wrote: > On 2025-03-25 12:39:56 -0700, Noah Misch wrote: > > On Tue, Mar 25, 2025 at 02:58:37PM -0400, Andres Freund wrote: > > > There are 2 1/2 ways around this: > > > > > > 1) Stop using IOSQE_ASYNC

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Tue, Mar 25, 2025 at 02:58:37PM -0400, Andres Freund wrote: > On 2025-03-25 08:58:08 -0700, Noah Misch wrote: > > While having nagging thoughts that we might be releasing FDs before io_uring > > gets them into kernel custody, I tried this hack to maximize FD turnover: >

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Tue, Mar 25, 2025 at 11:57:58AM -0400, Andres Freund wrote: > On 2025-03-25 07:11:20 -0700, Noah Misch wrote: > > On Mon, Mar 24, 2025 at 10:52:19PM -0400, Andres Freund wrote: > > > If we want to implement it, I think we could introduce PGAIO_RS_WARN, > > > whic

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Tue, Mar 25, 2025 at 11:26:14AM -0400, Andres Freund wrote: > On 2025-03-25 06:33:21 -0700, Noah Misch wrote: > > On Mon, Mar 24, 2025 at 10:30:27PM -0400, Andres Freund wrote: > > > On 2025-03-24 17:45:37 -0700, Noah Misch wrote: > > > > (We may be due for a test

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Mon, Mar 24, 2025 at 10:52:19PM -0400, Andres Freund wrote: > On 2025-03-24 19:20:37 -0700, Noah Misch wrote: > > On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > > > +static pg_attribute_always_inline PgAioResult > > > +buffer_readv_complete_one(u

Re: AIO v2.5

2025-03-25 Thread Noah Misch
On Mon, Mar 24, 2025 at 10:30:27PM -0400, Andres Freund wrote: > On 2025-03-24 17:45:37 -0700, Noah Misch wrote: > > (We may be due for a test mode that does smgrreleaseall() at every > > CHECK_FOR_INTERRUPTS()?) > > I suspect we are. I'm a bit afraid of even tr

Re: AIO v2.5

2025-03-24 Thread Noah Misch
On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > Subject: [PATCH v2.11 09/27] bufmgr: Implement AIO read support [I checked that v2.12 doesn't invalidate these review comments, but I didn't technically rebase the review onto v2.12's line numbers.] > static void > TerminateBuffer

Re: AIO v2.5

2025-03-23 Thread Noah Misch
On Sun, Mar 23, 2025 at 11:11:53AM -0400, Andres Freund wrote: > On 2025-03-22 17:20:56 -0700, Noah Misch wrote: > > On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > > > Not sure yet how to best disable testing io_uring in this case. We can't > > >

Re: AIO v2.5

2025-03-23 Thread Noah Misch
commit 247ce06b wrote: > + pgaio_io_reopen(ioh); > + > + /* > + * To be able to exercise the reopen-fails path, allow > injection > + * points to trigger a failure at this point. > + */ > +

Re: AIO v2.5

2025-03-22 Thread Noah Misch
On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > Attached v2.11, with the following changes: > - Added an error check for FileStartReadV() failing > > FileStartReadV() actually can fail, if the file can't be re-opened. I > thought it'd be important for the error message to dif

Re: AIO v2.5

2025-03-22 Thread Noah Misch
On Thu, Mar 20, 2025 at 09:58:37PM -0400, Andres Freund wrote: > Attached v2.11 > Subject: [PATCH v2.11 05/27] aio: Add io_method=io_uring Apart from some isolated cosmetic points, this is ready to commit: > + ereport(ERROR, > + errcode(err

Re: AIO v2.5

2025-03-20 Thread Noah Misch
On Thu, Mar 20, 2025 at 01:05:05PM -0400, Andres Freund wrote: > On 2025-03-19 18:17:37 -0400, Andres Freund wrote: > > On 2025-03-19 14:25:30 -0700, Noah Misch wrote: > > > > +* marked as failed. In case of a partial read, some > > > > buffers may

Re: AIO v2.5

2025-03-19 Thread Noah Misch
On Wed, Mar 19, 2025 at 06:17:37PM -0400, Andres Freund wrote: > On 2025-03-19 14:25:30 -0700, Noah Misch wrote: > > commit 55b454d wrote: > > > aio: Infrastructure for io_method=worker > > > > > + /* Try to launch one. */ > > > +

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-19 Thread Noah Misch
On Wed, Mar 19, 2025 at 06:45:20PM -0400, Andres Freund wrote: > On 2025-03-19 12:55:53 -0700, Noah Misch wrote: > > On Mon, Mar 17, 2025 at 07:52:02PM -0400, Andres Freund wrote: > > > @@ -362,12 +397,16 @@ smgrreleaseall(void) > > > if (SMgrRelationHash == N

Re: race condition in pg_class

2025-03-18 Thread Noah Misch
On Tue, Mar 18, 2025 at 03:03:52PM -0400, Andres Freund wrote: > Subject: [PATCH v1] meson: Flush stdout in testwrap > > Otherwise the progress won't reliably be displayed during a test. > --- > src/tools/testwrap | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/src/tools/testwrap b/src/

Re: AIO v2.5

2025-03-12 Thread Noah Misch
On Tue, Mar 11, 2025 at 07:55:35PM -0400, Andres Freund wrote: > On 2025-03-11 12:41:08 -0700, Noah Misch wrote: > > On Mon, Sep 16, 2024 at 01:51:42PM -0400, Andres Freund wrote: > > > On 2024-09-16 07:43:49 -0700, Noah Misch wrote: > What do we want to do for Conditiona

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-03-11 Thread Noah Misch
On Wed, Mar 12, 2025 at 09:46:27AM +0900, Michael Paquier wrote: > On Tue, Mar 11, 2025 at 01:57:49PM -0700, Noah Misch wrote: > > Thanks for crafting back-branch versions. I've queued a task to confirm I > > get > > the same result. > > Thanks for that. That

Re: dblink query interruptibility

2025-03-11 Thread Noah Misch
On Thu, Mar 06, 2025 at 02:57:01PM +0100, Andreas Karlsson wrote: > On 11/22/23 2:29 AM, Noah Misch wrote: > > Something as simple as the following doesn't respond to cancellation. In > > v15+, any DROP DATABASE will hang as long as it's running: > One of our cust

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-03-11 Thread Noah Misch
On Mon, Mar 10, 2025 at 02:25:28PM +0900, Michael Paquier wrote: > On Thu, Mar 06, 2025 at 01:44:30PM -0600, Nathan Bossart wrote: > > On Thu, Mar 06, 2025 at 11:30:13AM -0800, Noah Misch wrote: > >> 1. Make v14 and v13 skip WAL recycling and preallocation during archive >

Re: AIO v2.5

2025-03-11 Thread Noah Misch
On Mon, Sep 16, 2024 at 01:51:42PM -0400, Andres Freund wrote: > On 2024-09-16 07:43:49 -0700, Noah Misch wrote: > > For non-sync IO methods, I gather it's essential that a process other than > > the > > IO definer be scanning for incomplete IOs and completing them. >

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-03-11 Thread Noah Misch
On Thu, Dec 19, 2024 at 02:44:53PM +0900, Michael Paquier wrote: > On Wed, Dec 18, 2024 at 08:51:20PM -0500, Andres Freund wrote: > > I don't think the issue is actually quite as unlikely to be hit as reasoned > > in > > the commit message. The crash has indeed to happen between the link() and >

Re: BackgroundPsql swallowing errors on windows

2025-03-09 Thread Noah Misch
On Sun, Mar 09, 2025 at 12:47:34PM -0400, Andres Freund wrote: > On 2025-02-16 17:52:36 -0800, Noah Misch wrote: > > On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: > > > On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: > > > >Noah Misch writes:

Re: Refactoring postmaster's code to cleanup after child exit

2025-03-05 Thread Noah Misch
On Tue, Mar 04, 2025 at 05:50:34PM -0500, Andres Freund wrote: > On 2024-12-09 00:12:32 +0100, Tomas Vondra wrote: > > [23:48:44.444](1.129s) ok 3 - reserved_connections limit > > [23:48:44.445](0.001s) ok 4 - reserved_connections limit: matches > > process ended prematurely at > > /home/user/work/

Re: Licence preamble update

2025-02-28 Thread Noah Misch
On Fri, Feb 28, 2025 at 12:07:26PM -0500, Tom Lane wrote: > Noah Misch writes: > > On Thu, Feb 27, 2025 at 04:56:05PM +, Dave Page wrote: > >> --- a/COPYRIGHT > >> +++ b/COPYRIGHT > >> @@ -1,5 +1,5 @@ > >> PostgreSQL Database Management Syst

Re: Licence preamble update

2025-02-27 Thread Noah Misch
On Thu, Feb 27, 2025 at 04:56:05PM +, Dave Page wrote: > Per some brief discussion on the core list, the attached patch updates the > licence preamble to more accurately reflect the use of Postgres vs. > PostgreSQL (see https://www.postgresql.org/about/policies/project-name/ for > background fr

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-20 Thread Noah Misch
On Wed, Feb 19, 2025 at 10:48:29AM -0800, Masahiko Sawada wrote: > Thank you for reviewing the patches. I've fixed these issues and > attached the updated patches. Looks good. > I have one question about the 0001 patch; since we add > 'default_char_signedness' field to ControlFileData do we need

Re: GetRelationPath() vs critical sections

2025-02-20 Thread Noah Misch
On Thu, Feb 20, 2025 at 12:40:57PM -0500, Andres Freund wrote: > On 2025-02-20 14:00:10 +1300, Thomas Munro wrote: > > On Wed, Feb 19, 2025 at 3:35 PM Andres Freund wrote: > > > After thinking about this for an embarassingly long time, I think there's > > > actually a considerably better answer fo

Re: Issues with 2PC at recovery: CLOG lookups and GlobalTransactionData

2025-02-18 Thread Noah Misch
On Thu, Jan 30, 2025 at 03:36:20PM +0900, Michael Paquier wrote: > And I am beginning a new thread about going through an issue that Noah > has mentioned at [1], which is that the 2PC code may attempt to do > CLOG lookups at very early stage of recovery, where the cluster is not > in a consistent s

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-18 Thread Noah Misch
Apart from two doc issues, this is ready: On Tue, Feb 18, 2025 at 01:23:20PM -0800, Masahiko Sawada wrote: > On Mon, Feb 17, 2025 at 2:57 PM Noah Misch wrote: > > On Fri, Jan 17, 2025 at 05:11:41PM -0800, Masahiko Sawada wrote: > +However, when upgrading from Postgre

Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

2025-02-17 Thread Noah Misch
On Fri, Jan 17, 2025 at 05:11:41PM -0800, Masahiko Sawada wrote: > Thank you for reviewing the patches. I agree with all the comments you > made. I've addressed them and I've attached new version patches that > now have some regression tests. > Subject: [PATCH v3 4/5] pg_upgrade: Add --set-char-si

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 08:42:50PM -0500, Andres Freund wrote: > On February 16, 2025 7:50:18 PM EST, Tom Lane wrote: > >Noah Misch writes: > >> On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > >>> I think that > >>> IPC::Run may be screwin

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 06:18:44PM -0500, Tom Lane wrote: > Noah Misch writes: > > From the slow proxy's perspective, it can't rule out the program under test > > having done those two write() calls. The proxy doesn't have enough > > information to reconst

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 03:55:10PM -0500, Andres Freund wrote: > On 2025-02-16 10:47:40 -0800, Noah Misch wrote: > > On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > > > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > > > On Thu, Feb 13, 2025 at 1

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Sun, Feb 16, 2025 at 01:02:01PM -0500, Andres Freund wrote: > On 2025-02-16 09:39:43 -0800, Noah Misch wrote: > > On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > > > I suspect what's happening is that the communication with the > > > extern

Re: BackgroundPsql swallowing errors on windows

2025-02-16 Thread Noah Misch
On Thu, Feb 13, 2025 at 12:39:04PM -0500, Andres Freund wrote: > My understanding is that IPC::Run uses a proxy process on windows to execute > subprocesses and then communicates with that over TCP (or something along > those lines). Right. > I suspect what's happening is that the communication w

Re: Decision by Monday: PQescapeString() vs. encoding violation

2025-02-15 Thread Noah Misch
On Sat, Feb 15, 2025 at 01:23:51PM -0500, Andres Freund wrote: > On 2025-02-15 13:08:03 -0500, Tom Lane wrote: > > I studied the v3 patch a little and realized that it only fixes the > > behavior for the case of a complete-but-invalid multibyte character. > > If we have an incomplete character at t

Decision by Monday: PQescapeString() vs. encoding violation

2025-02-14 Thread Noah Misch
The security team has a question below about how best to proceed with a recent behavior change. Commit 5dc1e42b4fa6a4434afa7d7cdcf0291351a7b873 for this week's CVE-2025-1094 changed how PQescapeString()[1] reacts to input that is not valid in the client encoding. Before that commit, the function

Re: "postmaster became multithreaded" is reachable

2025-01-23 Thread Noah Misch
On Thu, Jan 23, 2025 at 01:51:32PM -0500, Tom Lane wrote: > Noah Misch writes: > > On Thu, Jan 23, 2025 at 01:22:13PM -0500, Tom Lane wrote: > >> An alternative idea (that maybe should also be done in PostmasterMain) > >> is to report the HINT only #ifdef __darw

Re: "postmaster became multithreaded" is reachable

2025-01-23 Thread Noah Misch
On Thu, Jan 23, 2025 at 01:22:13PM -0500, Tom Lane wrote: > ExitPostmaster() says: > > /* > * There is no known cause for a postmaster to become multithreaded after > * startup. Recheck to account for the possibility of unknown causes. > * This message uses LOG level, because a

Re: Issue with markers in isolation tester? Or not?

2025-01-22 Thread Noah Misch
On Sun, Jan 19, 2025 at 09:55:32PM +0100, Michail Nikolaev wrote: > > Stepping back, any variation in the total ordering of step-start and > > step-complete events necessarily changes the expected output. Hence, we > want > > to freeze that ordering. We could do that directly, by having > isola

  1   2   3   4   5   6   7   8   9   10   >