Re: Cleaning up historical portability baggage

2025-06-10 Thread Thomas Munro
On Tue, Jun 10, 2025 at 10:59 PM Michael Banck wrote: > I don't have an opinion here, I think it would be ok to just define it > to 16 if it is undefined and if the Hurd people want something better at > some point, they should submit patches. Cool. I will go ahead and do that, as you proposed,

Re: Cleaning up historical portability baggage

2025-06-09 Thread Thomas Munro
On Tue, Jun 10, 2025 at 2:25 AM Andres Freund wrote: > On 2025-06-09 15:25:22 +0200, Michael Banck wrote: > > On Thu, Aug 11, 2022 at 10:02:29PM +1200, Thomas Munro wrote: > > > Remove configure probe for sys/uio.h. > > > > Removing the configure probe is fi

Re: [PING] fallocate() causes btrfs to never compress postgresql files

2025-06-09 Thread Thomas Munro
905fd90732 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Sat, 31 May 2025 22:50:22 +1200 Subject: [PATCH] Add file_extend_method setting. BTRFS's compression is reported to be disabled by posix_fallocate(), so offer a way to turn it off by setting it to either write or ftruncate instead. M

Re: Non-reproducible AIO failure

2025-06-08 Thread Thomas Munro
On Sat, Jun 7, 2025 at 6:47 AM Andres Freund wrote: > On 2025-06-06 14:03:12 +0300, Konstantin Knizhnik wrote: > > There is really essential difference in code generated by clang 15 (working) > > and 16 (not working). > > There also are code gen differences between upstream clang 17 and apple's >

Re: Update Windows CI Task Names: Server 2022 + VS 2022 Upgrade

2025-06-05 Thread Thomas Munro
On Thu, Jun 5, 2025 at 8:48 PM Nazir Bilal Yavuz wrote: > > Hmm, for the purposes of [0], I think it might be better to keep the > > image at VS 2019 for now. Unless there are specific reasons why VS 2022 > > would be of use now? > > Thomas was thinking of trying

Re: Custom Glibc collation version strings under LOCPATH

2025-06-04 Thread Thomas Munro
On Thu, Jun 5, 2025 at 3:44 AM Joe Conway wrote: > On 6/4/25 09:52, Joe Conway wrote: > > On 6/4/25 00:03, Thomas Munro wrote: > >> I'm interested in hearing about other concrete > >> examples of the locale-recompilation technique failing to be perfect, > >

Re: Non-reproducible AIO failure

2025-06-04 Thread Thomas Munro
On Thu, Jun 5, 2025 at 8:02 AM Andres Freund wrote: > On 2025-06-03 08:00:01 +0300, Alexander Lakhin wrote: > > 2025-06-03 00:19:09.282 EDT [25175:1] LOG: !!!pgaio_io_before_start| ioh: > > 0x104c3e1a0, ioh->op: 1, ioh->state: 1, ioh->result: 0, ioh->num_callbacks: > > 2, ioh->generation: 21694 >

Re: Custom Glibc collation version strings under LOCPATH

2025-06-04 Thread Thomas Munro
:00:00 2001 From: Thomas Munro Date: Wed, 4 Jun 2025 12:19:53 +1200 Subject: [PATCH v2] Load optional collation version from glibc LOCPATH. One technique for dealing with glibc locale definition changes across Linux distribution upgrades or migrations is to compile the locale definitions from

Custom Glibc collation version strings under LOCPATH

2025-06-03 Thread Thomas Munro
y. [1] https://www.mail-archive.com/austin-group-l@opengroup.org/msg12849.html From ab504665cc51814bbe0d8757d35e331fd9b6a41a Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Wed, 4 Jun 2025 12:19:53 +1200 Subject: [PATCH] Load optional collation version from glibc LOCPATH. One technique for dealing

Re: C11 / VS 2019

2025-06-03 Thread Thomas Munro
On Wed, Jun 4, 2025 at 2:02 AM Tom Lane wrote: > Yura Sokolov writes: > > Will it mean we can implement atomics in term of C11 atomics? > > Any such change would have to be supported by extensive performance > testing to verify that there's not a regression on any supported > platform. Yeah, it'

Re: [PING] fallocate() causes btrfs to never compress postgresql files

2025-06-02 Thread Thomas Munro
On Mon, Jun 2, 2025 at 10:14 PM Dimitrios Apostolou wrote: > On Sun, 1 Jun 2025, Thomas Munro wrote: > > Or for a completely different approach: I wonder if ftruncate() would > > be more efficient on COW systems anyway. The minimum thing we need is > > for the file syste

Re: [PING] fallocate() causes btrfs to never compress postgresql files

2025-05-31 Thread Thomas Munro
Or for a completely different approach: I wonder if ftruncate() would be more efficient on COW systems anyway. The minimum thing we need is for the file system to remember the new size, 'cause, erm, we don't. All the rest is probably a waste of cycles, since they reserve real space (or fail to) la

Re: [PING] fallocate() causes btrfs to never compress postgresql files

2025-05-31 Thread Thomas Munro
least allows experimentation. From 8607189eb19302c509eed78a7a2db55b9a2d70b3 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Sat, 31 May 2025 22:50:22 +1200 Subject: [PATCH 1/2] Add io_min_fallocate setting. BTRFS's compression is reported to be disabled by posix_fallocate(), so offer a way to turn it off. The previous c

get speed help

2025-05-29 Thread Dias Thomas
Hi all, if 2 records per second, add speed, 4 gb ram, 1 index, is it faster than postgres?

Re: Non-reproducible AIO failure

2025-05-27 Thread Thomas Munro
On Mon, May 26, 2025 at 12:05 PM Tom Lane wrote: > Thomas Munro writes: > > Could you guys please share your exact repro steps? > > I've just been running 027_stream_regress.pl over and over. > It's not a recommendable answer though because the failure > pro

Re: Automatically sizing the IO worker pool

2025-05-26 Thread Thomas Munro
BTW I would like to push 0001 and 0002 to master/18. They are are not behaviour changes, they just fix up a bunch of inconsistent (0001) and misleading (0002) variable naming and comments to reflect reality (in AIO v1 the postmaster used to assign those I/O worker numbers, now the postmaster has i

Re: Automatically sizing the IO worker pool

2025-05-26 Thread Thomas Munro
On Sun, May 25, 2025 at 7:20 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Sun, Apr 13, 2025 at 04:59:54AM GMT, Thomas Munro wrote: > > It's hard to know how to set io_workers=3. If it's too small, > > io_method=worker's small submission queue over

Re: Non-reproducible AIO failure

2025-05-25 Thread Thomas Munro
On Sun, May 25, 2025 at 3:22 PM Tom Lane wrote: > Thomas Munro writes: > > Can you get a core and print *ioh in the debugger? > > So far, I've failed to get anything useful out of core files > from this failure. The trace goes back no further than > > (lldb) bt

Re: Non-reproducible AIO failure

2025-05-24 Thread Thomas Munro
On Sun, May 25, 2025 at 9:00 AM Alexander Lakhin wrote: > Hello Thomas, > 24.05.2025 14:42, Thomas Munro wrote: > > On Sat, May 24, 2025 at 3:17 PM Tom Lane wrote: > >> So it seems that "very low-probability issue in our Mac AIO code" is > >> the most p

Re: I/O worker and ConfigReload

2025-05-24 Thread Thomas Munro
On Sun, May 25, 2025 at 1:56 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > I've been rebasing the patch for online resizing of shared memory, and > noticed something strange about IoWorkerMain: although it sets the > handler SignalHandlerForConfigReload, it doesn't look like it acts upon > Conf

Re: Non-reproducible AIO failure

2025-05-24 Thread Thomas Munro
On Sat, May 24, 2025 at 3:17 PM Tom Lane wrote: > So it seems that "very low-probability issue in our Mac AIO code" is > the most probable description. There isn't any macOS-specific AIO code so my first guess would be that it might be due to aarch64 weak memory reordering (though Andres speculat

Re: Improve hash join's handling of tuples with null join keys

2025-05-11 Thread Thomas Munro
On Tue, May 6, 2025 at 12:12 PM Tomas Vondra wrote: > On 5/6/25 01:11, Tom Lane wrote: > > The attached patch is a response to the discussion at [1], where > > it emerged that lots of rows with null join keys can send a hash > > join into too-many-batches hell, if they are on the outer side > > of

Re: disabled SSL log_like tests

2025-05-06 Thread Thomas Munro
On Wed, May 7, 2025 at 4:34 PM Tom Lane wrote: > Thanks, I'll look into reporting it tomorrow. In the meantime, > I couldn't help noticing that the backtraces went through > lib/libssl/tls13_legacy.c, which doesn't give a warm feeling > about how supported they think our usage is (and perhaps als

Re: disabled SSL log_like tests

2025-05-06 Thread Thomas Munro
On Wed, May 7, 2025 at 1:18 PM Tom Lane wrote: > So it seems like this might be a simple oversight in > ssl_sigalg_pkey_ok(), which doesn't make any such correction: > > if (sigalg->key_type != EVP_PKEY_id(pkey)) > return 0; Nice detective work. > Anyone know anything abo

Re: disabled SSL log_like tests

2025-05-05 Thread Thomas Munro
If you run the not-yet-enabled-by-default OpenBSD CI task on master, ssl/001_ssltests fails in "intermediate client certificate is untrusted", recently uncommented by commit e0f373ee. I think it might be telling us that LibreSSL's x509_store_ctx_get_current_cert() is giving us the client certifica

Re: Adding pg_dump flag for parallel export to pipes

2025-04-25 Thread Thomas Munro
On Tue, Apr 8, 2025 at 7:48 AM Hannu Krosing wrote: > Just to bring this out separately : Does anybody have any idea why pipe > commands close inside tests ? > > Re: 003-pg_dump_basic_tests has a few basic validation tests for > correctmflag combinations. We need to write more automated tests in

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Thomas Munro
On Sat, Apr 26, 2025 at 5:43 AM Tom Lane wrote: > Andres Freund writes: > > It's kinda sad to not have any test that tests a larger > > io_combine_limit/io_max_combine_limit - as evidenced by this bug that'd be > > good. However, not all platforms have PG_IOV_MAX > 16, so it seems like it'd > > b

Re: AIX support

2025-04-24 Thread Thomas Munro
On Mon, Apr 7, 2025 at 10:04 PM Heikki Linnakangas wrote: > I'm surprised how big the difference is, because I actually expected the > compiler to detect the memory-zeroing loop and replace it with some > fancy vector instructions (does powerpc have any?). It certainly does, and we've played with

Re: Changing shared_buffers without restart

2025-04-21 Thread Thomas Munro
On Mon, Apr 21, 2025 at 9:30 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > Yeah, that would work and will allow to avoid MAP_FIXED and mremap, which are > questionable from portability point of view. This leaves memfd_create, and I'm > still not completely clear on it's portability -- it seems

Re: Changing shared_buffers without restart

2025-04-18 Thread Thomas Munro
On Thu, Nov 21, 2024 at 8:55 PM Peter Eisentraut wrote: > On 19.11.24 14:29, Dmitry Dolgov wrote: > >> I see that memfd_create() has a MFD_HUGETLB flag. It's not very clear how > >> that interacts with the MAP_HUGETLB flag for mmap(). Do you need to > >> specify > >> both of them if you want hu

Re: Changing shared_buffers without restart

2025-04-18 Thread Thomas Munro
On Fri, Apr 18, 2025 at 9:17 PM Thomas Munro wrote: > On Fri, Apr 18, 2025 at 7:25 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Thanks for sharing. I need to do more thorough tests, but after a quick > > look I'm not sure about that. ftruncate will take care

Re: Changing shared_buffers without restart

2025-04-18 Thread Thomas Munro
On Fri, Apr 18, 2025 at 7:25 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > On Thu, Apr 17, 2025 at 03:22:28PM GMT, Ashutosh Bapat wrote: > > > > In an offlist chat Thomas Munro mentioned that just ftruncate() would > > be enough to resize the shared memory

Re: Changing shared_buffers without restart

2025-04-17 Thread Thomas Munro
On Fri, Apr 18, 2025 at 3:54 AM Thomas Munro wrote: > I contemplated that once before, when I wrote a quick demo patch[1] to > implement huge_pages=on for FreeBSD (ie explicit rather than > transparent). I used a different function, not the Linuxoid one but Oops, I forgot to supply th

Re: NUMA shared memory interleaving

2025-04-16 Thread Thomas Munro
On Thu, Apr 17, 2025 at 1:58 AM Thomas Munro wrote: > I have no answers but I have speculated for years about a very > specific case (without any idea where to begin due to lack of ... I > guess all this sort of stuff): in ExecParallelHashJoinNewBatch(), > workers split up and tr

Re: NUMA shared memory interleaving

2025-04-16 Thread Thomas Munro
On Wed, Apr 16, 2025 at 9:14 PM Jakub Wartak wrote: > 2. Should we also interleave DSA/DSM for Parallel Query? (I'm not an > expert on DSA/DSM at all) I have no answers but I have speculated for years about a very specific case (without any idea where to begin due to lack of ... I guess all this

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-04-15 Thread Thomas Munro
On Tue, Apr 15, 2025 at 5:44 AM Robert Haas wrote: > On Thu, Apr 10, 2025 at 11:15 PM Thomas Munro wrote: > > The new streaming BHS isn't just issuing probabilistic hints about > > future access obtained from a second iterator. It has just one shared > > iterator c

Using pg_bitutils.h in tidbitmap.c.

2025-04-14 Thread Thomas Munro
tidbitmap.c's operations loop over all the bits, but could leap over zeros with bitmap instructions like bitmapset.c. Hard to imagine that matters, but I think it also comes out easier to read? From 7eec670224270c098c2fb3ad4b7748b0769a1e95 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date

Re: Automatically sizing the IO worker pool

2025-04-13 Thread Thomas Munro
On Mon, Apr 14, 2025 at 5:45 AM Jose Luis Tallon wrote: > On 12/4/25 18:59, Thomas Munro wrote: > > It's hard to know how to set io_workers=3. > > Hmmm enable the below behaviour if "io_workers=auto" (default) ? Why not just delete io_workers? If you really wa

Automatically sizing the IO worker pool

2025-04-12 Thread Thomas Munro
independent of this topic, and might be potential cleanups/fixes for master/v18. The last is a simple latency test. Ideas, testing, flames etc welcome. From 1dbba36f67df5d3d34a990613d6d68d15caf1b17 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Sat, 29 Mar 2025 13:25:27 +1300 Subject: [PATCH

Re: ci: Allow running mingw tests by default via environment variable

2025-04-12 Thread Thomas Munro
; > > not just master. It's kind of annoying to have CI infrastructure changes > > > like > > > this that differ between branches. Thoughts? > > > > I am sharing patches per branch until we come up with a better > > solution. Some remarks: > > &

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-04-10 Thread Thomas Munro
On Fri, Apr 11, 2025 at 5:50 AM James Hunter wrote: > I am looking at the pre-streaming code, in PG 17, as I am not familiar > with the PG 18 "streaming" code. Back in PG 17, nodeBitmapHeapscan.c > maintained two shared TBM iterators, for PQ. One of the iterators was > the actual, "fetch" iterator

Re: Streaming relation data out of order

2025-04-10 Thread Thomas Munro
uld need. */ queue_overflow = io_combine_limit - 1; The patch clearly lacks a comment to explain that mystery. Will add. Thanks! > I think all these variables really ought to be unsigned. None of them ever > could be negative afaict. Maybe... I tend to default to signed

Re: BAS_BULKREAD vs read stream

2025-04-10 Thread Thomas Munro
On Tue, Apr 8, 2025 at 2:20 PM Andres Freund wrote: > In the attached I implemented the above idea, with some small additional > refinements: LGTM. How I wish EXPLAIN would show some clues about strategies...

Re: 64 bit numbers vs format strings

2025-04-10 Thread Thomas Munro
On Sun, Mar 30, 2025 at 6:24 AM Peter Eisentraut wrote: > I have committed v2-0001, omitting the parts that I had flagged in my > review. I have also committed v2-0002. From my perspective, this can > conclude this thread. Thank you! Fingers crossed that the translation updates go as smoothly

Re: Cannot find a working 64-bit integer type on Illumos

2025-04-10 Thread Thomas Munro
On Wed, Mar 26, 2025 at 12:36 PM Tom Lane wrote: > Daniel Gustafsson writes: > >> On 26 Mar 2025, at 00:01, Tom Lane wrote: > >> How did that work before? Perhaps somebody just now added a libpq > >> dependency to pg_regress.c? > > > I believe the libpq dependency came in 66d6086cbcbfc8 which w

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-04-09 Thread Thomas Munro
On Wed, Apr 9, 2025 at 1:46 PM James Hunter wrote: > On Mon, Apr 7, 2025 at 7:34 PM Thomas Munro wrote: > > On Thu, Feb 13, 2025 at 1:40 PM Melanie Plageman > > wrote: > > > Thomas mentioned this to me off-list, and I think he's right. We > > > likely n

Re: Streaming relation data out of order

2025-04-09 Thread Thomas Munro
v18, but it's why I used relblocknumber % 3 != 2 and not eg relblocknumber % 2 != 1 above. From 3b460cb102c8af49a0af0d3b08bf86a436564fd3 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Sat, 15 Feb 2025 14:47:25 +1300 Subject: [PATCH v2 1/4] Move read stream modulo arithmetic into functions

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-04-08 Thread Thomas Munro
On Thu, Feb 13, 2025 at 1:40 PM Melanie Plageman wrote: > Thomas mentioned this to me off-list, and I think he's right. We > likely need to rethink the way parallel bitmap heap scan workers get > block assignments for reading and prefetching to make it more similar > to paralle

Re: CREATE DATABASE with filesystem cloning

2025-04-08 Thread Thomas Munro
On Wed, Mar 26, 2025 at 12:47 AM Nazir Bilal Yavuz wrote: > Rebased version of the patch is attached. I noticed that it wasn't working on a Mac I tested because guc_tables.c needed to include and fixed that. I also did some minor editig of the docs, mostly removing the mention of specific file

Re: Some read stream improvements

2025-04-06 Thread Thomas Munro
On Mon, Apr 7, 2025 at 1:29 PM Andres Freund wrote: > On 2025-04-03 14:43:40 +1300, Thomas Munro wrote: > > On Thu, Apr 3, 2025 at 11:17 AM Andres Freund wrote: > > > I don't actually see any reason for buffer_limit to be a 16bit quantity? > > > It's &

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-04 Thread Thomas Munro
On Thu, Mar 20, 2025 at 2:38 AM Daniel Gustafsson wrote: > > On 19 Mar 2025, at 05:57, Tom Lane wrote: > > > > BTW, I was pretty seriously disheartened just now to realize that > > this feature was implemented by making libpq depend on libcurl. > > I'd misread the relevant commit messages to say

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-04-04 Thread Thomas Munro
On Thu, Mar 20, 2025 at 12:06 PM Andres Freund wrote: > Ah - it effectively is already in a critical section, just a weirdly spelled > one: > > 2025-03-19 19:00:06.398 EDT [2156613][client backend][0/3:0][psql] LOG: > statement: DROP TABLE foo; > 2025-03-19 19:00:06.404 EDT [2156613][client bac

Re: Some read stream improvements

2025-04-02 Thread Thomas Munro
On Thu, Apr 3, 2025 at 11:17 AM Andres Freund wrote: > I don't actually see any reason for buffer_limit to be a 16bit quantity? It's > just to clamp things down, right? Ugh. It might be worth just flipping this whole thing over to ints, let me look into that...

Re: Some read stream improvements

2025-03-30 Thread Thomas Munro
On Tue, Mar 18, 2025 at 5:56 AM Andres Freund wrote: > So one thing is that the pin count differs by 1 at the start of the scan. No > idea why. > > I still don't know what drives the difference between freebsd and the rest, > but IIUC the reason this fails is just that we are holding too many buff

Re: AIO v2.5

2025-03-26 Thread Thomas Munro
On Thu, Mar 27, 2025 at 10:41 AM Andres Freund wrote: > > > Subject: [PATCH v2.12 13/28] Enable IO concurrency on all systems > > > > Consider also updating this comment to stop focusing on prefetch; I think > > changing that aligns with the patch's other changes: > > > > /* > > * How many buffer

Re: Cannot find a working 64-bit integer type on Illumos

2025-03-25 Thread Thomas Munro
On Wed, Mar 26, 2025 at 10:34 AM Nathan Bossart wrote: > anaconda seems to be upset about this one [0]. I've spent all of 30 > seconds looking at it so far, but it appears to be using an old version of > the header file. > > In file included from pg_regress.c:34: > /usr/local/include/libpq-fe.h:6

Re: Cannot find a working 64-bit integer type on Illumos

2025-03-25 Thread Thomas Munro
On Wed, Mar 5, 2025 at 6:00 AM Peter Eisentraut wrote: > I agree with your patch 0001-Deprecate-pg_int64.patch. I don't see a > reason to keep the current arrangement around pg_int64. Thanks for looking! Pushed.

Re: AIO v2.5

2025-03-25 Thread Thomas Munro
On Tue, Mar 25, 2025 at 2:18 PM Andres Freund wrote: > Attached v2.12, with the following changes: Here's a tiny fixup to make io_concurrency=0 turn on READ_BUFFERS_SYNCHRONOUSLY as mooted in a FIXME. Without this, AIO will still run at level 1 even if you asked for 0. Feel free to squash, or i

Re: AIO v2.5

2025-03-24 Thread Thomas Munro
On Tue, Mar 25, 2025 at 11:55 AM Andres Freund wrote: > If a callback may sometimes need to block, it can still opt into > READ_STREAM_USE_BATCHING, by submitting all staged IO before blocking. > > The hardest part is to explain the flag. Here's my current attempt: > > /* --- > * Opt-in to using

Re: AIO v2.5

2025-03-23 Thread Thomas Munro
On Mon, Mar 24, 2025 at 5:59 AM Andres Freund wrote: > On 2025-03-23 08:55:29 -0700, Noah Misch wrote: > > An IO in PGAIO_HS_STAGED clearly blocks closing the IO's FD, and an IO in > > PGAIO_HS_COMPLETED_IO clearly doesn't block that close. For > > io_method=worker, > > closing in PGAIO_HS_SUBMI

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-19 Thread Thomas Munro
On Thu, Mar 20, 2025 at 11:02 AM Thomas Munro wrote: > On Thu, Mar 20, 2025 at 10:04 AM Tom Lane wrote: > > Thomas Munro writes: > > > How feasible/fragile/weird would it be to dlopen() it on demand? . o O { There may also be security reasons to reject the idea, would need to look into that... }

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-19 Thread Thomas Munro
On Thu, Mar 20, 2025 at 11:19 AM Tom Lane wrote: > Thomas Munro writes: > > It would increase the build dependencies, assuming a package > > maintainer wants to enable as many features as possible, but it would > > *not* increase the 'package requires' footprint, m

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-19 Thread Thomas Munro
On Thu, Mar 20, 2025 at 10:04 AM Tom Lane wrote: > Thomas Munro writes: > > How feasible/fragile/weird would it be to dlopen() it on demand? > > FWIW, that would not really move the needle one bit so far as > my worries are concerned. What I'm unhappy about is the very

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-18 Thread Thomas Munro
On Wed, Mar 19, 2025 at 5:17 PM Tom Lane wrote: > fe-auth-oauth-curl.c:1392:49: error: data argument not used by format string > [-Werror,-Wformat-extra-args] > 1392 | actx_error(actx, "deleting kqueue timer: %m", > timeout); > | ~~

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-18 Thread Thomas Munro
hat I would :-)). > Thomas has shown me a side quest to get rid of the second > kqueue instance, but so far that is not bearing fruit and we shouldn't > wait on it. Cool, thanks for looking into it anyway. (Feel free to post a nonworking patch with a got-stuck-here-because problem statement...)

Re: Allow io_combine_limit up to 1MB

2025-03-17 Thread Thomas Munro
they could just use PG_IOV_MAX instead; it's perhaps a little less clear as names go, but it's also pretty confusing that there's a macro with the same name as a GUC... From 1200802f3d647a8c9107b2148595dab4b6a1bd82 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Wed, 12 Feb 2025 13:

Re: Snapshot related assert failure on skink

2025-03-17 Thread Thomas Munro
On Tue, Mar 18, 2025 at 12:59 AM Tomas Vondra wrote: > On 3/17/25 12:36, Tomas Vondra wrote: > > I'm still fiddling with the script, trying to increase the probability > > of the (apparent) race condition. On one machine (old Xeon) I can hit it > > very easily/reliably, while on a different machin

Re: 64 bit numbers vs format strings

2025-03-17 Thread Thomas Munro
On Mon, Mar 17, 2025 at 11:52 PM Thomas Munro wrote: > tmunro@s11-sparc:~/gettext-hacking$ gcc test.c > tmunro@s11-sparc:~/gettext-hacking$ ./a.out > la réponse est 42 And just to be paranoid, I checked a few more things: the .mo definitely contains the literal "PRId64" (rear

Re: 64 bit numbers vs format strings

2025-03-17 Thread Thomas Munro
On Mon, Mar 17, 2025 at 8:03 PM Peter Eisentraut wrote: > On 15.03.25 03:42, Thomas Munro wrote: > > So we should make pgoff_t a typedef for int64_t everywhere. It's a > > bit annoying that we have to teach everyone to remember to use PRId64 > > to print it, though. >

Re: 64 bit numbers vs format strings

2025-03-17 Thread Thomas Munro
On Mon, Mar 17, 2025 at 8:09 PM Tom Lane wrote: > Peter Eisentraut writes: > > This is not really possible. The behavior is baked deeply into > > the gettext code. (Also note that you don't only need support in > > xgettext, which is part of our build system, but also in the runtime > > librar

Re: Available disk space per tablespace

2025-03-15 Thread Thomas Munro
On Sun, Mar 16, 2025 at 1:09 AM Christoph Berg wrote: > Hmm. Is looping on EINTR worth the trouble? I was just wondering if it might be one of those oddballs that ignores SA_RESTART, but I guess that doesn't seem too likely (I mean, first you'd probably have to have a reason to sleep or some othe

Re: Available disk space per tablespace

2025-03-15 Thread Thomas Munro
On Sat, Mar 15, 2025 at 4:40 AM Christoph Berg wrote: > I'm still unconvinced if we should use statfs() instead of statvfs() > on *BSD or if their manpage is just trolling us and statvfs is just > fine. > > DESCRIPTION >The statvfs() and fstatvfs() functions fill the structure pointed to b

Re: Available disk space per tablespace

2025-03-15 Thread Thomas Munro
On Sun, Mar 16, 2025 at 1:17 AM Laurenz Albe wrote: > On Sat, 2025-03-15 at 13:09 +0100, Christoph Berg wrote: > > Do we care about any of these? > > > > AIX > > We dropped support for it, but there are efforts to change that. FWIW AIX does have it, according to its manual, in case it comes back.

Re: 64 bit numbers vs format strings

2025-03-14 Thread Thomas Munro
On Mon, Mar 10, 2025 at 10:49 PM Peter Eisentraut wrote: > On 02.03.25 22:08, Thomas Munro wrote: > > Good plan, thanks. Here's a rebase. > > I think this went ok, and we can proceed here. Cool, I'll post a new patch soon, but first a question about this bit: > -

Re: Some read stream improvements

2025-03-14 Thread Thomas Munro
I have pushed the new pin limit patches, after some more testing and copy editing. I dropped an unnecessary hunk (in read_stream_reset(), a change I'd like to make but it didn't belong in this commit) and dropped the word "Soft" from GetSoftPinLimit() as it wasn't helping comprehension and isn't ev

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-13 Thread Thomas Munro
On Fri, Mar 14, 2025 at 12:23 PM Andres Freund wrote: > On 2025-03-14 11:31:47 +1300, Thomas Munro wrote: > > 2. Somehow tell elog.c not to call CHECK_FOR_INTERRUPTS() instead. > > Also yuck, but at least elog.c is already the right place to clean > > state up on non-local

Re: md.c vs elog.c vs smgrreleaseall() in barrier

2025-03-13 Thread Thomas Munro
On Fri, Mar 14, 2025 at 5:03 AM Andres Freund wrote: > If there is any LOG/DEBUG elog inside functions like mdreadv(), mdwritev(), > mdextend(), be it directly or indirectly, the functions become unsafe. The > problem is that at the end of errfinish() there is a CFI: > > /* > * Ch

Re: Some read stream improvements

2025-03-13 Thread Thomas Munro
On Wed, Mar 12, 2025 at 8:29 AM Andres Freund wrote: > On 2025-03-12 07:35:46 +1300, Thomas Munro wrote: > > On Thu, Feb 27, 2025 at 11:20 PM Andres Freund wrote: > > > On 2025-02-27 11:19:55 +1300, Thomas Munro wrote: > > I wonder if we should use temp_buffers - 100? Th

Re: Some read stream improvements

2025-03-11 Thread Thomas Munro
On Thu, Feb 27, 2025 at 11:20 PM Andres Freund wrote: > On 2025-02-27 11:19:55 +1300, Thomas Munro wrote: > > On Wed, Feb 26, 2025 at 10:55 PM Andres Freund wrote: > > > I was working on expanding tests for AIO and as part of that wrote a test > > > for > >

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-07 Thread Thomas Munro
On Sat, Mar 8, 2025 at 6:31 AM Jacob Champion wrote: > On Thu, Mar 6, 2025 at 9:13 PM Thomas Munro wrote: > > I don't see that behaviour on my Mac with a simple program, and that > > seems like it couldn't possibly be intended. > > What version of macOS? > &

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-06 Thread Thomas Munro
On Fri, Mar 7, 2025 at 9:57 AM Jacob Champion wrote: > 2) macOS's EVFILT_TIMER implementation seems to be different from the > other BSDs. On Mac, when you re-add a timer to a kqueue, any existing > timer-fired events for it are not cleared out and the kqueue might > remain readable. This breaks a

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-04 Thread Thomas Munro
(why passing sometimes then?) but I'm sure all will become clear with your patch... > On Mon, Mar 3, 2025 at 8:11 PM Thomas Munro wrote: > > I think that is telling us that a non-blocking socket can be in a > > state that is not yet connected enough even to tell you its local >

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-03-03 Thread Thomas Munro
On Tue, Mar 4, 2025 at 1:07 PM Jacob Champion wrote: > # [libcurl] * getsockname() failed with errno 22: Invalid argument Weird. > Later, Curl reconnects via IPv6 -- this time succeeding -- but then > the response gets mangled in some way. I assume headers are being > truncated, based on Cur

Re: ci: Allow running mingw tests by default via environment variable

2025-03-03 Thread Thomas Munro
tically using an environment variable defined under "Repository Settings", for example: REPO_CI_AUTOMATIC_TRIGGER_TASKS="mingw netbsd openbsd" This will enable cfbot to turn them on by default when running tests for the Commitfest app. It is not subject to fre

Re: Adding NetBSD and OpenBSD to Postgres CI

2025-03-03 Thread Thomas Munro
On Tue, Mar 4, 2025 at 11:39 AM Andres Freund wrote: > On 2025-03-04 11:35:30 +1300, Thomas Munro wrote: > > I propose to turn these on by default tomorrow, unless someone speaks > > up with reasons why we shouldn't. > I'm worried about enabling them by default eating

Re: Adding NetBSD and OpenBSD to Postgres CI

2025-03-03 Thread Thomas Munro
On Sun, Mar 2, 2025 at 1:37 AM Thomas Munro wrote: > On Sat, Mar 1, 2025 at 6:24 AM Nazir Bilal Yavuz wrote: > > I think I found the problem, sd0j's fstype is not a swap. It worked like > > that: > > > > $ disklabel -E sd0 > > $ umount /usr/obj > > $

Re: Flaky 003_start_stop.pl test

2025-03-03 Thread Thomas Munro
On Tue, Mar 4, 2025 at 5:48 AM Andres Freund wrote: > On 2025-02-24 11:26:56 +0100, Jelte Fennema-Nio wrote: > > [1]: https://cirrus-ci.com/task/5571017969500160?logs=test_world#L256 > > A possibly related test failure is: > > https://cirrus-ci.com/task/5005337046024192?logs=check_world#L259 They

Re: Allow io_combine_limit up to 1MB

2025-03-03 Thread Thomas Munro
On Wed, Feb 12, 2025 at 3:24 PM Thomas Munro wrote: > On Wed, Feb 12, 2025 at 3:22 PM Andres Freund wrote: > > On 2025-02-12 13:59:21 +1300, Thomas Munro wrote: > > > How about just maintaining it in a new variable > > > effective_io_combine_limit, whenever

Re: Cannot find a working 64-bit integer type on Illumos

2025-03-02 Thread Thomas Munro
On Tue, Dec 10, 2024 at 3:02 PM Thomas Munro wrote: > On Thu, Dec 5, 2024 at 12:16 PM Tom Lane wrote: > > Now you already snuck the camel's nose under the > > tent by including stdint.h there, and maybe these additional headers > > wouldn't do any further damag

Re: Adding NetBSD and OpenBSD to Postgres CI

2025-03-01 Thread Thomas Munro
://cirrus-ci.com/build/5275349266726912 From 68232b16632fcc9469017ec219308c798e0d5391 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Fri, 28 Feb 2025 22:19:07 +1300 Subject: [PATCH v2 1/2] ci: Use a RAM disk for NetBSD and OpenBSD. Put the RAM disk setup for all three *BSD CI tasks into a common script, replacing the o

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-28 Thread Thomas Munro
On Sat, Mar 1, 2025 at 10:57 AM Jacob Champion wrote: > After your patch gets us past the zero timeout bug, NetBSD next runs into > > failed to fetch OpenID discovery document: Unsupported protocol > (libcurl: Received HTTP/0.9 when not allowed)' > > ...but only for a single test (nonempty oau

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-28 Thread Thomas Munro
On Sat, Mar 1, 2025 at 6:37 AM Jacob Champion wrote: > Should we maybe consider just doing that across the board, and put up > with the inefficiency? Admittedly 1ms is a lot more dead time than > 1ns... Last time I checked, NetBSD is still using scheduler ticks (100hz periodic interrupt) for all

Re: Adding NetBSD and OpenBSD to Postgres CI

2025-02-28 Thread Thomas Munro
On Tue, Feb 18, 2025 at 11:33 AM Thomas Munro wrote: > Maybe we could try this? > > https://man.netbsd.org/mount_tmpfs.8 > https://man.openbsd.org/mount_tmpfs.8 NetBSD's test_world: 10:30 -> 3:23 OpenBSD test_world: 15:45 - >9:10 I think NetBSD would finish around 2n

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-02-28 Thread Thomas Munro
he additional * code complexity is worth it. [1] https://cirrus-ci.com/task/6354435774873600 [2] https://github.com/NetBSD/src/blob/67c7c4658e77aa4534b6aac8c041d77097c5e722/sys/kern/kern_event.c#L1375 From 7deb153caf552c9bcb380f88eddbca94be33a0c2 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date:

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-27 Thread Thomas Munro
On Fri, Feb 28, 2025 at 2:29 PM Thomas Munro wrote: > On Fri, Feb 28, 2025 at 11:58 AM Melanie Plageman > wrote: > > On Thu, Feb 27, 2025 at 1:08 PM Tom Lane wrote: > > > I wonder if it'd be a good idea to add something like > > > > > >

Re: Confine vacuum skip logic to lazy_scan_skip

2025-02-27 Thread Thomas Munro
errors, i.e. passing > > a per_buffer_data pointer when there's no per-buffer data. > > I think this is a good stopgap. I was discussing adding this assert > off-list with Thomas and he wanted to detail his more ambitious plans > for type safety improvements in the read stream

Re: Some read stream improvements

2025-02-27 Thread Thomas Munro
On Thu, Feb 27, 2025 at 11:19 AM Thomas Munro wrote: > On Wed, Feb 26, 2025 at 10:55 PM Andres Freund wrote: > > I was working on expanding tests for AIO and as part of that wrote a test > > for > > temp tables -- our coverage is fairly awful, there were many

Re: Some read stream improvements

2025-02-26 Thread Thomas Munro
On Wed, Feb 26, 2025 at 10:55 PM Andres Freund wrote: > I was working on expanding tests for AIO and as part of that wrote a test for > temp tables -- our coverage is fairly awful, there were many times during AIO > development where I knew I had trivially reachable temp table specific bugs > but

Re: MAX_BACKENDS size (comment accuracy)

2025-02-22 Thread Thomas Munro
On Sun, Feb 23, 2025 at 4:16 AM Andres Freund wrote: > We do count the number of lwlock share lockers and the number of buffer > refcounts within those bits. And obviously 0 lockers/refcounts has to be > valid. So I think the limit is correct? Ah, right. That makes perfect sense. The 18 bits ne

Re: MAX_BACKENDS size (comment accuracy)

2025-02-21 Thread Thomas Munro
On Sat, Feb 22, 2025 at 7:27 AM Andres Freund wrote: +#define MAX_BACKENDS_BITS18 +#define MAX_BACKENDS((1 << MAX_BACKENDS_BITS)-1) +1 for working forwards from the bits. But why not call it PROC_NUMBER_BITS? After 024c5211 and 024c5211^, the ID for backends is a ProcNumber,

  1   2   3   4   5   6   7   8   9   10   >