Re: Non-reproducible AIO failure

2025-06-10 Thread Andres Freund
Hi, On 2025-06-10 21:09:18 +0300, Konstantin Knizhnik wrote: > > On 10/06/2025 8:41 pm, Andres Freund wrote: > > I was able to reproduce it with gcc, too. > > I've reproduced it without that bitfield, unfortunately :(. > But also only at MacOS? Correct. > I wo

Re: Non-reproducible AIO failure

2025-06-10 Thread Andres Freund
Hi, On 2025-06-10 17:28:11 +0300, Konstantin Knizhnik wrote: > On 09/06/2025 2:05 am, Thomas Munro wrote: > > On Sat, Jun 7, 2025 at 6:47 AM Andres Freund wrote: > > > On 2025-06-06 14:03:12 +0300, Konstantin Knizhnik wrote: > > > > There is really essential differe

Re: Cleaning up historical portability baggage

2025-06-09 Thread Andres Freund
is that if it doesn't exist on the BF, it's unsupported. Also note that Hurd is not listed as a supported OS: https://www.postgresql.org/docs/devel/supported-platforms.html We can't design for OS that we don't know it's used with postgres and/or how that OS works / what it supports. So I reject the premise that this is a regression. If you want to argue that we should add support for Hurd, you can do that, but that's obviously a different discussion. Greetings, Andres Freund

Re: Non-reproducible AIO failure

2025-06-08 Thread Andres Freund
Hi, On 2025-06-06 15:37:45 -0400, Andres Freund wrote: > There shouldn't be any concurrent accesses here, so I don't really see how the > above would explain the problem (the IO can only ever be modified by one > backend, initially the "owning backend", then, when

Re: md5 password deprecation might cause problems with PgBouncer setups

2025-06-06 Thread Andres Freund
ple PgBouncer instances. Which leads me to wonder if what pgbouncer should do is to recommend a different number of scram iterations? That seems better than recommending md5 or password auth. Greetings, Andres Freund [1] https://datatracker.ietf.org/doc/html/rfc7677#section-4

Re: Non-reproducible AIO failure

2025-06-06 Thread Andres Freund
his struct, we have > to use primitive int types not bit-fields. I think we have to do that, if for no other reason than how bad the generated code is. But I would still like to understand what actually causes the problem here - I'm far from sure it's actually the bitfield. Greetings, Andres Freund

Re: Non-reproducible AIO failure

2025-06-06 Thread Andres Freund
e > at aio.c:384 > ``` > As you can see pg16 just updates third byte of the structure (`op` field). > It is not affecting 'state' field at all. It's really weird code. I think the state store would be in the call to pgaio_io_update_state(). > While clang16 is also updating `state` field! I assume this was intending to refer to clang15? FWIW, I don't think it does update the state field in the assembly you shared. This is very odd code, but I don't immediately see a bug in it :( Greetings, Andres Freund

Re: Unnecessary connection overhead due copy-on-write (mainly openssl)

2025-06-06 Thread Andres Freund
ashes, e.g. the startup packet timeout is also handled by _exit() - and it can be triggered remotely. ISTM that if crypto providers can't handle _exit(), we have a bigger problem. Alternatively we could try deferring more of openssl's initialization to outside of postmaster - but that doesn't seem particularly realistic. Greetings, Andres Freund

Re: Retiring some encodings?

2025-06-05 Thread Andres Freund
on path to UTF-8 it's really not practically usable IMO. Greetings, Andres Freund

We should lazy-initialize the deadlock checker state memory

2025-06-05 Thread Andres Freund
's obviously trivial. Greetings, Andres Freund [1] https://postgr.es/m/hgs2vs74tzxigf2xqosez7rpf3ia5e7izalg5gz3lv3nqfptxx%40thanmprbpl4e

Unnecessary connection overhead due copy-on-write (mainly openssl)

2025-06-05 Thread Andres Freund
enssl I see a significant improvement, just due fewer mmaps for the libraries: -> 4865 TPS Further disabling openssl and zlib interestingly does not help, interestingly. Greetings, Andres Freund [1] https://postgr.es/m/CAFbpF8OA44_UG%2BRYJcWH9WjF7E3GA6gka3gvH6nsrSnEe9H0NA%40mail.gmail.com [2] https://docs.openssl.org/3.1/man3/OPENSSL_init_crypto/#name

Re: postmaster uses more CPU in 18 beta1 with io_method=io_uring

2025-06-05 Thread Andres Freund
Hi, On 2025-06-05 12:47:52 -0400, Tom Lane wrote: > Andres Freund writes: > > I think this is a big enough pitfall that it's, obviously assuming the patch > > has a sensible complexity, worth fixing this in 18. RMT, anyone, what do you > > think? > > Let's s

Re: postmaster uses more CPU in 18 beta1 with io_method=io_uring

2025-06-05 Thread Andres Freund
ed. I think this is a big enough pitfall that it's, obviously assuming the patch has a sensible complexity, worth fixing this in 18. RMT, anyone, what do you think? Greetings, Andres Freund [1] https://fosstodon.org/@axboe/114630982449670090 [2] https://pastebin.com/7M3C8aFH

Re: Non-reproducible AIO failure

2025-06-04 Thread Andres Freund
latform? IME the theory that there is something outside of a subsystem randomly overwriting memory is almost always wrong, but I'm kinda starting to suspect it here nonetheless. I wonder if it's worth trying to put a fair bit of "padding" memory before/after AIO's shared memory region(s). If that were to stop the crashes, it'd likely be something random writing there. Unfortunately continuing to crash wouldn't necessarily guarantee it's aio specific. Hm. We had issues with wrong size calculations of shmem regions before, what about writing a patch that puts mprotect(PROT_NONE) pages before/after each ShmemAlloc'd region? Greetings, Andres Freund

Re: gcc 15 "array subscript 0" warning at level -O3

2025-06-04 Thread Andres Freund
Hi, On 2025-04-25 15:58:29 -0400, Andres Freund wrote: > On 2025-04-25 13:37:15 -0400, Tom Lane wrote: > > Buildfarm member serinus has been producing the identical warning for > > some time. I'd been ignoring that because it runs "experimental gcc", > > but I

Re: ABI Compliance Checker GSoC Project

2025-06-04 Thread Andres Freund
age/read_stream.h > > 50: * the ReadStreamBlockNumberCB callback to abide by the restrictions of AIO > 66:struct ReadStream; > 67:typedef struct ReadStream ReadStream; No. It just makes the *name* of the struct visible. The type's definition is in the .c file and therefore not visible outside of read_stream.c. Greetings, Andres Freund

Re: SpinLockAcquire and SpinLockRelease is broken on ARM/ARM64? (Re: sinvaladt.c: remove msgnumLock, use atomic operations on maxMsgNum)

2025-06-03 Thread Andres Freund
heaper to release on common platforms). Greetings, Andres Freund

Re: autoprewarm_dump_now

2025-06-03 Thread Andres Freund
? No, seems like a thing that pretty obviously should be fixed. Greetings, Andres Freund

Re: Persist injection points across server restarts

2025-06-03 Thread Andres Freund
equire injection points before consistency also can't wait for injection points using the SQL interface or such, so most of the stuff has to be written in C anyway. And if so, you also can attach to injection points in the relevant shared_preload_libraries entry. Greetings, Andres Freund

Re: CHECKPOINT unlogged data

2025-05-30 Thread Andres Freund
On 2025-05-30 19:23:04 +0200, Christoph Berg wrote: > Re: Nathan Bossart > > This patch also adds an IMMEDIATE option, which I proposed some time ago > > [0]. I ended up withdrawing it due to general skepticism about its > > Thanks for the pointer, I did not go that far back when looking for > ol

Re: CHECKPOINT unlogged data

2025-05-30 Thread Andres Freund
Hi, On May 30, 2025 12:55:28 PM EDT, Christoph Berg wrote: >Re: Andres Freund >> Hi, >> >> On 2025-05-30 18:17:45 +0200, Christoph Berg wrote: >> > A customer reported to use CHECKPOINT before shutdowns to make the >> > shutdowns themselves fast

Re: CHECKPOINT unlogged data

2025-05-30 Thread Andres Freund
e idea makes sense, so there's the patch. I've seen the need for this quite a few times, fwiw. So +1 to the idea. Greetings, Andres Freund

Re: fix: propagate M4 env variable to flex subprocess

2025-05-28 Thread Andres Freund
Hi, On 2025-05-17 23:32:24 -0400, J. Javier Maestro wrote: > On Tue, May 13, 2025 at 11:54 AM Andres Freund wrote: > > Bilal, I think you wrote this originally, do you recall? > > > > It seems like an issue beyond just M4... > > > > IIRC the rest of the tools i

Re: wrong query results on bf leafhopper

2025-05-28 Thread Andres Freund
n tests passed. The failure on indri is rather different than your issue, I doubt there's an overlap between the problems... I think we should spruce up 027_stream_regress.pl a bit around this. Before the "regression tests pass" check we should a) check if primary is still alive b) check if standby is still alive and then, iff a) & b) pass, in addition to printing the entire regression test file, we should add the head and tail of regression.diffs to the failure message, so one can quickly glean what went wrong. Greetings, Andres Freund

Re: Non-reproducible AIO failure

2025-05-27 Thread Andres Freund
Hi, On 2025-05-27 14:43:14 -0400, Tom Lane wrote: > Andres Freund writes: > > I just meant that it seems that I can't reproduce it for some as of yet > > unknown reason. I've now been through 3k+ runs of 027_stream_regress, > > without > > a single f

Re: Non-reproducible AIO failure

2025-05-27 Thread Andres Freund
Hi, On 2025-05-27 10:12:28 -0400, Tom Lane wrote: > Andres Freund writes: > > This is on a m4 mac mini. I'm wondering if there's some hardware specific > > memory ordering issue or disk speed based timing issue that I'm just not > > hitting. > >

Re: Non-reproducible AIO failure

2025-05-27 Thread Andres Freund
m4 mac mini. I'm wondering if there's some hardware specific memory ordering issue or disk speed based timing issue that I'm just not hitting. Greetings, Andres Freund

Re: Understanding when VM record needs snapshot conflict horizon

2025-05-23 Thread Andres Freund
will only contain > an update to the VM and I'm trying to determine when I need a snapshot > conflict horizon in the record. You need to include it if the replay of the record might invalidate existing snapshots. I can't immediately think of a case where that would happen without more than just a VM update. Greetings, Andres Freund

Re: Why our Valgrind reports suck

2025-05-23 Thread Andres Freund
Hi, On 2025-05-22 21:48:24 -0400, Tom Lane wrote: > Andres Freund writes: > >> But this is the last step to get to zero reported leaks in a run of the > >> core > >> regression tests, so let's do it. > > > I assume that's just about the core te

Re: Why our Valgrind reports suck

2025-05-22 Thread Andres Freund
t; But this is the last step to get to zero reported leaks in a run of the core > regression tests, so let's do it. I assume that's just about the core tests, not more? I.e. I can't make skink enable leak checking? Greetings, Andres Freund

Re: Minor adjustment to pg_aios output naming

2025-05-22 Thread Andres Freund
ing. I think the change is fine, but I was also a bit surprised about the speed the change went in... Greetings, Andres Freund

Re: Tree-walker callbacks vs -Wdeprecated-non-prototype

2025-05-22 Thread Andres Freund
reen > today. I have more work to do to fix that, which might take some time > to get back to. Presumably due to conflict resolution, this commit added an empty meson.build to REL_13_STABLE. I'll remove that... Greetings, Andres Freund

Re: strange perf regression with data checksums

2025-05-21 Thread Andres Freund
Hi, On 2025-05-19 15:45:01 -0400, Peter Geoghegan wrote: > On Mon, May 19, 2025 at 3:37 PM Andres Freund wrote: > > I think we can do better - something like > > > > #ifdef PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY > > lsn = PageGetLSN(page); > > #else >

MERGE issues around inheritance

2025-05-21 Thread Andres Freund
er. Greetings, Andres Freund [1] https://postgr.es/m/smj2mopf4t654xinuyyq4azik2d5zp2g3lsv2o7vficegqq5c5%40v35iq4llqjpw [2] MERGE into measurement m USING new_measurement nm ON (m.city_id = nm.city_id and m.logdate=nm.logdate) WHEN MATCHED AND nm.peaktemp IS NULL THEN DELETE WHEN MATCHED THEN

Re: Violation of principle that plan trees are read-only

2025-05-20 Thread Andres Freund
Hi, On 2025-05-20 16:18:57 -0400, Tom Lane wrote: > Andres Freund writes: > >> I'm tempted to back-patch this: the plan tree damage seems harmless at > >> present, but maybe it'd become less harmless with future fixes. > > > There are *some* cases where

Re: Violation of principle that plan trees are read-only

2025-05-20 Thread Andres Freund
matter? Not sure if such cases are reachable via views (and thus pg_dump) or postgres_fdw, but it seems possible. Greetings, Andres Freund

Re: Violation of principle that plan trees are read-only

2025-05-20 Thread Andres Freund
Hi, On 2025-05-20 10:59:22 -0400, Andres Freund wrote: > On 2025-05-18 19:31:33 -0400, Tom Lane wrote: > > While chasing down Valgrind leakage reports, I was disturbed > > to realize that some of them arise from a case where the > > executor scribbles on the plan tree it

Re: Violation of principle that plan trees are read-only

2025-05-20 Thread Andres Freund
re is exactly one case in our tests where the tupledescs aren't equal :( I've not dug fully into this, but I thought I should send this email to confirm that I'm looking into the issue. Greetings, Andres Freund

Re: AIO v2.5

2025-05-19 Thread Andres Freund
Hi, On 2025-05-08 19:22:27 -0700, Noah Misch wrote: > On Thu, May 08, 2025 at 09:06:18PM -0400, Andres Freund wrote: > > On 2025-05-02 20:05:11 -0700, Noah Misch wrote: > > > On Wed, Apr 30, 2025 at 04:00:35PM -0400, Andres Freund wrote: > > > We do need to hold inter

Re: strange perf regression with data checksums

2025-05-19 Thread Andres Freund
G_HAVE_8BYTE_SINGLE_COPY_ATOMICITY lsn = PageGetLSN(page); #else buf_state = LockBufHdr(bufHdr); lsn = PageGetLSN(page); UnlockBufHdr(bufHdr, buf_state); #endif All perf relevant systems support reading 8 bytes without a chance of tearing... Greetings, Andres Freund

Re: wrong query results on bf leafhopper

2025-05-19 Thread Andres Freund
Hi, On 2025-05-19 12:49:26 +0930, Robins Tharakan wrote: > Hi Andres, > > On Fri, 16 May 2025 at 22:49, Andres Freund wrote: > > > > There have been other odd things on leafhopper, see e.g.: > > > https://www.postgresql.org/message-id/35d87371-f3ab-42c8-9aac-bb39

wrong query results on bf leafhopper

2025-05-16 Thread Andres Freund
&dt=2024-12-16%2023%3A43%3A03 There have been other odd things on leafhopper, see e.g.: https://www.postgresql.org/message-id/35d87371-f3ab-42c8-9aac-bb39ab5bd987%40gmail.com https://postgr.es/m/Z4npAKvchWzKfb_r%40paquier.xyz Greetings, Andres Freund

Re: fix: propagate M4 env variable to flex subprocess

2025-05-13 Thread Andres Freund
g* FLEX_TMP_DIR to the caller's environment. Bilal, I think you wrote this originally, do you recall? It seems like an issue beyond just M4... Greetings, Andres Freund

spurious pg_amcheck test failure

2025-05-13 Thread Andres Freund
st - afaict the pg_amcheck query has an ORDER BY that should prevent that? ") AS combined_records" "\nORDER BY pattern_id NULLS LAST, datname"); Greetings, Andres Freund

Re: Add partial :-variable expansion to psql \copy

2025-05-13 Thread Andres Freund
416192 https://api.cirrus-ci.com/v1/task/5702107409416192/logs/cores.log Greetings, Andres Freund

Re: Serverside SNI support in libpq

2025-05-13 Thread Andres Freund
Hi, On 2025-02-27 14:38:24 +0100, Daniel Gustafsson wrote: > The attached v6 rebase contains this as well as your tests as well as > general cleanup and comment writing etc. This is not passing CI on windows... https://cirrus-ci.com/build/4765059278176256 Greetings, Andres

Re: Memoize ANTI and SEMI JOIN inner

2025-05-13 Thread Andres Freund
Hi, On 2025-04-10 19:36:14 +0900, Richard Guo wrote: > On Wed, Apr 9, 2025 at 6:18 PM David Rowley wrote: > > On Wed, 9 Apr 2025 at 18:48, Richard Guo wrote: > > > Perhaps we could spend some planner cycles proving inner_unique for > > > anti joins, so that Memoize nodes can be considered for th

Re: Test mail for pgsql-hackers

2025-05-13 Thread Andres Freund
On 2025-03-12 15:56:08 +0530, Ashutosh Bapat wrote: > Large part of your patch is renaming files which are not related to > this patch. Can you please clean that up. Since this has not been fixed, and the patch has not passed CI since at least mid March, I'll close this CF entry.

Re: using index to speedup add not null constraints to a table

2025-05-13 Thread Andres Freund
t/001_constraint_validation.pl, > > So I created a file: > src/test/modules/test_misc/t/008_indexscan_validate_notnull.pl > for TAP tests. The tests have not passed in a few weeks: https://cirrus-ci.com/github/postgresql-cfbot/postgresql/cf%2F5444 Greetings, Andres Freund

Re: Why our Valgrind reports suck

2025-05-09 Thread Andres Freund
Hi, On 2025-05-09 11:29:43 -0400, Andres Freund wrote: > We currently don't reset TopMemoryContext at exit, which, obviously, does > massively increase the number of leaks. But OTOH, without that there's not a > whole lot of value in the leak check... Briefly looking throu

Re: Why our Valgrind reports suck

2025-05-09 Thread Andres Freund
y don't reset TopMemoryContext at exit, which, obviously, does massively increase the number of leaks. But OTOH, without that there's not a whole lot of value in the leak check... Greetings, Andres Freund

Re: Support for runtime parameters in injection points, for AIO tests

2025-05-08 Thread Andres Freund
Hi, On 2025-05-09 08:19:39 +0900, Michael Paquier wrote: > On Thu, May 08, 2025 at 06:16:46PM +, Greg Burd wrote: > > IMO seems like a good idea, happy to help further "cleanup the AIO > > code" with some direction as to what needs to be done. > > Thanks for posting a rebase. Yes, I would li

Re: AIO v2.5

2025-05-08 Thread Andres Freund
going during beta. And yes, I think this should involve some benchmarking too - generally a good thing to do during beta, and if it involves AIO, all the better. > FWIW I'm asking because of the RMT, but I'm also willing to do some of > the tests, if needed - but it'd be good to get some guidance. More testing would be welcome! Greetings, Andres Freund

Re: AIO v2.5

2025-05-08 Thread Andres Freund
Hi, On 2025-05-02 20:05:11 -0700, Noah Misch wrote: > On Wed, Apr 30, 2025 at 04:00:35PM -0400, Andres Freund wrote: > > pgaio_io_wait_for_free() does what it says on the tin. For that, after a > > bunch > > of other things, finds the oldest in-fligh

Re: PostgreSQL 18 Beta 1 release announcement draft

2025-05-06 Thread Andres Freund
Hi, On 2025-05-06 16:53:46 -0400, Bruce Momjian wrote: > On Tue, May 6, 2025 at 11:55:53AM -0400, Andres Freund wrote: > > On 2025-05-05 15:07:01 -0400, Jonathan S. Katz wrote: > > > ### Performance > > > > > > PostgreSQL 18 introduces an asynchronous I/O (

Re: PostgreSQL 18 Beta 1 release announcement draft

2025-05-06 Thread Andres Freund
date the > integrity of stored data, are now enabled by default on new PostgreSQL > clusters. You can choose to disable this behavior using the `initdb > --no-data-checksums` command. Should we point out that this might require adjustments for upgrade scripts? Probably not in the release announcement? Greetings, Andres Freund

Re: AIO v2.5

2025-04-30 Thread Andres Freund
's a bit coarse grained. It would need to happen in a few places, which isn't great either. Other suggestions? Thanks again for finding and reporting this Alexander! Greetings, Andres Freund

Re: gcc 15 "array subscript 0" warning at level -O3

2025-04-25 Thread Andres Freund
gt;datatype->typlen == 1); > assign_simple_var(estate, var, BoolGetDatum(state), false, false); > > which should be OK since we're expecting the "found" variable to > be boolean. That does silence the warning, but of course only > in --enable-cassert builds. One way to address this is outlined here: https://postgr.es/m/20230316172818.x6375uvheom3ibt2%40awork3.anarazel.de https://postgr.es/m/20240207203138.sknifhlppdtgtxnk%40awork3.anarazel.de I've been wondering about adding wrapping something like that in a pg_assume(expr) or such. Greetings, Andres Freund

Re: Back-patch of: avoid multiple hard links to same WAL file after a crash

2025-04-25 Thread Andres Freund
Andres, what do you think about making one of those skink configuration > changes? Alternatively, I could make the test poll until archiving catches > up. However, that would take skink about 30min, and I expect little value > from 30min of valgrind instrumenting the "cp" command. I just changed the config to --trace-children=no. There already is a valgrind run in progress, so it won't be in effect for the next run. Greetings, Andres Freund

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Andres Freund
Hi, On 2025-04-25 10:26:09 -0400, Tom Lane wrote: > Andres Freund writes: > > void > > assign_io_max_combine_limit(int newval, void *extra) > > { > > io_max_combine_limit = newval; > > io_combine_limit = Min(io_max_combine_limit, io_c

Re: AIO v2.5

2025-04-25 Thread Andres Freund
aim() and the generation increase / state update. The fix is simple, increment generation and state before resetting fields. Will push the fix for that soon. Greetings, Andres Freund

Re: Allow io_combine_limit up to 1MB

2025-04-25 Thread Andres Freund
ity ensues. Besides the obvious fix, I think we should also add Assert(len <= io_max_combine_limit); to pgaio_io_set_handle_data_32/64, that'd have made the bug much more obvious. Greetings, Andres Freund

Re: AIO v2.5

2025-04-23 Thread Andres Freund
Hi, On 2025-04-20 18:00:00 +0300, Alexander Lakhin wrote: > 31.03.2025 02:46, Andres Freund wrote: > > I've pushed most of these after some very light further editing. Yay. > > Thanks > > a lot for all the reviews! > > > > So far the buildfarm hasn&#

Re: Non-reproducible AIO failure

2025-04-23 Thread Andres Freund
t; data? The weird thing is that the concrete symptom here doesn't actually involve anything that could really race - the relevant state is only updated by the process itself. But of course there has to be something narrow here, otherwise we'd have seen this before :(. Greetings, Andres Freund

Re: pgsql: Add function to get memory context stats for processes

2025-04-22 Thread Andres Freund
Hi, On 2025-04-17 10:42:45 -0400, Robert Haas wrote: > On Tue, Apr 15, 2025 at 6:11 AM Andres Freund wrote: > > There very well could be a CFI - but it better be somewhere where the > > in-memory state is consistent. Otherwise an error inside raised in the CFI > > would lea

Re: ZStandard (with dictionaries) compression support for TOAST compression

2025-04-22 Thread Andres Freund
after a while. I don't see any argument why that would be ok, really. > But maybe not. It seems even worse to me than undroppable compression > methods, because you'll probably not have that many compression methods > ever, but you could have a large number of dictionaries eventually. Agreed on the latter. Greetings, Andres Freund

Re: Changing shared_buffers without restart

2025-04-18 Thread Andres Freund
Hi, On April 18, 2025 11:17:21 AM GMT+02:00, Thomas Munro wrote: > Doesn't that achieve the goal with fewer steps, using only >portable* POSIX stuff, and keeping all pointers stable? I understand >that pointer stability may not be required (I can see roughly how that >argument is constructed),

Re: Support for runtime parameters in injection points, for AIO tests

2025-04-15 Thread Andres Freund
s OK > for me. I can see arguments for going either way on that one. > The code is shaped so as we can rely on InjectionPointCallback to define the > shape of a callback. I don't know what this means though? Greetings, Andres Freund

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-15 Thread Andres Freund
_domain_acceptable' >   undefined reference to `nghttp2_session_check_request_allowed' > > I assume the many libs listed in Libs.private in libcurl.pc are not added > automatically for this check? The configure test shouldn't link statically, so this doesn't make sense to me? Greetings, Andres Freund

Re: AIO v2.5

2025-04-15 Thread Andres Freund
this error on three different machines (all are running > Ubuntu 24.04, two with kernel version 6.8, one with 6.11), with PGDATA > located on tmpfs. That's another variable to try - so far I've been trying this on 6.15.0-rc1 [1]. I guess I'll have to set up a ubuntu 24.04 VM

Re: BitmapHeapScan streaming read user and prelim refactoring

2025-04-15 Thread Andres Freund
ormation about 1) the # in-flight IOs 2) the # completed IOs 3) current position 4) reada d) provides *per-stream* control over readahead. You want to be only as aggressive about prefetching as you need to be, which very well can differ between different parts of a query tree. e) provides one central point to implement readahead logic As I said above, that's not to say that we'll only ever want to do readahead via a the read stream interface. Greetings, Andres Freund

Re: Recent pg_rewind test failures in buildfarm

2025-04-15 Thread Andres Freund
Hi, On 2025-04-15 06:04:54 -0400, Andres Freund wrote: > See also > https://postgr.es/m/dwrkeszz6czvtkxzr5mqlciy652zau5qqnm3cp5f3p2po74ppk%40omg4g3cc6dgq Huh, turns out my emails of the last few days had gotten stuck. I only noticed when trying to open the above link and it was dead.

Re: pgsql: Add function to get memory context stats for processes

2025-04-15 Thread Andres Freund
Hi, On 2025-04-14 10:03:28 -0400, Robert Haas wrote: > On Thu, Apr 10, 2025 at 4:05 PM Andres Freund wrote: > > I don't know of existing discussion, but it seems rather fundamental to me - > > if either DSA or memory contexts could be inconsistent at a CFI(), how could > &

Re: Recent pg_rewind test failures in buildfarm

2025-04-15 Thread Andres Freund
g problem, so maybe we just started hitting > this by chance. Still ... anybody have an idea? See also https://postgr.es/m/dwrkeszz6czvtkxzr5mqlciy652zau5qqnm3cp5f3p2po74ppk%40omg4g3cc6dgq I am fairly certain it's the fault of 039549d70f6 Greetings, Andres Freund

pending_since assertion failure on skink

2025-04-15 Thread Andres Freund
Flush the IO statistics of active WAL senders more frequently Which afaict could lead to having pending stats as known to pgstat_report_stat() which then could be flushed "behind pgstat_report_stat()'s back" due to the new pgstat_flush_io() and pgstat_flush_backend() calls. We might just have to give up on that assertion, I guess? Greetings, Andres Freund

Re: ci: Allow running mingw tests by default via environment variable

2025-04-14 Thread Andres Freund
Hi, On April 14, 2025 9:22:40 PM GMT+02:00, Nathan Bossart wrote: >On Sat, Apr 12, 2025 at 08:18:35PM +1200, Thomas Munro wrote: >> Is it safe to assume that CI changes come under the same code-freeze >> exception as tests and docs? I would assume so, it's just scripting >> glue in .cirrus.XXX

Re: Buildfarm: Enabling injection points on basilisk/dogfish (Alpine / musl)

2025-04-13 Thread Andres Freund
Hi, On April 13, 2025 7:27:33 PM GMT+02:00, Wolfgang Walther wrote: >Andrew Dunstan: >> >> On 2025-04-12 Sa 10:10 PM, Noah Misch wrote: >>> On Sat, Apr 12, 2025 at 07:51:06PM +0200, Wolfgang Walther wrote: With injection points enabled, I get the following errors in test_aio:

Re: Regression test fails when 1) old PG is installed and 2) meson/ninja build is used

2025-04-11 Thread Andres Freund
executed with PATH=/srv/dev/build/postgres/m-dev-assert/tmp_install//srv/dev/install/postgres/m-dev-assert/bin: LD_LIBRARY_PATH=/srv/dev/build/postgres/m-dev-assert/tmp_install//srv/dev/install/postgres/m-dev-assert/lib/x86_64-linux-gnu Can you check whether your meson-logs/testlog.txt shows the appropriate PATH/LD_LIBRARY_PATH and whether libpq is in the right place? Greetings, Andres Freund

Re: Add pg_buffercache_evict_all() and pg_buffercache_mark_dirty[_all]() functions

2025-04-11 Thread Andres Freund
to warrant its own function. You can do that with pg_buffercache_evict(), it'll be slower than pg_buffercache_evict_all(), but given that determining the numa node already is somewhat expensive, I'm not sure it's going to make that big a difference. Greetings, Andres Freund

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2025-04-10 Thread Andres Freund
Hi, On 2025-04-11 00:47:19 +0200, Matthias van de Meent wrote: > On Fri, 11 Apr 2025 at 00:27, Andres Freund wrote: > > > > Hi, > > > > On 2025-03-09 14:13:52 +0200, Alexander Korotkov wrote: > > > I've revised commit message, comments, formattin

Re: MergeJoin beats HashJoin in the case of multiple hash clauses

2025-04-10 Thread Andres Freund
Hi, On 2025-03-09 14:13:52 +0200, Alexander Korotkov wrote: > I've revised commit message, comments, formatting etc. > I'm going to push this if no objections. I'm rather confused as to why this is a thing to push at this point? This doesn't seem to be a bugfix and it's post feature freeze. Andr

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-10 Thread Andres Freund
Hi, On 2025-04-07 18:38:19 +0200, Peter Eisentraut wrote: > On 07.04.25 16:43, Andres Freund wrote: > > There recently was a breakage of building with PG on macos with meson, due > > to > > the meson folks implementing a feature request to move away from using > >

Re: pgsql: Add function to get memory context stats for processes

2025-04-10 Thread Andres Freund
alking over the allocations made in the context, similar releasing a dsa? Greetings, Andres Freund

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-09 Thread Andres Freund
ts fields that autovacuum uses > are supplied the correct values > for the different thresholds they need to calculate, as described here [0] You so far have not outlined a single scenario where the current behaviour causes an issue. Greetings, Andres Freund

Re: Streaming relation data out of order

2025-04-09 Thread Andres Freund
ve a large enough queue_size, but this doesn't look like it's safe against *index + n > PG_INT16_MAX. The value would wrap into the negative. I think all these variables really ought to be unsigned. None of them ever could be negative afaict. Afaict that would lead to the correct behaviour when wapping around. Greetings, Andres Freund

Re: Draft for basic NUMA observability

2025-04-09 Thread Andres Freund
ly free to use the larger range here. Greetings, Andres Freund

Re: Add missing PGDLLIMPORT markings

2025-04-09 Thread Andres Freund
Hi, On April 9, 2025 12:58:23 PM EDT, Robert Haas wrote: >On Wed, Apr 9, 2025 at 11:28 AM Andres Freund wrote: >> FWIW, the AIO ones really don't make sense to make public - the only reason >> for those variables to exists is so they can be put into an array of >> ca

Re: Draft for basic NUMA observability

2025-04-09 Thread Andres Freund
Hi, On 2025-04-09 17:28:31 +0200, Tomas Vondra wrote: > On 4/9/25 17:14, Andres Freund wrote: > > I'd mention that the includes of postgres.h/fmgr.h is what caused missing > > build-time dependencies and via that failures on buildfarm member dogfish. > > > > No

Re: Add missing PGDLLIMPORT markings

2025-04-09 Thread Andres Freund
But I guess we don't really have a way to tell mark_pgdllimport.pl that. I did mark the internal AIO variables that maybe kinda somewhat insanely used by an extension as PGDLLIMPORT (c.f. aio_internal.h)... Greetings, Andres Freund

Re: Draft for basic NUMA observability

2025-04-09 Thread Andres Freund
H 2/2] ci: Check for missing dependencies in meson build > > Extends the meson build on Debian to also check for missing dependencies > by executing > > ninja -t missingdeps > > right after the build. This highlights unindended dependencies. > > Reviewed-by: Andres

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-09 Thread Andres Freund
Hi, On 2025-04-07 09:41:25 -0700, Jacob Champion wrote: > On Mon, Apr 7, 2025 at 7:21 AM Andres Freund wrote: > > On 2025-04-04 17:27:46 -0700, Jacob Champion wrote: > > I think otherwise > > we run into the danger of the wrong library version being loaded in some > &g

Re: Enhancing Memory Context Statistics Reporting

2025-04-08 Thread Andres Freund
Hi, On 2025-04-08 01:17:17 +0200, Daniel Gustafsson wrote: > > On 7 Apr 2025, at 17:43, Andres Freund wrote: > > >> + /* > >> + * Hold the process lock to protect writes to process specific memory. Two > >> + * processes publishing statistics do not bl

Re: Modern SHA2- based password hashes for pgcrypto

2025-04-08 Thread Andres Freund
Hi, On 2025-04-07 09:09:39 +0200, Bernd Helmle wrote: > Am Sonntag, dem 06.04.2025 um 23:02 -0400 schrieb Andres Freund: > > On 2025-04-05 19:22:58 +0200, Alvaro Herrera wrote: > > > I have pushed this now, hoping it won't explode. > > > > I have a WIP pat

Re: Draft for basic NUMA observability

2025-04-08 Thread Andres Freund
Hi, On 2025-04-09 01:10:09 +0200, Tomas Vondra wrote: > On 4/8/25 15:06, Andres Freund wrote: > > Hi, > > > > On 2025-04-08 17:44:19 +0500, Kirill Reshke wrote: > >> On Mon, 7 Apr 2025 at 23:00, Tomas Vondra wrote: > >>> I'll let the CI run th

Re: Draft for basic NUMA observability

2025-04-08 Thread Andres Freund
Hi, On 2025-04-09 00:47:59 +0200, Tomas Vondra wrote: > On 4/8/25 16:59, Andres Freund wrote: > > Hi, > > > > On 2025-04-08 09:35:37 -0400, Andres Freund wrote: > >> On April 8, 2025 9:21:57 AM EDT, Tomas Vondra wrote: > >>> On 4/8/25 15:06, Andres F

Re: Finalizing read stream users' flag choices

2025-04-08 Thread Andres Freund
b) There are plenty cases where READ_STREAM_SEQUENTIAL *hurts* with seqscans and io_method==sync, e.g. if there are plenty pre-existing buffers in s_b. c) It doesn't have an effect with io_method != sync But it'll depend a bit on our experiences. Greetings, Andres Freund

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-08 Thread Andres Freund
hat there's an issue increasing libpq's indirect footprint substantially, but I don't think that's due to curl's reputation or anything. It's just needing a significantly higher number of shared libraries, which comes with runtime and distribution overhead. Greetings, Andres Freund

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-04-08 Thread Andres Freund
t had been in development for a *long* time. It wasn't a secret that it had a dependency on curl. I don't think it's fair to penalize a feature's authors to not finish implementing a complicated and completely new requirement within 17 days. Greetings, Andres Freund

Re: Draft for basic NUMA observability

2025-04-08 Thread Andres Freund
Hi, On 2025-04-08 09:35:37 -0400, Andres Freund wrote: > On April 8, 2025 9:21:57 AM EDT, Tomas Vondra wrote: > >On 4/8/25 15:06, Andres Freund wrote: > >> On 2025-04-08 17:44:19 +0500, Kirill Reshke wrote: > >> I think it's not right that something in

Re: Draft for basic NUMA observability

2025-04-08 Thread Andres Freund
Hi, On April 8, 2025 9:21:57 AM EDT, Tomas Vondra wrote: >On 4/8/25 15:06, Andres Freund wrote: >> Hi, >> >> On 2025-04-08 17:44:19 +0500, Kirill Reshke wrote: >>> On Mon, 7 Apr 2025 at 23:00, Tomas Vondra wrote: >>>> I'll let the CI run t

  1   2   3   4   5   6   7   8   9   10   >