Hi, On 2025-02-11 17:55:39 -0500, Tom Lane wrote: > Andres Freund <and...@anarazel.de> writes: > > So the issue would actually be that we're currently doing set_max_safe_fds() > > too late, not too early :/ > > Well, we'd rather set_max_safe_fds happen after semaphore creation, > so that it doesn't have to be explicitly aware of whether semaphores > consume FDs. Could we have it be aware of how many FDs *will be* > needed for io_uring, but postpone creation of those until after we > jack up RLIMIT_NOFILE?
Yes, that should be fairly trivial to do. It'd require a call from postmaster into the aio subsystem, after set_max_safe_fds() is done, but I think that'd be ok. I'd be inclined to not make that a general mechanism for now. > I guess the other way would be to have two rounds of RLIMIT_NOFILE > adjustment, before and after shmem creation. That seems ugly but > shouldn't be very time-consuming. Yea, something like that could also work. At some point I was playing with calling AcquireExternalFD() the appropriate number of times during shmem reservation. That turned out to work badly right now, because it relies on max_safe_fds() being set *and* insists on numExternalFDs < max_safe_fds / 3. One way to deal with this would be for AcquireExternalFD() to first increase the rlimit, if necessary and possible, then start to close LRU files and only then fail. Arguably that'd have the advantage of e.g. postgres_fdw not stomping quite so hard on VFDs, because we'd first increase the limit. Of course it'd also mean we'd increase the limit more often. But I don't think it's particularly expensive. Greetings, Andres Freund