> On May 19, 2020, at 11:14 AM, Tom Most <t...@freecog.net> wrote:
>
> On Tue, May 19, 2020, at 5:41 AM, Jean-Paul Calderone wrote:
>> On Tue, May 19, 2020 at 8:33 AM Barry Scott <barry.sc...@forcepoint.com
>> <mailto:barry.sc...@forcepoint.com>> wrote:
>> Why not waitpid for all children (pid=0 or pid=-1) in a single call?
>>
>> Because then you might reap a child process someone launched with a
>> different API - eg the Python subprocess module.
>
> Exactly. I am currently using this as a workaround, but it's not a general
> solution. You need to be careful because subprocess use hides in many places.
> For example, beware uuid.uuid1().
So, pidfd's cool, we should totally use it. Also we should use posix_spawn and
maybe some other stuff too. But I wonder if there's any heuristic we could use
to speed up our current strategy, like ordering the to-reap list by putting
things with no open FDs at the front of it? And optimistically assuming that
once we've found something to reap, maybe we can stop? And maybe it should run
in a cooperator, rather than just blocking the reactor indefinitely? Honestly
it had not occurred to me that people were managing 20k+ python interpreters at
a time with spawnProcess (although, yikes, amazing, you've gotta talk more
about this application, and what kind of hardware you're on!)
Anyway there's lot of exciting room for performance improvements here, it would
be nice to have some benchmarks for this kind of high-load process-monitor
application.
-g
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
