> On May 28, 2020, at 3:13 PM, Tom Most <t...@freecog.net> wrote: > > On Fri, May 22, 2020, at 10:54 PM, Glyph wrote: >> So, pidfd's cool, we should totally use it. Also we should use posix_spawn >> and maybe some other stuff too. But I wonder if there's any heuristic we >> could use to speed up our current strategy, like ordering the to-reap list >> by putting things with no open FDs at the front of it? And optimistically >> assuming that once we've found something to reap, maybe we can stop? And >> maybe it should run in a cooperator, rather than just blocking the reactor >> indefinitely? > > Those all sound like reasonable optimizations with little downside to me.
Great! Anybody feel like filing some tickets? :) >> Honestly it had not occurred to me that people were managing 20k+ python >> interpreters at a time with spawnProcess (although, yikes, amazing, you've >> gotta talk more about this application, and what kind of hardware you're on!) > > Nonono, that was a synthetic stress test. It did spawn Python processes, but > they immediately called `exec sleep` to conserve memory. > > The result of the test was that the service would run out of memory _well_ > before inefficiencies in Twisted's reaping scheme became a problem. Aah, that's less exciting ;). -g _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
