[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround
Guido van Rossum wrote: > Is there some kind of optimized communication possible yet between > subinterpreters? (Otherwise I still worry that it's no better than > subprocesses -- and it could be worse because when one > subinterpreter experiences a hard crash or runs out of memory, all > others have to die with it.) The use case that I have in mind with subinterpreters is Windows. With its lack of fork() and the way it spawns a fresh interpreter process it always feels a bit weird to use multiprocessing on Windows. Would it be faster and/or cleaner to start a new in-process subinterpreter instead? ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/MVYVEDXIW24I5LX2RW26P3FVBBYIO6TT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PoC: Subinterpreters 4x faster than sequential execution or threads on CPU-bound workaround
On Fri, May 8, 2020 at 12:30 AM Sebastian Krause wrote: > > Guido van Rossum wrote: > > Is there some kind of optimized communication possible yet between > > subinterpreters? (Otherwise I still worry that it's no better than > > subprocesses -- and it could be worse because when one > > subinterpreter experiences a hard crash or runs out of memory, all > > others have to die with it.) > > The use case that I have in mind with subinterpreters is > Windows. With its lack of fork() and the way it spawns a fresh > interpreter process it always feels a bit weird to use > multiprocessing on Windows. Would it be faster and/or cleaner to > start a new in-process subinterpreter instead? Subinterpreters don't support fork() either -- they can't share any objects, so each one has to start from a blank slate and go through the Python startup sequence, re-import all modules from scratch, etc. Subinterpreters do get to skip the OS process spawn overhead, but most of the startup costs are the same. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/RIK75U3ROEHWZL4VENQSQECB4F4GDELV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2020-05-01 - 2020-05-08) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open7459 (+32) closed 44857 (+64) total 52316 (+96) Open issues with patches: 2996 Issues opened (66) == #36543: Remove old-deprecated ElementTree features (part 2) https://bugs.python.org/issue36543 reopened by scoder #40028: Math module method to find prime factors for non-negative int https://bugs.python.org/issue40028 reopened by mark.dickinson #40228: Make setting line number in frame more robust. https://bugs.python.org/issue40228 reopened by vstinner #40471: Grammar typo in issubclass docstring https://bugs.python.org/issue40471 opened by alexpovel #40474: Code coverage report not entirely accurate https://bugs.python.org/issue40474 opened by Lewis Ball #40477: Launcher on Catalina https://bugs.python.org/issue40477 opened by Auerhahn #40478: allow finding nmake.exe executable in MSVCCompiler https://bugs.python.org/issue40478 opened by japm48 #40479: Port _hashlib to OpenSSL 3.0.0 https://bugs.python.org/issue40479 opened by christian.heimes #40481: Add include and exclude filters to zipapp cli https://bugs.python.org/issue40481 opened by JarosÅaw Wygoda #40482: _hashlib: register Python names as OpenSSL aliases https://bugs.python.org/issue40482 opened by christian.heimes #40483: Implementing a verifier function to verify integrity of AST no https://bugs.python.org/issue40483 opened by BTaskaya #40484: Document existing compiler flags under ast https://bugs.python.org/issue40484 opened by BTaskaya #40485: Provide an abstraction for a select-able Event https://bugs.python.org/issue40485 opened by paravoid #40486: pathlib's iterdir doesn't specify what happens if directory co https://bugs.python.org/issue40486 opened by facundobatista #40487: Unexpected exception handler behavior in Jupyter when returnin https://bugs.python.org/issue40487 opened by jeanmonet #40492: -m cProfile -o f.pstats with a script that does chdir() writes https://bugs.python.org/issue40492 opened by Anthony Sottile #40494: collections.abc.Callable and type variables https://bugs.python.org/issue40494 opened by serhiy.storchaka #40495: compileall: option to hardlink duplicate optimization levels b https://bugs.python.org/issue40495 opened by frenzy #40497: subprocess.check_output() accept the check keyword argument https://bugs.python.org/issue40497 opened by remi.lapeyre #40498: Holding spacebar on button widget permanently makes it SUNKEN https://bugs.python.org/issue40498 opened by PythonAmateur742 #40500: test_multiprocessing_fork leaks processes on PPC64LE RHEL8 LTO https://bugs.python.org/issue40500 opened by vstinner #40501: Deprecate and remove ctypes usage in uuid https://bugs.python.org/issue40501 opened by steve.dower #40502: PyNode_New() does not initialize n->n_col_offset https://bugs.python.org/issue40502 opened by Tomasz Pytel #40503: PEP 615: Add zoneinfo module https://bugs.python.org/issue40503 opened by p-ganssle #40505: getpath.c doesn't know about lib64 https://bugs.python.org/issue40505 opened by lemburg #40506: add support for os.Pathlike filenames in zipfile.ZipFile.write https://bugs.python.org/issue40506 opened by d.ragusa #40507: FileNotFound error raised by os.exec* doesn't contain filename https://bugs.python.org/issue40507 opened by russelldavis #40509: In argparse, allow REMAINDER(...) arguments in a mutually excl https://bugs.python.org/issue40509 opened by Shani Armon #40510: [regression] ZipFile fails to round trip on some files https://bugs.python.org/issue40510 opened by DragonSA #40511: IDLE yellow hint frame blinks when entering () in strings in f https://bugs.python.org/issue40511 opened by wyz23x2 #40512: Meta issue: per-interpreter GIL https://bugs.python.org/issue40512 opened by vstinner #40513: Move _PyRuntimeState.ceval to PyInterpreterState https://bugs.python.org/issue40513 opened by vstinner #40514: Add --experimental-isolated-subinterpreters build option https://bugs.python.org/issue40514 opened by vstinner #40515: test_ssl.py hangs with SSL 1.1 built with no threads https://bugs.python.org/issue40515 opened by mig28suarez #40516: GCC 9 compiler warnings on MacOS Catalina https://bugs.python.org/issue40516 opened by remi.lapeyre #40519: Preserve AttributeError exception context in __getattr__ https://bugs.python.org/issue40519 opened by Arusekk #40521: Make tuple, dict, frame free lists, unicode interned strings, https://bugs.python.org/issue40521 opened by vstinner #40522: Subinterpreters: get the current Python interpreter state from https://bugs.python.org/issue40522 opened by vstinner #40525: zipapps execute symlinks as if they are code https://bugs.python.org/issue40525 opened by Anthony Sottile #40528: Improve / Clear ASDL generator https://bugs.
[Python-Dev] PEP 615 (zoneinfo) implementation ready for review
Hey all, The feature freeze is coming up on us fast, and the PEP 615 implementation is more or less ready to be integrated into the standard library (may need one or two little tweaks, but it's well past the "minimum viable product" stage). Normally I'd wait longer for someone to volunteer for the task of reviewing, but given the somewhat tight timeline and the fact that the code and tests alone (not including the documentation) are 6000 lines, I figured it's better to give people a head's up that I'm looking for reviewers. I've already had a few reviews on this code when it was first merged to the reference implementation, but there's also a decent chunk of otherwise unreviewed code. - The implementation and tests are in PR #19909: https://github.com/python/cpython/pull/19909 - The documentation and What's New entry is in #20006: https://github.com/python/cpython/pull/20006 There's also one other feature that I did not originally include in the PEP but which I think is a reasonable feature request that we are likely to get, which is a way to list all the time zones available on the system. The reference implementation includes an implementation for that feature in the property test suite, and it would be easy for me to port it over; I'll do that if there are no objections before the feature freeze, I've opened this BPO issue to track the discussion: https://bugs.python.org/issue40536 Thanks! Paul signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/5PBRWEDMKTVR2G7GBXL7HKRLQ73RVNFG/ Code of Conduct: http://python.org/psf/codeofconduct/
