[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset be3b4aa2ad28 by doko in branch '2.7': - Issue #13150, #17512: sysconfig no longer parses the Makefile and config.h http://hg.python.org/cpython/rev/be3b4aa2ad28 -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset d174cb3f5b9e by Benjamin Peterson in branch '2.7': backout 66e30c4870bb for breaking OSX (#13150) http://hg.python.org/cpython/rev/d174cb3f5b9e -- ___ Python tracker _

[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset 66e30c4870bb by doko in branch '2.7': - Issue #13150: sysconfig no longer parses the Makefile and config.h files http://hg.python.org/cpython/rev/66e30c4870bb -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Matthias Klose added the comment: the current ability to cross-build python now relies on being able to run the build python with the host library, using the _sysconfigdata.py from the host. if somebody decides to implement _sysconfigdata as a C extension, please ensure that this information

[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset 677e625e2ef1 by Victor Stinner in branch 'default': Issue #13150: Add a comment in _sysconfigdata to explain the origin of this file http://hg.python.org/cpython/rev/677e625e2ef1 -- ___ Python tracker <

[issue13150] Most of Python's startup time is sysconfig

Roumen Petrov added the comment: 10x for solution, 10x for commit . Good bye cross compilation! Any attempt to improve python build system to support cross-build, multilib build, build outside source tree with different options is useless. -- nosy: +rpetrov ___

[issue13150] Most of Python's startup time is sysconfig

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue13150] Most of Python's startup time is sysconfig

Éric Araujo added the comment: BTW, distutils2 backports the sysconfig module and cfg file from 3.3, so now the two versions will diverge. -- ___ Python tracker ___ ___

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: Done! If someone wants to give life to the C approach, they are welcome :) -- resolution: -> fixed stage: -> committed/rejected status: open -> closed ___ Python tracker ___

[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset 70160b53117f by Antoine Pitrou in branch 'default': Issue #13150: sysconfig no longer parses the Makefile and config.h files http://hg.python.org/cpython/rev/70160b53117f -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: > > A Python solution would be more useful for other implementations > > if enough of the sysconfig info is not CPython specific. > That’s the point: the info currently parsed at runtime by sysconfig is > specific to CPython (Makefile and pyconfig.h), so adding

[issue13150] Most of Python's startup time is sysconfig

Éric Araujo added the comment: > Since #9878 proposes an *alternate* solution to *part* of the > sysconfig problem, I disagree with 'supersede'. It’s also an older issue. > A Python solution would be more useful for other implementations > if enough of the sysconfig info is not CPython specific

[issue13150] Most of Python's startup time is sysconfig

Changes by John O'Connor : -- nosy: +jcon ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue13150] Most of Python's startup time is sysconfig

Terry J. Reedy added the comment: Since #9878 proposes an *alternate* solution to *part* of the sysconfig problem, I disagree with 'supersede'. A Python solution would be more useful for other implementations if enough of the sysconfig info is not CPython specific. A CPython design feature i

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: > Actually, #9878 should supersede this bug: it proposes to generate a C > module to avoid parsing Makefile and pyconfig.h, and your patch > proposes to generate a Python module with the same goal. Well, #9878 doesn't have a patch, but perhaps Barry is willing

[issue13150] Most of Python's startup time is sysconfig

Éric Araujo added the comment: Actually, #9878 should supersede this bug: it proposes to generate a C module to avoid parsing Makefile and pyconfig.h, and your patch proposes to generate a Python module with the same goal. -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue13150] Most of Python's startup time is sysconfig

Éric Araujo added the comment: #9878 should also help with start-up time. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue13150] Most of Python's startup time is sysconfig

Ross Lagerwall added the comment: #11454 is another case where pre-parsing and pickling the regular expressions in the email module may improve import time considerably. -- nosy: +rosslagerwall ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: Pre-parsing and building a cached module of built-time variables (from Makefile and pyconfig.h) under POSIX also removes more than 15% of startup time. Patch attached. -- keywords: +patch Added file: http://bugs.python.org/file23377/sysconfigdata.patc

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: Arg damn roundup e-mail gateway. I wanted to paste: >>> pickle.dumps(re.compile('')) b'\x80\x03cre\n_compile\nq\x00X\x00\x00\x00\x00q\x01K \x86q\x02Rq\x03.' -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: > I am curious: wouldn't be a way of keeping the compiled expressions in > a static cache somewhere, so we would compile them just once and have > both import time and runtime fast ? Runtime shouldn't be affected. The re module has its own LRU caching. That sa

[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset ed0bc92fed68 by Antoine Pitrou in branch 'default': Use a dict for faster sysconfig startup (issue #13150) http://hg.python.org/cpython/rev/ed0bc92fed68 -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Tarek Ziadé added the comment: I am curious: wouldn't be a way of keeping the compiled expressions in a static cache somewhere, so we would compile them just once and have both import time and runtime fast ? -- ___ Python tracker

[issue13150] Most of Python's startup time is sysconfig

Roundup Robot added the comment: New changeset df950158dc33 by Antoine Pitrou in branch 'default': Issue #13150: The tokenize module doesn't compile large regular expressions at startup anymore. http://hg.python.org/cpython/rev/df950158dc33 -- nosy: +python-dev ___

[issue13150] Most of Python's startup time is sysconfig

Antoine Pitrou added the comment: Actually, a big part of that is compiling some regexes in the tokenize module. Just relying on the re module's internal caching shaves off 20% of total startup time. Before: $ time ./python -S -c 'import tokenize' real0m0.034s user0m0.030s sys 0

[issue13150] Most of Python's startup time is sysconfig

Changes by Nadeem Vawda : -- nosy: +nadeem.vawda ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue13150] Most of Python's startup time is sysconfig

Changes by Ezio Melotti : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue13150] Most of Python's startup time is sysconfig

New submission from Antoine Pitrou : sysconfig is imported and used by site.py. $ time ./python -S -c '' real0m0.019s user0m0.013s sys 0m0.005s $ time ./python -S -c 'import sysconfig' real0m0.047s user0m0.046s sys 0m0.002s $ time ./python -S -c 'import sysconfig; sys