New submission from Neil Schemenauer <nas-pyt...@arctrix.com>:
Note: This is a proof of concept and not ready for merging as is.
This is based on 'frozen_modules' from Jeethu Rao <jee...@jeethurao.com>, via
Larry Hastings. Larry's git branch was:
g...@github.com:larryhastings/cpython.git
not_another_use_of_the_word_frozen
Usage:
- Compile Python as normal
- Run "make regen-freeze-startup" to re-generate Python/frozenmodules_code.c
- Compile Python a second time
Changes from Larry's branch:
- Move static C code generation tool to Tools/freeze2
- Move _serializer to Modules
- Rebase on Python 3.10.0b1
- determine startup modules by running sys.executable
- use importlib.util.find_spec() to get code objects
- fix ref-counting when setting __path__
- put static frozen modules in frozen_code_objects dict
- reduce set of "bad" modules as it seems only _collections_abc needs
exclusion
- fix the is_frozen_package() and is_frozen() functions to find
static frozen code
It's not passing all unit tests yet but I'm somewhat hopeful there are no deep
problems. Porting the changes from 3.6 to 3.8 and then to 3.10 was not too
horrible. There was a few changes to PyGC_Head, to the PyCodeObject structure
and to the runtime initialization process. That gives me some hope that it
wouldn't be too burdensome to maintain this in the long-term. Mostly it would
be updating _serialize.c to keep up with PyCodeObject changes.
Based on benchmarking with 3.8, this gives a decent speedup for startup of a
trival program, e.g. python -c "True". I measure it as taking 76% of the time.
The savings are mostly in marshal.c but there is also some
importlib/filesystem overhead that's removed.
----------
messages: 395243
nosy: nascheme
priority: low
severity: normal
stage: patch review
status: open
title: Generate static PyCodeObjects for faster startup
type: performance
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue44331>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
