GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/836855908 (All green. phew.)
(This is a quick V2 RFC to freshen the patches on the list and consolidate some fixes made so far. See changes below.) This patch series creates a mandatory venv during configure time and uses it to install meson. The eventual goal of all of this is to ensure that the Python used to run meson is the same Python used to run Sphinx, tests, and any build-time scripting we have. As it stands, meson and sphinx (and their extensions) may run in a different Python environment than the one configured and chosen by the user at configure/build time. We can ensure that everything uses the same Python and has the same packages available with a bit of up-front work. See https://www.qemu.org/2023/03/24/python/ for more information. Changes in V2: - Each workaround in mkvenv is added in a separate patch now to better isolate parts of the design and what they were added to address. - Introduction into configure is split into two parts, the first of which is quite a bit cleaner and introduces a lot less logic into the configure file. - Avocado tests now use the configure-time venv. - Python 3.7 now works unconditionally without new dependencies, though we might drop it in favor of 3.8 soon. "Just in case", I suppose. - msys2 CI jobs now pass - there was a bug in the os.chmod call under windows. - NetBSD now works; mkvenv.py has better diagnostics for detecting NetBSD edge cases. Changes yet to come for further revisions to this series: - Need to add Sphinx support. - Need to add nested venv support. - PyPI support? (To drop meson git submodule?) - Installing meson from vendored .whl files for tarballs? (To drop --no-use-pep517 hack?) John Snow (10): python: add mkvenv.py tests: add python3-venv dependency mkvenv: Add better error message for missing pyexapt module tests/vm: Configure netbsd to use Python 3.10 tests/vm: add py310-expat to NetBSD mkvenv: generate console entry shims from inside the venv mkvenv: work around broken pip installations on Debian 10 configure: create a python venv unconditionally configure: remove --meson=; install meson to the pyvenv tests: Use configure-provided pyvenv for tests docs/devel/acpi-bits.rst | 6 +- docs/devel/testing.rst | 14 +- configure | 138 ++++-- .gitlab-ci.d/buildtest-template.yml | 4 +- .gitlab-ci.d/buildtest.yml | 6 +- python/scripts/mkvenv.py | 466 ++++++++++++++++++ .../org.centos/stream/8/x86_64/test-avocado | 4 +- scripts/device-crash-test | 2 +- tests/Makefile.include | 10 +- .../dockerfiles/debian-all-test-cross.docker | 3 +- .../dockerfiles/debian-hexagon-cross.docker | 3 +- .../dockerfiles/debian-riscv64-cross.docker | 3 +- .../dockerfiles/debian-tricore-cross.docker | 3 +- tests/requirements.txt | 7 +- tests/vm/netbsd | 2 + 15 files changed, 590 insertions(+), 81 deletions(-) create mode 100644 python/scripts/mkvenv.py -- 2.39.2
