On Sunday, February 11, 2024 at 1:46:40 PM UTC-8 Matthias Koeppe wrote: I'll make an attempt to quantify this cost
Here's an illustration of the workflow for making python_build a standard "wheel" package, as proposed in https://groups.google.com/g/sage-devel/c/MIU-xo9b7pc: *$ git checkout -b python_build_standard upstream/develop*branch 'python_build_standard' set up to track 'upstream/develop'. Switched to a new branch 'python_build_standard' *$ ls build/pkgs/python_build*SPKG.rst dependencies distros requirements.txt type The package already exists as a "pip" package (requirements.txt). Let's re-create it as a standard "wheel" package. *$ mv build/pkgs/python_build build/pkgs/build$ ./sage -package create build --pypi --type standard*Downloading tarball from https://pypi.io/packages/py3/b/build/build-1.0.3-py3-none-any.whl to .../upstream/build-1.0.3-py3-none-any.whl [......................................................................] *$ mv build/pkgs/build build/pkgs/python_build$ ls build/pkgs/python_build*SPKG.rst checksums.ini dependencies distros install-requires.txt package-version.txt requirements.txt type *$ git rm -f build/pkgs/python_build/requirements.txt*rm 'build/pkgs/python_build/requirements.txt' Now, after removing requirements.txt, it's a wheel package. Let's review the changes that "sage -package create" made. *$ git --no-pager diff build/pkgs/python_build/dependencies* diff --git a/build/pkgs/python_build/dependencies b/build/pkgs/python_build/dependencies index b72a6d1c776..47296a7bace 100644 --- a/build/pkgs/python_build/dependencies +++ b/build/pkgs/python_build/dependencies @@ -1,4 +1,4 @@ - pyparsing tomli packaging | $(PYTHON_TOOLCHAIN) $(PYTHON) + | $(PYTHON_TOOLCHAIN) $(PYTHON) ---------- All lines of this file are ignored except the first. Our old version was better, go back to it. (The script "sage -package create" does not know how to find the dependencies; https://github.com/sagemath/sage/pull/36740 prepares an improvement, needs review.) *$ git checkout -- build/pkgs/python_build/dependencies* Commit the changes: *$ git add build/pkgs/python_build$ git commit -m "build/pkgs/python_build: Change to a normal standard package"*[python_build_standard 43f6b2b8ef9] build/pkgs/python_build: Change to a normal standard package 4 files changed, 7 insertions(+), 1 deletion(-) create mode 100644 build/pkgs/python_build/checksums.ini rename build/pkgs/python_build/{requirements.txt => install-requires.txt} (100%) create mode 100644 build/pkgs/python_build/package-version.txt Test it: *$ make python_build*make -j16 build/make/Makefile --stop ./bootstrap -d [...] rm -rf config/install-sh config/compile config/config.guess config/config.sub config/missing configure build/make/Makefile-auto.in make --no-print-directory python_build-SAGE_VENV-no-deps [python_build-1.0.3] Using cached file .../upstream/build-1.0.3-py3-none-any.whl [python_build-1.0.3] python_build-1.0.3 [python_build-1.0.3] ==================================================== [python_build-1.0.3] Setting up build directory for python_build-1.0.3 [...] [python_build-1.0.3] Using pip 23.3.1 from .../local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/pip (python 3.11) [python_build-1.0.3] Looking in links: .../local/var/lib/sage/venv-python3.11/var/lib/sage/wheels [python_build-1.0.3] Processing .../local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/build-1.0.3-py3-none-any.whl (from -r .../local/var/lib/sage/venv-python3.11/var/lib/sage/scripts/python_build/spkg-requirements.txt (line 1)) [python_build-1.0.3] Requirement already satisfied: packaging>=19.0 in .../local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages (from build@ file://.../local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/build-1.0.3-py3-none-any.whl->-r .../local/var/lib/sage/venv-python3.11/var/lib/sage/scripts/python_build/spkg-requirements.txt (line 1)) (23.2) [python_build-1.0.3] Requirement already satisfied: pyproject_hooks in .../local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages (from build@ file://.../local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/build-1.0.3-py3-none-any.whl->-r .../local/var/lib/sage/venv-python3.11/var/lib/sage/scripts/python_build/spkg-requirements.txt (line 1)) (1.0.0) [python_build-1.0.3] Installing collected packages: build [python_build-1.0.3] changing mode of .../local/var/lib/sage/venv-python3.11/bin/pyproject-build to 755 [python_build-1.0.3] Successfully installed build-1.0.3 [python_build-1.0.3] Successfully installed python_build-1.0.3 [...] Sage build/upgrade complete! It did not complain about dependencies, so we seem to be good. But the "pyproject_hooks" that it was happy to find comes from the previous installation, we don't have it as a package. Let's create it as a standard package. *$ ./sage -package create pyproject_hooks --pypi --type standard*Downloading tarball from https://pypi.io/packages/py3/p/pyproject_hooks/pyproject_hooks-1.0.0-py3-none-any.whl to .../upstream/pyproject_hooks-1.0.0-py3-none-any.whl [......................................................................] *$ make pyproject_hooks*make -j16 build/make/Makefile --stop ./bootstrap -d [...] make --no-print-directory pyproject_hooks-SAGE_VENV-no-deps [pyproject_hooks-1.0.0] Using cached file .../upstream/pyproject_hooks-1.0.0-py3-none-any.whl [pyproject_hooks-1.0.0] pyproject_hooks-1.0.0 [...] [pyproject_hooks-1.0.0] Found existing installation: pyproject_hooks 1.0.0 [pyproject_hooks-1.0.0] Uninstalling pyproject_hooks-1.0.0: [pyproject_hooks-1.0.0] Successfully uninstalled pyproject_hooks-1.0.0 [pyproject_hooks-1.0.0] Using pip 23.3.1 from .../local/var/lib/sage/venv-python3.11/lib/python3.11/site-packages/pip (python 3.11) [pyproject_hooks-1.0.0] Looking in links: .../local/var/lib/sage/venv-python3.11/var/lib/sage/wheels [pyproject_hooks-1.0.0] Processing .../local/var/lib/sage/venv-python3.11/var/lib/sage/wheels/pyproject_hooks-1.0.0-py3-none-any.whl (from -r .../local/var/lib/sage/venv-python3.11/var/lib/sage/scripts/pyproject_hooks/spkg-requirements.txt (line 1)) [pyproject_hooks-1.0.0] Installing collected packages: pyproject_hooks [pyproject_hooks-1.0.0] Successfully installed pyproject_hooks-1.0.0 [...] Sage build/upgrade complete! No more dependencies to take care of, we are done. *$ git add build/pkgs/pyproject_hooks$ git commit -m "build/pkgs/pyproject_hooks: New, python_build dependency"*[python_build_standard 58ab4c838e3] build/pkgs/pyproject_hooks: New, python_build dependency 6 files changed, 28 insertions(+) create mode 100644 build/pkgs/pyproject_hooks/SPKG.rst create mode 100644 build/pkgs/pyproject_hooks/checksums.ini create mode 100644 build/pkgs/pyproject_hooks/dependencies create mode 100644 build/pkgs/pyproject_hooks/install-requires.txt create mode 100644 build/pkgs/pyproject_hooks/package-version.txt create mode 100644 build/pkgs/pyproject_hooks/type *$ git push -u origin HEAD*Enumerating objects: 22, done. Counting objects: 100% (22/22), done. Delta compression using up to 12 threads Compressing objects: 100% (14/14), done. Writing objects: 100% (18/18), 1.92 KiB | 163.00 KiB/s, done. Total 18 (delta 6), reused 7 (delta 0), pack-reused 0 remote: Resolving deltas: 100% (6/6), completed with 3 local objects. remote: remote: Create a pull request for 'python_build_standard' on GitHub by visiting: remote: https://github.com/mkoeppe/sage/pull/new/python_build_standard remote: To https://github.com/mkoeppe/sage.git * [new branch] HEAD -> python_build_standard branch 'python_build_standard' set up to track 'origin/python_build_standard'. (That's https://github.com/sagemath/sage/pull/37300) -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/d0ed27de-da47-4757-9097-4f434fd75b57n%40googlegroups.com.
