What's described here is a very well known fact: "pip install" (or "sage
--pip install") might mess up your venv, anyone who works with large
collections of python packages is well aware of it. Talking about it as
something "underdeveloped in Sage" is a bit strange, these are just
well-known features and deficiencies of pip.
Yet, pytest, a pip package, is installed and used rather regularly in Sage,
and nobody gets hurt. It is found to be safe to install this particular
package this way ("pip install pytest" does not affect anything in the Sage
venv, that's why).
There is no reason not to promote it to standard and keep it a pip package.
(Same applies to any other well-isolated from the rest of Sage bunch of
packages, e.g. Sphinx+Jupyter, i.e. the vast majority of Python packages -
but that's another story, let's not discuss it here now).
> It's really very surprising to me why anyone would be championing using
these kinds of package types more.
We have had these discussions already, why it still comes as "surprising"
to you is unclear. The extra tax we pay by not using pip
packages more has started to outweight the benefits long ago. People don't
want to spend extra time and other resources
to maintain and build often somewhat outdated versions of packages which
are available from PyPI in a ready way.
------------------------------
By the way, we certainly experimented a lot with combining exclusively
pip-installed packages with sagelib package built with Meson, and,
surprise, it worked.
The precaution of not using "pip install" in the Sage venv is akin to the
same precaution done by most, if not all, Linux distros, which just forbid
installing packages with pip into the main python environment. (one needs a
venv to use pip install).
Sage is a white crow among other python packages, in its way how it demands
hundreds of particular versions
of extra python packages to come along. The white crows are evolutionary
disadvantaged, and this applies to the Python universe
just as well as to the biosphere.
Dima
On Thursday, June 6, 2024 at 3:48:10 AM UTC+1 Matthias Koeppe wrote:
> On Friday, May 31, 2024 at 9:38:34 AM UTC-7 Dima Pasechnik wrote:
>
> Before looking at
> https://groups.google.com/g/sage-devel/c/lPLoA7zaoyg/m/dGE1B1jQEQAJ
> we should look at this proposal again, as pytest is a very suitable
> candidate for the kinds of packages (standard pip packages)
> proposed here
>
>
> David requested over there that I explain a bit more about pip packages,
> so here goes.
>
> The "pip" package type was added to the Sage build system in 2015 by
> Jeroen as a quick and dirty way to keep packages such as "biopython",
> previously available as an "old-style SPKG", available for installing after
> switching to the "git layout" and phasing out "old-style SPKGs". We have
> kept this mechanism around for exactly such use cases: To advertise
> packages for which there is no real interest among developers in packaging
> them (and updating them) properly.
>
> And the "pip" package type has not seen any development since 2015. In
> particular, it has not been updated in any way to account for the use of
> build isolation that has become the default in the Python world around
> 2020. For example, say a "pip" package has a build dependency on numpy --
> then during the build phase it's not even going to use our installed numpy
> package but rather it's going to use a random binary wheel from PyPI. And
> all of that is resolved against the index server too, so there will be
> retroactive breakage.
>
> Moreover, when a "pip" package is installed the first time, the Sage
> distribution really just calls "pip install ..."; this contacts the index
> server (PyPI). And the Sage build system does not keep or check
> installation records for these packages, so it just calls "pip install ..."
> again *every time* that its installation is requested directly or
> indirectly as a dependency. And "pip install" will happily do anything that
> is requested, such as replacing any currently installed Python packages,
> including packages installed by the Sage distribution. Nothing is pinned,
> everything is up for grabs, and what is finally installed will also depend
> on the order of operations.
>
> It's really very surprising to me why anyone would be championing using
> these kinds of package types more.
>
> Well, it does sound all very nice and simple, but unfortunately in reality
> it's not. My suggestion to those who are actually interested in discussing
> the build system of the Sage distribution: Learn how the Sage distribution
> builds its Python packages, about its wheelhouse in
> "venv/var/lib/sage/wheels" and the generated requirements files in
> "venv/var/lib/sage/scripts", which is how the Sage distributions achieves a
> reliable and robust installation of its Python packages.
>
>
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sage-devel/db9fb9a1-1921-42e4-ab4a-58563b95d041n%40googlegroups.com.
