On Wed, Apr 9, 2025 at 10:21 PM Marc Culler <marc.cul...@gmail.com> wrote: > > On Wed, Apr 9, 2025 at 8:07 PM Nils Bruin <nbr...@sfu.ca> wrote: >> >> On Monday, 7 April 2025 at 08:59:40 UTC-7 marc....@gmail.com wrote: >> >> If the Python spkg were removed we would no longer be able to start from a >> standard build of Sage. >> >> Can you explain what the non-standard part would be > > > William says "the proposal is that people install Python somehow" which means > that there would be no specific supported way of doing it. > >> and why you are concerned it would be more fragile on the side of Sage? > > > In order to be usable in the app, the python would need to be installed > inside of the app. When Sage is built, a directory named > local/var/lib/sage/venv-python3.X.Y is created, along with a symlink to that > directory named venv which is at the top level, adjacent to the "local" > directory. Despite the name, the venv directory is not a venv, by Python's > definition, because it has no pyvenv.cfg file inside. So it really is a > python prefix directory, which Sage suggests is a venv. And when Sage builds > python it uses that directory as the prefix. When you "install" a "system > python" in Sage, it also uses the venv directory, but in a different way. > There are many ways to populate the venv. One way would be to put symlinks > inside so, for example, venv/bin/python3.12 might be a symlink to the > homebrew python executable and venv/lib/python3.12 might be a symlink to the > homebrew python library directory. Another way would be to copy the homebrew > files into the venv directory. The first method is totally useless for > building the app. The symlinks would work for running sage on the computer > where sage was built, but not on a different computer without homebrew or > without the same version of homebrew's python. Moreover, Apple will not > notarize an app containing symlinks that point outside of the app bundle, > whether or not they might work on a given system. I don't actually know > whether a pyvenv.cfg file gets created when using a "system python" but I > suspect not. And I don't know how the venv gets populated when a "system > python" is used. These would be things I would have to discover by trial and > error and adjust for. > > To me, when the "standard" is "people install Python somehow" there is no > commitment whatsoever about any of the crucial details of how the "system > python" is actually being "installed" inside of Sage.
As a matter of fact, with the recently added meson build configs, one can finally build sagelib as an absolutely normal Python package, by calling "pip install ." in an already created standard venv. So no more hacked Sage venv, just a normal Python venv. It cannot get any better than this. That's obviously assuming that all non-Python Sage dependencies are already installed. So can just use this build method and stop worrying about Sage venv. As a matter of fact, I tested it on macOS today, to proceed with fixing Sage build on the latest XCode. I have an Intel MacBook with a full Sage build environment installed in /usr/local. It's a mix of Homebrew, Homebrew taps by Macaulay2 people: https://github.com/Macaulay2/homebrew-tap and hand-installed stuff such as m4ri and m4rie (I'm planning to fix a few Homebrew formulas to facilitate this). I had in fact to update few Homebrew formulae for Macaulay2 taps, as they stopped working with the new XCode. see https://github.com/Macaulay2/homebrew-tap/pull/270. (that's of course a personal choice, some people might want a full hand-built setup). Just to get a taste of uv, I tried to do a build with a Python 3.12.9, and a venv, provided by uv. And it works, and it's 5-10 times faster than pip on various tasks. > Therefore there is no expectation of consistency from one Sage release to the > next and basically I would have to count on having it change our build system > all the time. This is not so any more - we have more or less converged to the point where you can build sagelib in a standard Python way. It remains to fix docbuild, in fact, but that should not be a big problem, I gather. > > Leaving the spkg in place would mean that there is an expectation of > consistency. In particular, if the Sage_macOS project were really the one > and only user of the spkg for each Sage release then there would be no > motivation for changing it. So it would continue to work. I would be > willing to update the python version in the spkg and make sure it works (for > us, the only user). But even that would have an expectation of failure > because other aspects of the build system might change in ways which would > force a change to the layout of the spkg.. We probably would not know about > these until we ran into the failure. > >> >> To me it looks you would start out with building python much in the same way >> as how it now gets built by sage's python3 package and then in the ensuing >> environment you would end up building sage in a completely standard way, >> with the python requirement satisfied by the python build you have just >> completed. > > > I see no reason to expect that if I have a build of python somewhere on my > system, Sage would be able to use it. The "people install Python somehow" > standard does not specify what such a python would need to look like. At the > most basic level, there are two different ways of building Python on macOS. > There are "Framework" builds and "prefix" builds. Which of those meet the > requirements of the "people install Python somehow" standard? (I would > probably prefer the "Framework" build since I use a Framework for sage > itself.) In any case the crucial question is: how will the venv directory be > populated by the Sage build system? And how often will that change? When > will someone decide to add a symlink somewhere in the venv, because it works > fine on the build system, and thereby break the process of making sage > self-contained and relocatable. > You can just put *any* python build of your choice and its venv anywhere on the system, or let a tool like uv do it for you, so that all the PATHs etc are set, etc. >> Given that the python build would complete *before* you even get started on >> building sage, I would expect that this doesn't affect the fragility of the >> sage build at all. > > > Probably not on the build system. But the app needs to work on *any* system. > >> Are you worried that sage would have trouble recognizing the python build as >> valid for its purposes? > No. Because sagelib (with meson build) is already a proper Python package, which should just shut up and get installed as a Python package of a Python. :-) (And the rest is just a semi-optional environment coming on top of sagelib, pretty much orthogonal to what's in the macOS app) Let me stop here, as hopefully this point alleviates all of the Marc concerns. Dima > > Since there is no proposal for specifying what properties such a python build > must have in order for sage to recognize it, that is something to worry > about, yes. And I outlined some details of those worries above. The details > of how the venv gets populated make a big difference. To make Sage > self-contained and relocatable it is necessary to control the shebang line in > every script and the rpath in every executable (which usually means editing > the absolute library path to make it relative and adding an rpath). > >> >> If anything, I'd expect that if sage would not be providing its own python >> package, it would be forced to be more tolerant on what python to accept. > > > I wouldn't expect that. I would expect maximum intolerance - you must use > conda or you must use homebrew. (And you must agree, counter to all > experience, that building python is an extremely difficult task that must be > left to python-building-experts.) I guess the good part of maximum > intolerance is that it is consistent, at some level. Although there is no > reason to expect the python-building-experts to be consistent between > releases of their own products. > >> So I can see how giving away full control of python could make *sage* more >> fragile, but at the same time we are already relying on externally provided >> pythons in many cases and that doesn't seem to cause huge problems. > > > As I have tried to indicate, it is more about the details of how an > installation of python somewhere on the system will get used to populate the > venv directory within Sage. It will also be important to know when in the > build process that happens. (If changes are needed to that python > installation, is it necessary to interrupt the build process to make the > changes? Is that even possible when building in parallel?) It is also > important for the venv construction to be done consistently from one sage > release to the next, > to avoid having to do a trial and error redesign of our build system for > every release of Sage.. > > - Marc > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To view this discussion visit > https://groups.google.com/d/msgid/sage-devel/CALcZXRF9t3PtcxJ_6FPba%3D0ZSzh08JdZ%2BdC0dw7qm1ZpHLXbew%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq21KPzwAM8w-VDnoCToHDV7bGHsj0UV647MZ-rkDQ09sQ%40mail.gmail.com.
