(from a different thread) On Fri, 8 Dec 2017, Richard Laager via devel wrote:
> I agree that your system does not have /usr/local in its sys.path by > default. [...] > 1) Ignore prefix and install to /usr. This is Fred Wright's solution and > is what is seemingly the point of fix_python_config.py. I think this is > wrong; prefix should be respected unless explicitly overridden by > --pythondir. Ideally, yes, but encumbered by the absence of the /usr/local/... path in the default sys.path on some Linux platforms. > 2) Set PYTHONPATH. Fred Wright said there was a consensus that this is > undesirable. There are a number of reasons why PYTHONPATH is an undesirable solution: 1) Environment variables are not managed in a structured manner, particularly in cases such as this where they need to be set up for *all* users on the system. Leaving the whole issue as an "exercise for the reader" is obnoxious, and the formerly existing instructions for using PYTHONPATH weren't even correct. Setting this up automatically is hacky, platform-specific, and sometimes site-specific. Updating or removing (in the event of an uninstall) environment variable setups reliably is even messier. 2) Setting up environment variables via shell startup scripts doesn't work in contexts where shell startup scripts aren't processed, such as ssh remote commands. 3) There are some contexts where environment variables are suppressed for security reasons. That's admittedly not too likely in this particular case, but nevertheless not impossible, especially when one allows for the possibility of third-party programs relying on the ntp libraries. 4) Most Python library locations are versioned, but PYTHONPATH is not. That can be either a plus or a minus, depending on the specifics, but when compiled extensions (which are only valid for the Python version they're built for) are involved, bypassing versioning is usually at best useless and sometimes downright nasty. 5) Any use of a persistent PYTHONPATH applies to *all* Python programs on the system, not just the ones that need it. The combination of this with #4 means that version-related screwups may be inflicted on *other* Python programs via PYTHONPATH. In short, relying on a persistent PYTHONPATH is like relying on a persistent LD_LIBRARY_PATH, only worse. > 3) Create a .pth file in the appropriate place, which is somewhere under > /usr. There are probably existing ones: I did investigate .pth files when this issue first arose. At the time, based on a strict interpretation of the requirements, I concluded that .pth files don't help. In the cases of interest, there are no locations within /usr/local where Python looks for .pth files, so if the goal is to avoid writing *anything* outside /usr/local while placing libraries under /usr/local (and not in any default sys.path location), then there's an insurmountable chicken-and-egg problem with .pth files. That being said, it seems upon reflection that busting PREFIX for a .pth file is a lot less onerous than busting it for the libraries themselves, particularly given that one can use a naming convention that avoids collisions (e.g., ntp-usr-local.pth in the /usr/local case). So, assuming that this degree of PREFIX violation is acceptable (as suggested above), then installing a .pth file looks like a viable solution. There are still a few details to be worked out: 1) Where to place the .pth file itself. The result of the unprefixed get_python_lib() looks like a plausible candidate, but that needs to be verified. 2) There are certain cases (notably OSX) where the normal placement of Python libraries is quite "unusual", and this isn't reflected in the behavior of the prefixed get_python_lib() (essentially due to dumbness of the latter). Meanwhile, the result of the unprefixed get_python_lib() may better conform to normal practice on the platform, and may also conform to PREFIX, albeit semi-coincidentally. In such cases, using the unprefixed get_python_lib() is preferable for consistency with the platform's normal practice (while still honoring PREFIX), but the exact condition for doing this needs to be worked out. 3) It would be desirable to limit the installation of .pth files to cases where they're actually needed (hopefully eventually becoming "nowhere" as more Python setups get their acts together). In principle, this means checking whether the chosen location is in the default sys.path, but that can't be determined reliably at configure time since nonexistent directories are omitted from sys.path, and Python doesn't publish a "potential" sys.path. The current workaround for that issue is a note in the documentation that the relevant directory may need to be created in advance, but that's rather kludgy and unreliable. Fortunately, for this particular usage, the decision can be deferred until install time, and if it's done as a post-install hook, then the directory will already have been created if needed, and checking sys.path will have the desired effect without any pre-creation needed. That does of course mean that the .pth file won't be a normal install product, and thus won't be automatically tracked by waf, and will need to be handled explicitly for uninstall as well. 4) It might be possible to fix PYTHONARCHDIR in the process, so that it can actually be used as intended (as the install location for compiled extensions). If there are no objections to this approach, I'll go ahead and implement and test it. > Given that this is fixed in the distro, I think the right trade-off is > to recommend #3, either in documentation and/or in a warning from waf if > the configured pythondir is not in sys.path. I would like to see the > fix_python_config.py code removed, as it is creating new breakage. Removing FixConfig altogether isn't the right thing in any case, since it's designed to be a general-purpose container for fixes to things that waf gets wrong about Python. E.g., there's another, subtler problem (related to the fundamental brokenness of python-config) which I don't yet have a completed solution for, but which will need more FixConfig code to fix. Fred Wright _______________________________________________ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
