Re: USES=python3.10+ and RUN_DEPENDS
Lexi Winter wrote: Moin Rahman: By default this will not be built in that case. If you want to build for python 3.10 you have to add USE_PYTHON=allflavors thanks (again). so for a normal port, is there any guidance on whether i should do this or not? is there a policy on this? this is a normal (desktop) application, not a library, so normally there would be no reason to build it for all Python versions. i don't see anything about this in https://wiki.freebsd.org/Python/PortsPolicy In order to specify USE_PYTHON=allflavors, all ports in the dependency chain have to specify it. This isn't really a policy issue, rather a technical one. -- Charlie Li ...nope, still don't have an exit line. OpenPGP_signature.asc Description: OpenPGP digital signature
Re: "py-" prefix shouldn't be needed for Python-based ports that are programs, not libraries
Yuri wrote:
I think that this section
https://wiki.freebsd.org/Python/PortsPolicy#Prefixing_.28py-.2A.29
should be amended:
Examples of compelling cases:
* The port is a program, and not a library.
There is no need to have flavors for Python ports that build only
programs and don't build libraries.
This distinction does not practically exist; any Python package, even if
primarily a program, can be specified and imported as a library in
another as a dependency. See meson, which had to grow flavours when
meson-python came about.
Putting -pyN as ${PKGNAMESUFFIX} when not the default flavour for those
cases can be considered however.
--
Charlie Li
...nope, still don't have an exit line.
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: "py-" prefix shouldn't be needed for Python-based ports that are programs, not libraries
Yuri wrote:
On 2/15/24 22:48, Charlie Li wrote:
This distinction does not practically exist; any Python package, even
if primarily a program, can be specified and imported as a library in
another as a dependency. See meson, which had to grow flavours when
meson-python came about.
This isn't true.
If the program has one main function and several helper
application-specific submodules - none of them can be used by any other
software as dependency because everything is application-specific.
Not every application-type package provides entry point(s) installed to
${BINDIR}. Further, Python package metadata doesn't care about any such
distinction, as dependencies are checked for presence within the same
distribution (same ${PYTHON_SITELIBDIR}). A further check can happen
with importing the "application", which again, has to be present in the
same distribution.
If the package has the description "Command line utility to xx" - this
likely means that this is just a command line application and nothing
more, and it shouldn't have the "py-" prefix.
Still have to account for cases where such ports may be built and used
against a non-default Python distribution. These cases are valid,
especially when a Python package does not support anything but at least
the latest Python version in our tree, but ${DEFAULT_VERSIONS} is not
said Python version. While the cluster builders don't bother unless
USE_PYTHON=allflavors is specified, those who build their own ports can
set BUILD_ALL_PYTHON_FLAVORS. PKGNAME clashes happen without some kind
of disambiguation.
--
Charlie Li
...nope, still don't have an exit line.
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: git: b9a3602bb3da - main - devel/py-pyproject_hooks: Update to 1.1.0
Po-Chuan Hsieh wrote: The branch main has been updated by sunpoet: URL: https://cgit.FreeBSD.org/ports/commit/?id=b9a3602bb3da661da313a970d4177a95a1233afc commit b9a3602bb3da661da313a970d4177a95a1233afc Author: Po-Chuan Hsieh AuthorDate: 2024-05-19 16:17:56 + Commit: Po-Chuan Hsieh CommitDate: 2024-05-19 17:15:43 + devel/py-pyproject_hooks: Update to 1.1.0 Changes:https://github.com/pypa/pyproject-hooks/blob/main/docs/changelog.rst https://pyproject-hooks.readthedocs.io/en/latest/changelog.html This is causing all setuptools builds under PEP-517 even upstream to fail: https://github.com/pypa/pyproject-hooks/issues/192 https://github.com/pypa/setuptools/issues/4333 So far the only fallout here is with devel/py-setuptools58 since it uses PEP-517 to build, but may want to consider rolling back pyproject_hooks for now in case it takes a bit for upstream to figure out. -- Charlie Li ...nope, still don't have an exit line. OpenPGP_signature.asc Description: OpenPGP digital signature
Re: git: b9a3602bb3da - main - devel/py-pyproject_hooks: Update to 1.1.0
Po-Chuan Hsieh wrote: IMHO, we do nót need to do anything right now as py-setuptools58 is not used by anything. It means we do not need to revert py-pyproject_hooks. Furthermore, I don't think current PY_SETUPTOOLS and devel/py-setuptools58 are needed in our ports tree. Since we could build almost every python port with pep517 (with minor fixes for the case in its name), we should always use devel/py-setuptools with correct version requirement rather than PY_SETUPTOOLS. In my case, there are 1,310 ports with USE_PYTHON=pep517 and only 3 ports with USE_PYTHON=distutils which are intentional for simplicity. We should encourage people to convert their ports to pep517 build. And we should preserve PY_SETUPTOOLS for conditional dependency on py-setuptools for Python 3.12+. setuptools needs to build itself under PEP-517, which I will handle as part of the next update when we are ready. We cannot build it the current way as we have it in the tree (using distutils as a hack) when we finally land Python 3.12 as distutils no longer exists there. pyproject-hooks needs to work in order to build setuptools itself under PEP-517. As long as there are any ports with USE_PYTHON=distutils, setuptools58 has to exist as the distutils method is explicitly not supported past this version, even if it works by accident. -- Charlie Li ...nope, still don't have an exit line. OpenPGP_signature.asc Description: OpenPGP digital signature
Re: devel/py-setuptools-scm does not build, if devel/py-setuptools_scm7 installed
Rainer Hurling wrote: This happens to devel/py-setuptools-scm, if devel/py-setuptools_scm7 is installed: Any ideas? I really need both installed because of dependencies, scm v7.1.0 and v8.0.4. This is intentional. Additionally, both should not have *any* runtime consumers unless they are themselves are solely build dependencies for others. I should really formally put this in writing in a better place than mailing lists and IRC, but building Python packages outside of isolated environments like jails or virtualenvs is not really supported anymore, not even in the greater Python community. -- Charlie Li ...nope, still don't have an exit line. OpenPGP_signature.asc Description: OpenPGP digital signature
Re: PYTHON_EXT_SUFFIX value for shared libraries, pyc files and others
Michael Zhilin wrote:
Hi,
I have a problem with PYTHON_EXT_SUFFIX when I try to build ports
(including python) with the option WITH_DEBUG.
I want to build subset of ports with WITH_DEBUG by poudriere, but
several python package builds have failed on phase "package" due to
missing artifacts. The root cause of failure is that ports uses same
macro PYTHON_EXT_SUFFIX for shared libraries and pyc files, but actual
file names have different suffixes:
- pyc files have the suffix ".cpython-39" as expected.
- so files have the suffix ".cpython-39d".
The "d" is ${PYTHON_ABIVER} and actual ABI flags of Python build.
According to Python specifications, all pyc files must have a suffix
without ABI flags. Shared libraries have suffixes with ABI flags, but I
didn't find any spec about it.
PEP-3147 for byte compilation files (.pyc) and PEP-3149 for extension
module files (.so).
The first idea came to me is to add an extra suffix PYTHON_EXTSO_SUFFIX
with value ".cpython-${PYTHON_SUFFIX}${PYTHON_ABIVER}". It's easy to add
it and replace all occurrences in pkg-plist and Makefile, but it may be
hard to maintain it in future.
Looks like some unfinished business from PR 252057. In theory, should be
able to append ${PYTHON_ABIVER} directly to ${PYTHON_EXT_SUFFIX}, since
${PYTHON_CMD}-config --abiflags returns the correct flags even if empty.
Because the ABI changes when you build Python itself under
--with-pydebug (WITH_DEBUG or DEBUG option), you cannot build only a
subset of Python ports under WITH_DEBUG; it's an all-or-nothing situation.
Byte compilation files are not compiled or installed when the port is
built under PEP-517. Compiling and installing byte compilation files
with distutils/setuptools at build/stage-time will be deprecated. A new
install-time hook is under development to replace this functionality,
which has the added benefit of less pkg-plist churn and framework
maintenance.
--
Charlie Li
…nope, still don't have an exit line.
OpenPGP_signature
Description: OpenPGP digital signature
Re: CHEESESHOP or GH
Einar Bjarni Halldórsson wrote: So, we can either continue fetching the dist from PyPI and then have no tests, or we can fetch from Github, including the tests. The guidelines say: Examples where temporarily using an alternate MASTER_SITES is warranted: Some files are not correctly packaged or included at all in the PyPI sdist, such as licenses, test suite files or data. Submit an upstream issue or PR in these cases. A source distribution (sdist) has not been uploaded yet In these cases, issues or pull request SHOULD be created upstream to rectify the problem, and MASTER_SITES switched to CHEESESHOP when the change is released in a later version. In this case, the upstream devs have flat out said they're not going to include the tests. Is it better to switch MASTER_SITE to github and run the tests or stay with PyPI and skip the tests? .einar [1] https://github.com/proxmoxer/proxmoxer/issues/126 I commented. This guideline isn't specific to python@ per se, but something that applies to all ports: proper release distributions (usually source tarballs) are preferred over source code management tags. For this port, you can go either way, but if you choose to fetch from GitHub, make sure to add a comment in the Makefile explaining the situation (and maybe reference the upstream issue). It's unfortunate when individual projects choose to disregard proper release engineering practices. Part of our job as port maintainers is to serve as that check-and-balance such that if projects want their stuff in software repositories, they have to put some consideration as well. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: textproc/py-pyphen: Change build to pep517
Kurt Jaeger wrote: It looks like it now requires devel/py-flit and/or devel/py-flit-core. Those are ports that are built with USE_PYTHON=distutils Can I easily change those to USE_PYTHON=pep517 or is there some magic involved ? For these specifically, there's some additional magic involved, due these being bootstrap components. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: textproc/py-pyphen: Change build to pep517
Kurt Jaeger wrote: Thanks. I'm testbuilding print/py-weasyprint and it fails with this: =>> Checking for filesystem violations... done =>> Error: Filesystem touched during build: extra: usr/local/lib/python3.9/site-packages/flit_core/__pycache__ extra: usr/local/lib/python3.9/site-packages/flit_core/vendor/__pycache__ extra: usr/local/lib/python3.9/site-packages/flit_core/vendor/tomli/__pycache__ =>> Cleaning up wrkdir I test on 12.3amd, 12.4amd, 13.1amd, 14.0i386 and 14.0amd. It happens on 12.4a and 14.0i -- not on the other ones ? So I guess it has to do with flit-core ? I'm lost on why this is happening. Any ideas ? Try this: https://reviews.freebsd.org/D38429 -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: devel/py-setuptools: Any plan to update to 65.x
Rainer Hurling wrote: Are there any plans for the near future to update devel/py-setuptools to at least v65.0.0? Yes, when USE_PYTHON=distutils is overhauled to, among other things, use devel/py-setuptools58. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: devel/py-setuptools_scm >= 7.0.0
Matthew Seaman wrote: Currently the ports has devel/py-setuptools_scm at version 6.4.2, but I'm seeing some python packages now wanting setup_tools_scm >= 7.0.0 Now, setuptools_scm is widely depended upon by many python packages, and updating it over a major version boundary would probably cause all sorts of problems. So, does anyone have any objections to my creating a devel/py-setuptools_scm7 port for the latest version, currently 7.1.0? Hard object. The delta is minor at best for ports, as we're only using it to "dynamically" fill in version metadata on the Python package side. I have 7.1.0 working locally, PEP-517-ified and all. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: devel/py-setuptools_scm >= 7.0.0
Moin Rahman wrote:
Apart from creating another port I would also advise whether if you can
add a feature knob for setuptools_scm in python.mk.
We don't need more feature knobs in python.mk, especially not those
related to setuptools.
On the other hand, once devel/py-setuptools58 becomes fully operational
for USE_PYTHON=distutils, setuptools itself cannot remain in
setuptools_scm's RUN_DEPENDS, since not every setuptools_scm consumer is
PEP-517-compliant (build-depends would try to install both setuptools
packages, which conflict). So a dependency shorthand variable, and not a
feature knob, encapsulating both setuptools{,_scm} can be considered.
Still have reservations even for this however.
--
Charlie Li
…nope, still don't have an exit line.
OpenPGP_signature
Description: OpenPGP digital signature
Re: textproc/py-sphinx causes filesystem violations
Thierry Thomas wrote: I’m trying to upgrade a simple port, math/py-fpylll; it requires textproc/py-sphinx to build the doc, and now it fails in poudriere with these errors: =>> Checking for filesystem violations... done =>> Error: Filesystem touched during build: extra: [bunch of bytecode directories] build of math/py-fpylll | py39-fpylll-0.5.9 ended at Thu Mar 30 14:32:44 CEST 2023 build time: 00:02:07 !!! build failure encountered !!! [00:10:22] Error: Build failed in phase: build_fs_violation How to solve this problem? Easiest way is to stop building as root (set BUILD_AS_NON_ROOT). Python writes bytecode on import by default if it does not already exist and the executing user has write privileges to the location(s). Additionally, D39306 and PR 270511 tackle this without introducing more bytecode, a known filesystem pollutant. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: py-setuptools
John T. Darrah wrote: Hello: I am writing you because I would like to inquire about a port you maintain: py-setuptools. I get the following error when trying to build hiawatha, which py-setuptools is a requirement for: Please see the PR regarding updating this, the situation is complicated and requires careful coordination. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: py-setuptools
On 19/05/2023 02:37, John T. Darrah wrote: Charlie: Is there anything I can do to help the situation? I'm familiar with ports enough that I think I can be useful. Not really. There really isn't much in a technical sense but a logistical/mechanical one. Everything has to be done in a specific order. -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: [Bug 262906] net-mgmt/py-pysnmp: abandonned source used
John W. O'Brien wrote: For net-mgmt/py-pysmi, I also had to patch pyproject.toml [2] to match the port name [3]. [2] https://github.com/lextudio/pysnmp/blob/v5.0.28/pyproject.toml#L2 [3] https://cgit.freebsd.org/ports/diff/net-mgmt/py-pysmi/files/patch-pyproject.toml?id=718622a56caf647e137c7896197e0d6b17dedddb Please don't do that unless you are performing name normalisation [0]. While this case involves the unfortunate death of the original author and maintainer, changing the metadata in this manner is still a lapse in software supply chain security/integrity, considering the wider Python package ecosystem's (most visibly in PyPI) chequered history in this area. [0] https://packaging.python.org/en/latest/specifications/name-normalization/ -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
Re: [Bug 262906] net-mgmt/py-pysnmp: abandonned source used
John W. O'Brien wrote: Do we expect that Lextudio's PEP 541 request to take over the PyPI package names is going to be denied? If not, it means we expect the upstream source names to change to match the current port names, and renaming now will require renaming again later. I struggle to see how incurring that churn serves the interests of "software supply chain security/integrity" at all. The decision to use the Lextudio source or not is the consequential one. The PEP-541 request [0] is irrelevant until the Python package name is formally renamed from pysnmp-lextudio. In this specific case, it seems that the process is stalled due to various concerns raised. In general, metadata inconsistencies, particularly typosquatting, still can happen too easily on PyPI, and causes more than just negative technical effects. While the possible churn is unfortunate, we need to maintain our due diligence in ensuring consistency in this area. [0] https://github.com/pypi/support/issues/2420 (for others following along) -- Charlie Li …nope, still don't have an exit line. OpenPGP_signature Description: OpenPGP digital signature
