Can't run setup.py offline due to setup_requires - setup.py calls home
Hi, I'm trying to build a debian package in offline environment (build server). To build this package, I need to ship all python dependencies as source packages and build them there. This is no problem for all, except one package that has build-time dependencies: Automat-0.70. debian/rules calls this pip to install all requirements from local package collection: pip3 install --log=... --no-cache --no-index --find-links=pypi --no-binary=":all:" -r requirements.txt Directory pypi contains ALL dependencies required for build. This works ok when build server has connection to network, but fails for offline builds. I pinpointed the problem to package Automat, that specifies some dependencies via setup_requires=[...]: setup( ..., setup_requires=[ 'setuptools-scm', 'm2r', ], ... ) Trying to build Automat locally by calling setup.py fails immediately in my offline environment: $ python setup.py ... distutils.errors.DistutilsError: Download error for https://files.pythonhosted.org/packages/39/e7/9fae11a45f5e1a3a21d8a98d02948e597c4afd7848a0dbe1a1ebd235f13e/m2r-0.2.1.tar.gz#sha256=bf90bad66cda1164b17e5ba4a037806d2443f2a4d5ddc9f6a5554a0322aaed99: [Errno 111] Connection refused Is there any way to stop Distutils from calling home? Best regards, Chris Narkiewicz -- https://mail.python.org/mailman/listinfo/python-list
Re: Can't run setup.py offline due to setup_requires - setup.py calls home
On 11/02/2019 15:57, Ben Finney wrote: > All of the build dependencies, *including* the ones specified in > ‘setup_requires’? Yes. easy_install simply doesn't look there. If I provide ~/.pydistutils.cfg with a path to find_links, it works ok. Config file in $HOME however is no-go for a CI or build servers, as I have no control over automated build environment (launchpad.net in this case). > To avoid the Setuptools bug, the PyPA recommends dropping the > ‘setup_requires’ option and instead specifying build dependencies in a > PEP 518 formatted metadata file > https://github.com/pypa/setuptools/issues/293>. Ok, I took Automat-0.7.tar.gz package and I modified it: 1) removed setup_requires 2) added pyproject.toml with content: [build-system] requires = ["setuptools-scm", "m2r"] 3) package is dropped into pypi directory with all dependencies. However, when I try to install Automat from source, it doesn't work. (venv)$ pip3 install --no-index --find-links=pypi --no-binary=':all:' --no-cache Automat I see that those build-time dependencies are not installed and build complains about missing scm and m2r packages. Is there any extra step I have to take? Best regards, Chris signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Can't run setup.py offline due to setup_requires - setup.py calls home
On 11/02/2019 19:30, Chris Narkiewicz via Python-list wrote: > Is there any extra step I have to take? Ok, I'll respond to myself, as this was really silly. Debian ships hopelessly obsolete pip 9.PEP 518 is supported in pip 10+. Cheers, Chris signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
pydistutils.cfg injection
Hi,
I'm trying to build a Python application in Launchpad and I'm currently
having some issues with distutils.
The build on Launchpad is constrained by 2 things:
1) the builder is isolated from network.
2) must be fully open source (can't ship pre-built binaries).
I vendored in my PyPI dependencies in form of source packages and I
pip install them:
pip instal --find-links=/my/own/vendored/pypi/packages
This works like a charm until pip tries to build a package that has
build-time dependencies and does not conform to PEP 518:
https://www.python.org/dev/peps/pep-0518/
(which is majority of packages, sadly).
Then, easy_install kicks in and it tries to fetch from network, as it
does not use pip's --find-links option.
Thanks to some earlier answer on this group, I found pydistutils.cfg
workaround for this issue, but this approach has some very annoying
limitation - cfg file is read from 3 pre-defined locations:
1) ${PWD}
2) distutils directory (which is inside system installed Python
distribution)
3) ${HOME}
All 3 are no-go in most automated environments.
I checked distutils source code and I have a proposal.
In ${PYTHON_INSTALL}/distutils/dist.py there is a function:
class Distribution:
def find_config_files(self):
...snip...
if self.want_user_cfg:
user_file = os.path.join(os.path.expanduser('~'), user_filename)
if os.path.isfile(user_file):
files.append(user_file)
...snip...
return files
How about adding an environment variable, let's say
DISTUTILS_CONFIG_PATH, and resolving it alongside ~?
For now, I need to re-define my ${HOME} to simulate this behaviour. This
hack might however break some other code that depends on proper ${HOME}
value, so while working for me, could still be an issue for somebody else.
What do you think? Is such change feasible? I can make a patch - how to
submit it?
Best regards,
Chris Narkiewicz
signature.asc
Description: OpenPGP digital signature
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to execute shell command in Python program?
Madhavan Bomidi wrote: > import subprocess > subprocess.call(['./opac'],shell=True) subprocess.call(['./opac', "my-input.inp"], shell=True) The array takes command with a list of arguments. This way you don't need to do space escaping and other Jujitsu gimmicks. If you want to feed the command from stdin, this becomes a bit more complicated as you need to start a subprocess and use PIPE to feed it. Cheers, Chris Narkiewicz signature.asc Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
