Adam Roberts created HADOOP-17522:
-------------------------------------
Summary: Looking to build Hadoop but start-build-env.sh errors
Key: HADOOP-17522
URL: https://issues.apache.org/jira/browse/HADOOP-17522
Project: Hadoop Common
Issue Type: Bug
Components: build
Reporter: Adam Roberts
Hey everyone, we're trying to build Hadoop from source and in building.txt we
see
The easiest way to get an environment with all the appropriate tools is by means
of the provided Docker config.
This requires a recent version of docker (1.4.1 and higher are known to work).
On Linux / Mac:
Install Docker and run this command:
$ ./start-build-env.sh
However, and admittedly this is on 3.1.4, which I want to build -
{{Step 26/36 : RUN pip2 install configparser==4.0.2 pylint==1.9.2}}{{
---> Running in acf678bc8314}}{{Collecting configparser==4.0.2}}{{ Downloading
https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl}}{{Collecting
pylint==1.9.2}}{{ Downloading
https://files.pythonhosted.org/packages/f2/95/0ca03c818ba3cd14f2dd4e95df5b7fa232424b7fc6ea1748d27f293bc007/pylint-1.9.2-py2.py3-none-any.whl
(690kB)}}{{Collecting singledispatch; python_version < "3.4" (from
pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/c5/10/369f50bcd4621b263927b0a1519987a04383d4a98fb10438042ad410cf88/singledispatch-3.4.0.3-py2.py3-none-any.whl}}{{Collecting
isort>=4.2.5 (from pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/a2/f7/f50fc9555dc0fe2dc1e7f69d93f71961d052857c296cad0fb6d275b20008/isort-5.7.0.tar.gz
(169kB)}}{{Collecting astroid<2.0,>=1.6 (from pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/8b/29/0f7ec6fbf28a158886b7de49aee3a77a8a47a7e24c60e9fd6ec98ee2ec02/astroid-1.6.6-py2.py3-none-any.whl
(305kB)}}{{Collecting backports.functools-lru-cache; python_version == "2.7"
(from pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/da/d1/080d2bb13773803648281a49e3918f65b31b7beebf009887a529357fd44a/backports.functools_lru_cache-1.6.1-py2.py3-none-any.whl}}{{Collecting
mccabe (from pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl}}{{Collecting
six (from pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl}}{{Collecting
enum34>=1.1.3; python_version < "3.4" (from
astroid<2.0,>=1.6->pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/6f/2c/a9386903ece2ea85e9807e0e062174dc26fdce8b05f216d00491be29fad5/enum34-1.1.10-py2-none-any.whl}}{{Collecting
wrapt (from astroid<2.0,>=1.6->pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/82/f7/e43cefbe88c5fd371f4cf0cf5eb3feccd07515af9fd6cf7dbf1d1793a797/wrapt-1.12.1.tar.gz}}{{Collecting
lazy-object-proxy (from astroid<2.0,>=1.6->pylint==1.9.2)}}{{ Downloading
https://files.pythonhosted.org/packages/69/fc/79080e582c17b7f45ac0bb4a13b0260992dcc2519b5a29ac6cde3e81b6fa/lazy_object_proxy-1.5.2-cp27-cp27mu-manylinux1_x86_64.whl
(52kB)}}{{Building wheels for collected packages: isort, wrapt}}{{ Running
setup.py bdist_wheel for isort: started}}{{ Running setup.py bdist_wheel for
isort: finished with status 'error'}}{{ Complete output from command
/usr/bin/python -u -c "import setuptools,
tokenize;__file__='/tmp/pip-build-u9WX19/isort/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
bdist_wheel -d /tmp/tmp60t8Japip-wheel- --python-tag cp27:}}{{
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution
option: 'python_requires'}}{{ warnings.warn(msg)}}{{ running
bdist_wheel}}{{ running build}}{{ running build_py}}{{ creating build}}{{
creating build/lib.linux-x86_64-2.7}}{{ creating
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/_version.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/settings.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/logo.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/hooks.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/wrap_modes.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/sections.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/parse.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/utils.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/setuptools_commands.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/files.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/format.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/wrap.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/place.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/profiles.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/__main__.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/output.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/core.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/exceptions.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/pylama_isort.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/literal.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/comments.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/io.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/identify.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/main.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/api.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/__init__.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/sorting.py ->
build/lib.linux-x86_64-2.7/isort}}{{ creating
build/lib.linux-x86_64-2.7/isort/_future}}{{ copying
isort/_future/_dataclasses.py -> build/lib.linux-x86_64-2.7/isort/_future}}{{
copying isort/_future/__init__.py ->
build/lib.linux-x86_64-2.7/isort/_future}}{{ creating
build/lib.linux-x86_64-2.7/isort/_vendored}}{{ creating
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/ordered.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/tz.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/decoder.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/encoder.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/__init__.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ creating
build/lib.linux-x86_64-2.7/isort/deprecated}}{{ copying
isort/deprecated/finders.py -> build/lib.linux-x86_64-2.7/isort/deprecated}}{{
copying isort/deprecated/__init__.py ->
build/lib.linux-x86_64-2.7/isort/deprecated}}{{ creating
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py38.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py39.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py35.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py37.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/all.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py3.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py27.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py2.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py36.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/__init__.py
-> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ error: can't copy
'isort/stdlibs': doesn't exist or not a regular file}}{{ }}{{
----------------------------------------}}{{ Failed building wheel for
isort}}{{ Running setup.py clean for isort}}{{ Running setup.py bdist_wheel
for wrapt: started}}{{ Running setup.py bdist_wheel for wrapt: finished with
status 'done'}}{{ Stored in directory:
/root/.cache/pip/wheels/b1/c2/ed/d62208260edbd3fa7156545c00ef966f45f2063d0a84f8208a}}{{Successfully
built wrapt}}{{Failed to build isort}}{{Installing collected packages:
configparser, six, singledispatch, isort, enum34, wrapt,
backports.functools-lru-cache, lazy-object-proxy, astroid, mccabe, pylint}}{{
Running setup.py install for isort: started}}{{ Running setup.py install for
isort: finished with status 'error'}}{{ Complete output from command
/usr/bin/python -u -c "import setuptools,
tokenize;__file__='/tmp/pip-build-u9WX19/isort/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record /tmp/pip-Pa5mnZ-record/install-record.txt
--single-version-externally-managed --compile:}}{{
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution
option: 'python_requires'}}{{ warnings.warn(msg)}}{{ running
install}}{{ running build}}{{ running build_py}}{{ creating build}}{{
creating build/lib.linux-x86_64-2.7}}{{ creating
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/_version.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/settings.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/logo.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/hooks.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/wrap_modes.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/sections.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/parse.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/utils.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/setuptools_commands.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/files.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/format.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/wrap.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/place.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/profiles.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/__main__.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/output.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/core.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/exceptions.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/pylama_isort.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/literal.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/comments.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/io.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/identify.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/main.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/api.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/__init__.py ->
build/lib.linux-x86_64-2.7/isort}}{{ copying isort/sorting.py ->
build/lib.linux-x86_64-2.7/isort}}{{ creating
build/lib.linux-x86_64-2.7/isort/_future}}{{ copying
isort/_future/_dataclasses.py -> build/lib.linux-x86_64-2.7/isort/_future}}{{
copying isort/_future/__init__.py ->
build/lib.linux-x86_64-2.7/isort/_future}}{{ creating
build/lib.linux-x86_64-2.7/isort/_vendored}}{{ creating
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/ordered.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/tz.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/decoder.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/encoder.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ copying
isort/_vendored/toml/__init__.py ->
build/lib.linux-x86_64-2.7/isort/_vendored/toml}}{{ creating
build/lib.linux-x86_64-2.7/isort/deprecated}}{{ copying
isort/deprecated/finders.py -> build/lib.linux-x86_64-2.7/isort/deprecated}}{{
copying isort/deprecated/__init__.py ->
build/lib.linux-x86_64-2.7/isort/deprecated}}{{ creating
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying isort/stdlibs/py38.py
-> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying
isort/stdlibs/py39.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/py35.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/py37.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/all.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/py3.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/py27.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/py2.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
copying isort/stdlibs/py36.py ->
build/lib.linux-x86_64-2.7/isort/stdlibs}}{{ copying
isort/stdlibs/__init__.py -> build/lib.linux-x86_64-2.7/isort/stdlibs}}{{
error: can't copy 'isort/stdlibs': doesn't exist or not a regular file}}{{
}}{{ ----------------------------------------}}{{Command "/usr/bin/python -u
-c "import setuptools,
tokenize;__file__='/tmp/pip-build-u9WX19/isort/setup.py';exec(compile(getattr(tokenize,
'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))"
install --record /tmp/pip-Pa5mnZ-record/install-record.txt
--single-version-externally-managed --compile" failed with error code 1 in
/tmp/pip-build-u9WX19/isort/}}
You are using pip version 8.1.1, however version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
If I add that into the Dockerfile, I get
{{}}{{ ---> Running in 5d4c46c61d29}}{{Traceback (most recent call last):}}{{
File "/usr/local/bin/pip2", line 7, in <module>}}{{ from
pip._internal.cli.main import main}}{{ File
"/usr/local/lib/python2.7/dist-packages/pip/_internal/cli/main.py", line 60}}{{
sys.stderr.write(f"ERROR: \{exc}")}}{{
^}}{{SyntaxError: invalid syntax}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
