This is an automated email from the ASF dual-hosted git repository.
laszlog pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new 8dd935a98 IMPALA-13558: Workaround Python 2 tarfile issue by patching
tarfile.py
8dd935a98 is described below
commit 8dd935a98f1ee758bd8f2a03f9bd778a8a2f433e
Author: Joe McDonnell <[email protected]>
AuthorDate: Wed Nov 20 16:53:16 2024 -0800
IMPALA-13558: Workaround Python 2 tarfile issue by patching tarfile.py
Ubuntu 20.04 introduced a bug in their Python 2.7's tarfile
functionality with 2.7.18-1~20.04.5. See
https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/2089071
This breaks the Impala build with a message like
"tarfile.ReadError: invalid header".
The bug has an attached tarfile.py with a workaround for the
issue. This change bootstrap_system.sh and boostrap_build.sh to
detect the bad tarfile.py and replace it with the patched tarfile.py.
Since this is comparing the hash, this will become a no-op once
Ubuntu fixes the issue.
Testing:
- Ran a build on Ubuntu 20.04
Change-Id: I1d0691611cf53ae6dd1099b97f0aa15b450e0996
Reviewed-on: http://gerrit.cloudera.org:8080/22088
Reviewed-by: Quanlong Huang <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
bin/bootstrap_build.sh | 16 ++++++++++++++++
bin/bootstrap_system.sh | 16 ++++++++++++++++
2 files changed, 32 insertions(+)
diff --git a/bin/bootstrap_build.sh b/bin/bootstrap_build.sh
index 2d96ca989..475e71259 100755
--- a/bin/bootstrap_build.sh
+++ b/bin/bootstrap_build.sh
@@ -39,6 +39,22 @@ sudo -E apt-get --yes install g++ gcc git libsasl2-dev
libssl-dev make python-de
source /etc/lsb-release
+# Ubuntu 20's Python 2.7.18-1~20.04.5 version has a bug in its tarfile support.
+# If we detect the affected tarfile.py, download a patched version and
overwrite it.
+if [[ $DISTRIB_ID == Ubuntu && $DISTRIB_RELEASE == 20.04 ]]; then
+ if [[ -f /usr/lib/python2.7/tarfile.py ]]; then
+ TARFILE_PY_HASH=$(sha1sum /usr/lib/python2.7/tarfile.py | cut -d' ' -f1)
+ if [[ "${TARFILE_PY_HASH}" == "6e1a6d9ea2a535cbb17fe266ed9ac76eb5e27b89"
]]; then
+ TMP_DIR=$(mktemp -d)
+ pushd $TMP_DIR
+ wget -nv https://launchpadlibrarian.net/759546541/tarfile.py
+ sudo cp tarfile.py /usr/lib/python2.7/tarfile.py
+ popd
+ rm -rf $TMP_DIR
+ fi
+ fi
+fi
+
JDK_VERSION=8
if [[ $DISTRIB_RELEASE = 14.04 ]]
then
diff --git a/bin/bootstrap_system.sh b/bin/bootstrap_system.sh
index 9c28338d7..a8ed4fabc 100755
--- a/bin/bootstrap_system.sh
+++ b/bin/bootstrap_system.sh
@@ -243,6 +243,22 @@ ubuntu16 apt-get --yes install python python-dev
python-setuptools
ubuntu18 apt-get --yes install python python-dev python-setuptools
ubuntu20 apt-get --yes install python python-dev python-setuptools
+# Ubuntu 20's Python 2.7.18-1~20.04.5 version has a bug in its tarfile support.
+# If we detect the affected tarfile.py, download a patched version and
overwrite it.
+if [[ $UBUNTU20 == true ]]; then
+ if [[ -f /usr/lib/python2.7/tarfile.py ]]; then
+ TARFILE_PY_HASH=$(sha1sum /usr/lib/python2.7/tarfile.py | cut -d' ' -f1)
+ if [[ "${TARFILE_PY_HASH}" == "6e1a6d9ea2a535cbb17fe266ed9ac76eb5e27b89"
]]; then
+ TMP_DIR=$(mktemp -d)
+ pushd $TMP_DIR
+ wget -nv https://launchpadlibrarian.net/759546541/tarfile.py
+ sudo cp tarfile.py /usr/lib/python2.7/tarfile.py
+ popd
+ rm -rf $TMP_DIR
+ fi
+ fi
+fi
+
# Required by Kudu in the minicluster
ubuntu20 apt-get --yes install libtinfo5
ubuntu22 apt-get --yes install libtinfo5
