Hello Riza Suminto, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22330
to look at the new patch set (#9).
Change subject: IMPALA-11980 (part 2): Fix absolute import issues for
impala_shell
......................................................................
IMPALA-11980 (part 2): Fix absolute import issues for impala_shell
Python 3 changed the behavior of imports with PEP328. Existing
imports become absolute unless they use the new relative import
syntax. This adapts the impala-shell code to use absolute
imports, fixing issues where it is imported from our test code.
There are several parts to this:
1. It moves impala shell code into shell/impala_shell.
This matches the directory structure of the PyPi package.
2. It changes the imports in the shell code to be
absolute paths (i.e. impala_shell.foo rather than foo).
This fixes issues with Python 3 absolute imports.
It also eliminates the need for ugly hacks in the PyPi
package's __init__.py.
3. This changes Thrift generation to put it directly in
$IMPALA_HOME/shell rather than $IMPALA_HOME/shell/gen-py.
This means that the generated Thrift code is rooted in
the same directory as the shell code.
4. This changes the PYTHONPATH to include $IMPALA_HOME/shell
and not $IMPALA_HOME/shell/gen-py. This means that the
test code is using the same import paths as the pypi
package.
With all of these changes, the source code is very close
to the directory structure of the PyPi package. As long as
CMake has generated the thrift files and the Python version
file, only a few differences remain. This removes those
differences by moving the setup.py / MANIFEST.in and other
files from the packaging directory to the top-level
shell/ directory. This means that one can pip install
directly from the source code. i.e. pip install $IMPALA_HOME/shell
This also moves the shell tarball generation script to the
packaging directory and changes bin/impala-shell.sh to use
Python 3.
Testing:
- Ran a regular core job with Python 2
- Ran a core job with Python 3 and verified that the absolute
import issues are gone.
Change-Id: Ica75a24fa6bcb78999b9b6f4f4356951b81c3124
---
M bin/impala-shell.sh
M bin/rat_exclude_files.txt
M bin/set-pythonpath.sh
M common/thrift/CMakeLists.txt
M shell/.gitignore
M shell/CMakeLists.txt
A shell/MANIFEST.in
R shell/README.md
M shell/gen_impala_build_version.sh
R shell/impala_shell/ImpalaHttpClient.py
R shell/impala_shell/TSSLSocketWithWildcardSAN.py
R shell/impala_shell/__init__.py
R shell/impala_shell/compatibility.py
R shell/impala_shell/cookie_util.py
R shell/impala_shell/exec_summary.py
R shell/impala_shell/impala_client.py
R shell/impala_shell/impala_shell.py
R shell/impala_shell/impala_shell_config_defaults.py
R shell/impala_shell/kerberos_util.py
R shell/impala_shell/option_parser.py
R shell/impala_shell/shell_exceptions.py
R shell/impala_shell/shell_output.py
R shell/impala_shell/thrift_printer.py
R shell/impala_shell/value_converter.py
D shell/packaging/MANIFEST.in
D shell/packaging/__init__.py
R shell/packaging/impala-shell
M shell/packaging/make_python_package.sh
R shell/packaging/make_shell_tarball.sh
R shell/requirements.txt
R shell/setup.py
M tests/beeswax/impala_beeswax.py
M tests/custom_cluster/test_hs2_fault_injection.py
M tests/custom_cluster/test_shell_interactive_reconnect.py
M tests/custom_cluster/test_thrift_socket.py
M tests/shell/test_cookie_util.py
M tests/shell/test_kerberos_util.py
M tests/shell/test_shell_client.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
41 files changed, 114 insertions(+), 168 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/22330/9
--
To view, visit http://gerrit.cloudera.org:8080/22330
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ica75a24fa6bcb78999b9b6f4f4356951b81c3124
Gerrit-Change-Number: 22330
Gerrit-PatchSet: 9
Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
