Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23319
to look at the new patch set (#2).
Change subject: IMPALA-14333: [part 1] Make all pytest compatible with Python3
......................................................................
IMPALA-14333: [part 1] Make all pytest compatible with Python3
Running exhaustive tests with env var IMPALA_USE_PYTHON3_TESTS=true
reveals some tests that require adjustment. This patch made such
adjustment, which mostly revolves around encoding difference and string
vs bytes type in Python3. Followings are the details:
Moved one test case from delimited-latin-text.test to
test_delimited_text.py for easier binary comparison.
Recreate structs.parq (content inside
functional_parquet.complextypes_structs) from
functional_orc_def.complextypes_structs so that querying both of them
will return the same result simply by setting option
convert_legacy_hive_parquet_utc_timestamps=true. Setting
timezone='Europe/Budapest' is not required anymore.
Skip testing HBase at test_except and test_partitioned_top_n because
HBase does not return return rows where alltypesagg.tinyint_col is null.
Add _run_query_with_client() in test_ranger.py to allow reusing single
Impala client for running several queries. Also make sure to close the
clients when test is done.
Added ImpalaTestSuite._convert_bytes() as utility function to decode
bytes in Python3. This often needed when inspecting return value of
subprocess.check_output() as string.
Implement DataTypeMetaclass.__lt__ to substitute
DataTypeMetaclass.__cmp__ that is ignored in Python3 (see
https://peps.python.org/pep-0207/).
Fix WEB_CERT_ERR difference in test_ipv6.py.
Fix trivial integer parsing in test_restart_services.py.
Fix various encoding issues in test_saml2_sso.py,
test_shell_commandline.py, and test_shell_interactive.py.
Switch parallel loops in test_stress_infra.py to simple loop that
fail-proof in Python3.
Reduce test dimension in test_date_queries.py and
test_decimal_queries.py for json file format because table date_tbl and
decimal_tiny is contrained only for json/none/none (functional_json).
Switch to binary comparson in test_iceberg.py where needed.
Removed unused vector argument and fix some trivial flake8 issues.
Simplify create_impala_shell_executable_dimension to skip testing
python2 impala-shell when IMPALA_USE_PYTHON3_TESTS=true.
Switching IMPALA_USE_PYTHON3_TESTS to true by default will be addressed
in next patch.
Testing:
Pass exhaustive tests with IMPALA_USE_PYTHON3_TESTS=true.
Change-Id: I401a93b6cc7bcd17f41d24e7a310e0c882a550d4
---
M shell/impala_shell/impala_shell.py
M testdata/ComplexTypesTbl/structs.parq
M
testdata/workloads/functional-query/queries/QueryTest/delimited-latin-text.test
M testdata/workloads/functional-query/queries/QueryTest/inline-view.test
M testdata/workloads/functional-query/queries/QueryTest/sort-complex.test
M testdata/workloads/functional-query/queries/QueryTest/top-n-complex.test
M tests/authorization/test_ranger.py
M tests/common/impala_test_suite.py
M tests/comparison/cluster.py
M tests/comparison/db_types.py
M tests/custom_cluster/test_ipv6.py
M tests/custom_cluster/test_restart_services.py
M tests/custom_cluster/test_saml2_sso.py
M tests/custom_cluster/test_scratch_disk.py
M tests/infra/test_stress_infra.py
M tests/query_test/test_date_queries.py
M tests/query_test/test_decimal_queries.py
M tests/query_test/test_delimited_text.py
M tests/query_test/test_iceberg.py
M tests/query_test/test_nested_types.py
M tests/query_test/test_queries.py
M tests/shell/test_shell_commandline.py
M tests/shell/test_shell_interactive.py
M tests/shell/util.py
M tests/statestore/test_statestore.py
M tests/stress/test_update_stress.py
M tests/util/shell_util.py
27 files changed, 333 insertions(+), 233 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/23319/2
--
To view, visit http://gerrit.cloudera.org:8080/23319
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I401a93b6cc7bcd17f41d24e7a310e0c882a550d4
Gerrit-Change-Number: 23319
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
