Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23319
Change subject: IMPALA-14333: [part 1] Make all pytest compatible with Python3 ...................................................................... IMPALA-14333: [part 1] Make all pytest compatible with Python3 Running exhaustive tests with env var IMPALA_USE_PYTHON3_TESTS=true reveals some tests that require adjustment. This patch made such adjustment, which mostly revolves around encoding difference and string vs bytes type in Python3. Followings are the details: Moved one test case from delimited-latin-text.test to test_delimited_text.py for easier binary comparison. Recreate structs.parq (content inside functional_parquet.complextypes_structs) from functional_orc_def.complextypes_structs so that querying both of them will return the same result simply by setting option convert_legacy_hive_parquet_utc_timestamps=true. Setting timezone='Europe/Budapest' is not required anymore. Skip testing HBase at test_except and test_partitioned_top_n because HBase does not return return rows where alltypesagg.tinyint_col is null. Add _run_query_with_client() in test_ranger.py to allow reusing single Impala client for running several queries. Also make sure to close the clients when test is done. Added ImpalaTestSuite._convert_bytes() as utility function to decode bytes in Python3. This often needed when inspecting return value of subprocess.check_output() as string. Implement DataTypeMetaclass.__lt__ to substitute DataTypeMetaclass.__cmp__ that is ignored in Python3 (see https://peps.python.org/pep-0207/). Fix WEB_CERT_ERR difference in test_ipv6.py. Fix trivial integer parsing in test_restart_services.py. Fix various encoding issues in test_saml2_sso.py, test_shell_commandline.py, and test_shell_interactive.py. Switch parallel loops in test_stress_infra.py to simple loop that fail-proof in Python3. Reduce test dimension in test_date_queries.py and test_decimal_queries.py for json file format because table date_tbl and decimal_tiny is contrained only for json/none/none (functional_json). Switch to binary comparson in test_iceberg.py where needed. Removed unused vector argument and fix some trivial flake8 issues. Simplify create_impala_shell_executable_dimension to skip testing python2 impala-shell when IMPALA_USE_PYTHON3_TESTS=true. Switching IMPALA_USE_PYTHON3_TESTS to true by default will be addressed in next patch. Testing: Pass exhaustive tests with IMPALA_USE_PYTHON3_TESTS=true. Change-Id: I401a93b6cc7bcd17f41d24e7a310e0c882a550d4 --- M shell/impala_shell/impala_shell.py M testdata/ComplexTypesTbl/structs.parq M testdata/workloads/functional-query/queries/QueryTest/delimited-latin-text.test M testdata/workloads/functional-query/queries/QueryTest/inline-view.test M testdata/workloads/functional-query/queries/QueryTest/sort-complex.test M testdata/workloads/functional-query/queries/QueryTest/top-n-complex.test M tests/authorization/test_ranger.py M tests/common/impala_test_suite.py M tests/comparison/cluster.py M tests/comparison/db_types.py M tests/custom_cluster/test_ipv6.py M tests/custom_cluster/test_restart_services.py M tests/custom_cluster/test_saml2_sso.py M tests/custom_cluster/test_scratch_disk.py M tests/infra/test_stress_infra.py M tests/query_test/test_date_queries.py M tests/query_test/test_decimal_queries.py M tests/query_test/test_delimited_text.py M tests/query_test/test_iceberg.py M tests/query_test/test_nested_types.py M tests/query_test/test_queries.py M tests/shell/test_shell_commandline.py M tests/shell/test_shell_interactive.py M tests/shell/util.py M tests/statestore/test_statestore.py M tests/stress/test_update_stress.py M tests/util/shell_util.py 27 files changed, 333 insertions(+), 233 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/19/23319/1 -- To view, visit http://gerrit.cloudera.org:8080/23319 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I401a93b6cc7bcd17f41d24e7a310e0c882a550d4 Gerrit-Change-Number: 23319 Gerrit-PatchSet: 1 Gerrit-Owner: Riza Suminto <[email protected]>
