ji chen has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/23793 )
Change subject: IMPALA-14092 Part3: Enable predicate pushdown for paimon table. ...................................................................... IMPALA-14092 Part3: Enable predicate pushdown for paimon table. This patch mainly enable the predicate pushdown feature for paimon table. It will analyze the filter predicates for the given paimon table, and extract the predicates that can be pushed down to paimon. these predicate will be applied in-advance to paimon table to exclude unnecessary partitions,data files for the scanning. We also introduce a query option: PAIMON_PREDICATE_PUSHDOWN_ENABLED to turn on/off the predicate pushdown, by default it is is enabled. Testing: - Add test_paimon_ppd in test_paimon.py. - Add testPaimonPredicate in PlannerTest.java - Add unit test PaimonImpalaExprConverterTest.java Change-Id: Iee07fa35de8381121a20b2976d6424626d705483 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/analysis/LikePredicate.java M fe/src/main/java/org/apache/impala/planner/PaimonScanNode.java A fe/src/main/java/org/apache/impala/planner/paimon/ImpalaExprConverter.java A fe/src/main/java/org/apache/impala/planner/paimon/PaimonPartitionPredicateVisitor.java A fe/src/main/java/org/apache/impala/planner/paimon/PredicateExtractor.java M fe/src/test/java/org/apache/impala/planner/PlannerTest.java A fe/src/test/java/org/apache/impala/planner/paimon/PaimonImpalaExprConverterTest.java A testdata/workloads/functional-planner/queries/PlannerTest/paimon-predicates-disabled.test A testdata/workloads/functional-planner/queries/PlannerTest/paimon-predicates.test A testdata/workloads/functional-query/queries/QueryTest/paimon-ppd-profile.test M testdata/workloads/functional-query/queries/QueryTest/paimon-query.test M tests/query_test/test_paimon.py 16 files changed, 905 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/93/23793/4 -- To view, visit http://gerrit.cloudera.org:8080/23793 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iee07fa35de8381121a20b2976d6424626d705483 Gerrit-Change-Number: 23793 Gerrit-PatchSet: 4 Gerrit-Owner: ji chen <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: ji chen <[email protected]>
