Hello Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24045
to look at the new patch set (#4).
Change subject: IMPALA-14737 Part2: Add relaxed predicate pushdown for LIKE
patterns with suffix
......................................................................
IMPALA-14737 Part2: Add relaxed predicate pushdown for LIKE patterns with suffix
Part 1 (committed as 540a3784e) added basic LIKE predicate pushdown
to Iceberg for simple prefix patterns ('abc%'), exact matches ('exact'),
and escaped wildcards ('asd\%'). Patterns with literal content after
wildcards (e.g., 'prefix%suffix', 'd%d') were rejected and not pushed
down at all.
This Part 2 patch enhances the implementation with "relaxed predicate
pushdown" for patterns with suffix. Instead of rejecting these patterns
completely, we now:
1. Push down a relaxed prefix predicate to Iceberg (e.g., startsWith(
'prefix')) for partition/file pruning
2. Retain the full LIKE predicate (e.g., LIKE 'prefix%suffix') in the
scan node for Impala to evaluate on the surviving rows
Additionally, this patch adds support for simple LIKE patterns in DROP
PARTITION and SHOW FILES operations. Previously, any LIKE predicate would
fail in these operations. Now:
- Simple prefix patterns (e.g., 's LIKE d%') work correctly
- Patterns with suffix are rejected with clear error messages to prevent
unintended data loss (e.g., DROP PARTITION with 'd%d' would incorrectly
drop all partitions starting with 'd')
This provides significant performance benefits by leveraging Iceberg's
metadata filtering while maintaining query correctness.
Example behavior for `SELECT ... WHERE s LIKE 'd%d'`:
- Before: Pattern rejected, all 3/3 partitions scanned, no pruning
benefit
- After: startsWith('d') pushed to Iceberg -> 1/3 partitions, full LIKE
'd%d' evaluated by Impala on surviving rows -> correct results
Testing:
- Updated iceberg-like-pushdown.test with relaxed predicate tests
- Updated DROP PARTITION tests to include relaxed predicate tests
- Updated SHOW FILES tests to include relaxed predicate tests
Change-Id: I97c11362f098507fa440eafde3c35bbc6d7092b3
---
M fe/src/main/java/org/apache/impala/analysis/AlterTableDropPartitionStmt.java
M
fe/src/main/java/org/apache/impala/analysis/IcebergPartitionExpressionRewriter.java
M fe/src/main/java/org/apache/impala/analysis/ShowFilesStmt.java
M fe/src/main/java/org/apache/impala/analysis/ShowStatsStmt.java
M fe/src/main/java/org/apache/impala/common/IcebergPredicateConverter.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
M
testdata/workloads/functional-planner/queries/PlannerTest/iceberg-predicates.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-drop-partition.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-like-pushdown.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-show-files-partition.test
10 files changed, 408 insertions(+), 144 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/24045/4
--
To view, visit http://gerrit.cloudera.org:8080/24045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I97c11362f098507fa440eafde3c35bbc6d7092b3
Gerrit-Change-Number: 24045
Gerrit-PatchSet: 4
Gerrit-Owner: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
