adriangb commented on code in PR #14567:
URL: https://github.com/apache/datafusion/pull/14567#discussion_r1951264573
##########
datafusion/physical-optimizer/src/pruning.rs:
##########
@@ -1710,6 +1717,56 @@ fn build_like_match(
Some(combined)
}
+// For predicate `col NOT LIKE 'foo%'`, we rewrite it as `(col_min NOT LIKE
'foo%' OR col_max NOT LIKE 'foo%')`. If both col_min and col_max have the
prefix foo, we skip the entire row group (as we can be certain that all data in
this row group has the prefix foo).
+fn build_not_like_match(
+ expr_builder: &mut PruningExpressionBuilder<'_>,
+) -> Option<Arc<dyn PhysicalExpr>> {
+ // col NOT LIKE 'prefix%' -> !(col_min LIKE 'prefix%' && col_max LIKE
'prefix%') -> (col_min NOT LIKE 'prefix%' || col_max NOT LIKE 'prefix%')
+
+ let min_column_expr = expr_builder.min_column_expr().ok()?;
+ let max_column_expr = expr_builder.max_column_expr().ok()?;
+
+ let scalar_expr = expr_builder.scalar_expr();
+
+ let pattern = extract_string_literal(scalar_expr)?;
+
+ let chars: Vec<char> = pattern.chars().collect();
+ for i in 0..chars.len() - 1 {
+ // Check if current char is a wildcard and is not escaped with
backslash
+ if (chars[i] == '%' || chars[i] == '_') && (i == 0 || chars[i - 1] !=
'\\') {
+ // Example: For pattern "foo%bar", the row group might include
values like
+ // ["foobar", "food", "foodbar"], making it unsafe to prune.
Review Comment:
Thanks Pitor.
It seems to me then that it is sensible to move forward with this. The
generated predicates are an internal implementation detail so if we want to
change this or add support for more complex cases in the future we can do so.
There is little downside to supporting only the `<constant-prefix>%` pattern
case for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
