rgbuilds commented on code in PR #25469:
URL: https://github.com/apache/datafusion/pull/25469#discussion_r4057567297
##########
datafusion/datasource-parquet/src/bloom_filter.rs:
##########
@@ -270,6 +270,114 @@ mod tests {
.unwrap()
}
+ #[tokio::test]
+ async fn test_tuple_in_bloom_pruning_preserves_correlation() -> Result<()>
{
Review Comment:
The unit test nicely covers both pruning and the crossed-pair correlation
case, while the SLT exercises the real Parquet scan path. Would it be valuable
to combine these in one end-to-end test with multiple row groups? For example,
the test could include:
1. A row group containing an exact tuple match.
2. A row group that statistics cannot eliminate but Bloom filters can,
verifying that it is counted as pruned in row_groups_pruned_bloom_filter.
3. A crossed-pair row group that satisfies the derived per-column guarantees
and therefore survives Bloom pruning, but produces no rows after evaluation of
the original tuple predicate.
This would verify that the derived per-column conditions are used only for
pruning, while the original tuple predicate is still applied for exact row
filtering.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
