jayzhan211 commented on code in PR #26105:
URL: https://github.com/apache/datafusion/pull/26105#discussion_r4306003104
##########
datafusion/sqllogictest/test_files/dynamic_filter_pushdown_config.slt:
##########
@@ -104,7 +104,8 @@ Plan with Metrics
03)----ProjectionExec: expr=[id@0 as id, value@1 as v, value@1 + id@0 as
name], metrics=[output_rows=10, <slt:ignore>]
04)------FilterExec: value@1 > 3, metrics=[output_rows=10, <slt:ignore>,
selectivity=100% (10/10)]
05)--------RepartitionExec: partitioning=RoundRobinBatch(4),
input_partitions=1, metrics=[output_rows=10, <slt:ignore>]
-06)----------DataSourceExec: file_groups={1 group:
[[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/test_data.parquet]]},
projection=[id, value], file_type=parquet, predicate=value@1 > 3 AND
DynamicFilter [ value@1 IS NULL OR value@1 > 800 ],
dynamic_rg_pruning=eligible, pruning_predicate=value_null_count@1 !=
row_count@2 AND value_max@0 > 3 AND (value_null_count@1 > 0 OR
value_null_count@1 != row_count@2 AND value_max@0 > 800),
required_guarantees=[], metrics=[output_rows=10, elapsed_compute=<slt:ignore>,
output_bytes=80.0 B, files_ranges_pruned_statistics=1 total → 1 matched,
row_groups_pruned_statistics=1 total → 1 matched -> 1 fully matched,
page_index_pages_pruned=0 total → 0 matched,
page_index_pages_skipped_by_fully_matched=1, limit_pruned_row_groups=0 total →
0 matched, bytes_processed=1147.0 B, bytes_scanned=210.0 B,
page_index_load_skipped=1, row_groups_pruned_dynamic_filter=0,
metadata_load_time=<slt:ignore>, scan_efficien
cy_ratio=18.31% (210/1.15 K)]
+06)----------DataSourceExec: file_groups={1 group:
[[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/dynamic_filter_pushdown_config/test_data.parquet]]},
projection=[id, value], file_type=parquet, predicate=value@1 > 3 AND
DynamicFilter [ value@1 IS NULL OR value@1 > 800 ],
dynamic_rg_pruning=eligible, pruning_predicate=value_null_count@1 !=
row_count@2 AND value_max@0 > 3 AND (value_null_count@1 > 0 OR
value_null_count@1 != row_count@2 AND value_max@0 > 800),
required_guarantees=[], metrics=[output_rows=10, elapsed_compute=<slt:ignore>,
output_bytes=272.0 B, files_ranges_pruned_statistics=1 total → 1 matched,
row_groups_pruned_statistics=1 total → 1 matched -> 1 fully matched,
row_groups_pruned_bloom_filter=1 total → 1 matched, page_index_pages_pruned=0
total → 0 matched, page_index_pages_skipped_by_fully_matched=1,
limit_pruned_row_groups=0 total → 0 matched, bytes_processed=1147.0 B,
bytes_scanned=210.0 B, page_index_load_skipped=1, row_groups_pruned_dynamic_f
ilter=0, metadata_load_time=<slt:ignore>, scan_efficiency_ratio=18.31%
(210/1.15 K)]
Review Comment:
Two sqllogictest files fail on the current head (CI hasn't run on this PR
yet):
- This line still includes `row_groups_pruned_bloom_filter=1 total → 1
matched`, which #25822 removed, so the actual output no longer has it. The
`output_bytes=272.0 B` value is correct. There's also an extra blank line after
it.
- The aggregate test at `explain_analyze.slt:492`, which came in from main,
still expects the old values (`1056.0 B` / `32.0 B`). With this change it
prints `1248.0 B` / `128.0 B`.
Regenerating both files should fix it:
```sh
cargo test --test sqllogictests -- explain_analyze
dynamic_filter_pushdown_config --complete
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
