Dandandan opened a new pull request, #24536: URL: https://github.com/apache/datafusion/pull/24536
## Which issue does this PR close? - Split out of #24456, which needs this to cost joins but does not depend on the rest of that PR. ## Rationale for this change A join knows how many rows it will emit but not how wide they are: every join type but semi and anti reports `total_byte_size: Precision::Absent`. Anything above the join that reads a size finds nothing. `hash_join_single_partition_threshold` is a byte threshold, so on a join over a join it has no bytes to compare and the decision falls to the row threshold alone. Any memory or cost estimate over a join subtree has the same gap. ## What changes are included in this PR? The width of one output row follows from the sides a join emits, which the join type already says: both sides for an inner or outer join, the preserved side for a semi or anti join, and one side plus a boolean for a mark join. Multiplied by the estimated cardinality, that gives the output size. The boolean a mark join appends is counted as one bit per row, the width of an Arrow `BooleanArray`, not one byte. Semi and anti joins already derive a size from their column statistics, which knows which columns survive rather than assuming an average row. That estimate is kept: the width fills a gap rather than replacing a better answer. ## Are these changes tested? Yes. A new unit test covers an inner join summing both sides, a mark join adding one bit per row, and a semi join keeping its column-derived size. `test_hash_join_partition_statistics` was reporting `Absent` for a hash join and now expects the size, which is the change this PR makes. The full sqllogictest suite passes unchanged (501/501), so no plan in it moves. ## Are there any user-facing changes? No API change. Joins now report a `total_byte_size` where they previously reported none, which `EXPLAIN ANALYZE` and statistics-driven decisions can read. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
