jayzhan211 commented on code in PR #25491:
URL: https://github.com/apache/datafusion/pull/25491#discussion_r4052461931
##########
datafusion/physical-plan/src/joins/hash_join/exec.rs:
##########
@@ -2854,6 +2920,7 @@ async fn collect_left_input(
null_equality: NullEquality,
null_aware: Option<NullAwareMode>,
array_map_created_count: Count,
+ prepared: bool,
Review Comment:
`prepared: bool` forks `collect_left_input` into two accounting regimes:
concat copy, row-index chain and hash/validity scratch are charged only when
`prepared`, though the normal path allocates the same memory. ~110 gated lines
inline in a 13-arg function. Move the sizing into `prepared.rs` helpers so the
shared path keeps one `if prepared` per charge
```diff
- let mut hashes_buffer = if prepared {
- let rows =
batches.iter().map(RecordBatch::num_rows).max().unwrap_or(0);
- // ... mask/validity arithmetic ...
- scratch_reservation.try_grow(bytes)?;
- Vec::with_capacity(rows)
- } else {
- Vec::new()
- };
+ let rows =
batches.iter().map(RecordBatch::num_rows).max().unwrap_or(0);
+ if prepared {
+ scratch_reservation.try_grow(prepared::hash_scratch_bytes(
+ rows,
+ &on_left,
+ &schema,
+ null_equality,
+ )?)?;
+ }
+ let mut hashes_buffer = Vec::with_capacity(rows);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
