NoahKusaba opened a new pull request, #2359:
URL: https://github.com/apache/datafusion-ballista/pull/2359
# Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases. You can
link an issue to this PR using the GitHub syntax. For example `Closes #123`
indicates that this PR will close issue #123.
-->
Closes #.
# Rationale for this change
`compute_partition_indices` allocated a fresh `Vec<Vec<u32>>` per input
batch —
`num_partitions` vectors grown from zero, then dropped once `push_batch`
copied
the indices out. The churn scales with the output partition count.
# What changes are included in this PR?
Copying the hash_buffer implementation that already exists in
compute_partition_indices,
it now writes into a caller-owned buffer whose inner `Vec`s are cleared
rather
than dropped, so capacity survives into the next batch. Partition assignment
is
unchanged.
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
