SubhamSinghal commented on PR #21962:
URL: https://github.com/apache/datafusion/pull/21962#issuecomment-4390372416
Thanks for the review @kosiew
I've updated the implementation to use unconditional grow() instead:
**Rationale:** The join key arrays are physically in memory. Not tracking
them gives the pool a stale view and could let concurrent operators
over-allocate. Since grow() is infallible (returns ()), the accounting is now
deterministic — no conditional path.
I've also added a regression test (spill_join_arrays_no_headroom) that
configures the pool smaller than join_arrays_mem,
verifying:
- The join completes correctly
- peak_mem_used > 0 (force-grow always tracks)
- runtime.memory_pool.reserved() == 0 after completion (grow/shrink
balanced)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
