andygrove commented on PR #3266:
URL:
https://github.com/apache/datafusion-comet/pull/3266#issuecomment-3796828831
@wForget I've now added the fix for closing the `ColumnarBatch` after
conversion (commit a5857078f6f).
The issue was that after calling `converter.convert(batch)`, the batch was
not being closed, which could lead to Arrow resource leaks. The fix wraps the
row iterator to call `batch.close()` when `hasNext` returns false:
```scala
new Iterator[InternalRow] {
override def hasNext: Boolean = {
val hasMore = result.hasNext
if (!hasMore) {
batch.close()
}
hasMore
}
override def next(): InternalRow = result.next()
}
```
This fix is applied to both the `doExecute()` path and the broadcast
execution path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
