viirya commented on code in PR #890:
URL: https://github.com/apache/datafusion-comet/pull/890#discussion_r1736801063
##########
spark/src/main/java/org/apache/comet/CometBatchIterator.java:
##########
@@ -45,12 +49,27 @@ public class CometBatchIterator {
* indicating the end of the iterator.
*/
public long[] next() {
+ // The native executor should have moved the previous batch, it is safe
for us to deallocate
+ // the ArrowSchema and ArrowArray base structures.
Review Comment:
Not always. A native operator could possibly save batches internally.
##########
spark/src/main/java/org/apache/comet/CometBatchIterator.java:
##########
@@ -45,12 +49,27 @@ public class CometBatchIterator {
* indicating the end of the iterator.
*/
public long[] next() {
+ // The native executor should have moved the previous batch, it is safe
for us to deallocate
+ // the ArrowSchema and ArrowArray base structures.
Review Comment:
Not exactly. A native operator could possibly save batches internally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
