[
https://issues.apache.org/jira/browse/ARROW-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281819#comment-17281819
]
David Li commented on ARROW-11569:
----------------------------------
[~lulijun] Just to double-check, are you also closing the ArrowRecordBatch you
generate? It's not clear from the snippet here + that's a common mistake.
> Out-of-heap memory leaks in FlightClient.getStream
> --------------------------------------------------
>
> Key: ARROW-11569
> URL: https://issues.apache.org/jira/browse/ARROW-11569
> Project: Apache Arrow
> Issue Type: Bug
> Components: FlightRPC
> Affects Versions: 3.0.0
> Environment: Java version: 8
> Flight&Arrow version: 3.0.0
> Reporter: lulijun
> Priority: Major
> Labels: FlightClient, MemoryLeak
>
> I'm trying to use Arrow and Flight in my Java project, I used
> FlightClient.getStream to request data from a remote node, but I found the
> allocator's allocated memory size grows with the request.
> Java version: 8
> Flight&Arrow version: 3.0.0
>
> Here's a snippet of my code, I found the allocated memory of flithStream
> still exist after root.close:
> try (FlightStream flightStream = client.getStream(ticket,
> CallOptions.timeout(1, TimeUnit.MILLISECONDS))) {
> VectorSchemaRoot root = flightStream.getRoot();
> root.clear();
> while (flightStream.next()) {
> // transfer root to ArrowRecordBatch and execute
> dispatchNext(shuffleChunk, root);
> root.clear();
> }
> root.close();
> } catch (FlightRuntimeException fe) {
> String errorMsg = "IndexReadOperator read index fail ={}
> "+indexChunkDesc.toString();
> LOGGER.error(errorMsg, fe);
> throw new NetworkException(errorMsg, fe);
> } catch (Exception e) {
> LOGGER.error("IndexReadOperator read index fail!", e);
> throw new NetworkException("IndexReadOperator getStream fail! ", e);
> }
>
> Is this a bug or am I using it incorrectly?
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
