paultanay commented on issue #4136: URL: https://github.com/apache/hertzbeat/issues/4136#issuecomment-5337979355
I traced the data path in the source and found what I believe is the root cause. In ArrowUtil.deserializeMultipleRoots(), every time the Manager receives a Netty message from a collector, a new RootAllocator() and one `ArrowStreamReader` per metrics group are created - neither is ever closed: ```java RootAllocator allocator = new RootAllocator(); // never closed ArrowStreamReader reader = new ArrowStreamReader(..., allocator); // never closed ``` Arrow allocators manage off-heap native buffers via reference counting - they don't release on GC, they require explicit .close(). The 1.27M ArrowBuf objects in your heap dump are direct evidence of this accumulation. Calling metricsData.close() in the Netty handler won't fix this because the allocator is a local variable inside deserializeMultipleRoots - it's unreachable from outside. There are two secondary leaks as well: the alerter consumer loop (MetricsRealTimeAlertCalculator.startCalculate()) and the warehouse consumer loop (DataStorageDispatch.startPersistentDataStorage()) both poll MetricsData objects without ever calling close() on them - despite MetricsData implementing AutoCloseable. The fix is wrapping the reader and allocator in try-with-resources inside deserializeMultipleRoots, and using try-with-resources in both consumer loops. Let me draft a PR for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
