Hongze, I don't realistically have time to iterate with you on this. I
agree that there could be easier apis to support leak detection. That being
said, just because things could be refactored to make something easier
isn't always a good enough reason to refactor them. I wonder if you can do
the fi
Thanks for sharing the information Laurent.
I'd think JDK will always provide options to leverage garbage collector when it
comes to cleaning up external resources
(like native memory) binding to a object. In panama, as you can see, the new
MemorySegment interfaces still allow users
to register
Hi Micah,
Please see my another reply. Do you think making the whole approach as "leak
detector/handler" will
help reducing the complexity of these semantics?
We can also remove the mode that disables manual-release. So when the delector
is enabled, that only
unclosed buffers will be handled by
Hi Jacques,
Thanks for sharing the detailed thoughts on this. Let me try addressing them as
following, with
probably re-prioritized order:
> Clearly this patch was driven by an implicit set of needs but it's hard to
> guess at what they are.
The issue the patch tries to solve is pretty simple:
Worth also observing that the Java community has been recognizant of the
issues with the current approach around direct byte buffer and GC, and has
come up with a proposal/implementation to avoid interaction with GC and
give more direct control. The proposal (actually, its 3rd iteration) is
describ
Clearly this patch was driven by an implicit set of needs but it's hard to
guess at what they are. As Laurent asked, what is the main goal here? There
may be many ways to solve this goal.
Some thoughts in general:
- The allocator is a finely tuned piece of multithreaded machinery that is
used on h
In addition to the points raised by Laurent, I'm concerned about having two
possible idioms in the same library. It means code written against the
library becomes less portable (you need to know how the memory allocator is
using GC or not).
I understand manual memory management in Java is tedious
We don't have to concern about that since no difference will be made on current
manual release path unless "MemoryChunkCleaner" is explicitly specified when
creating BufferAllocators. Manual ref counting will be only discard/ignored in
"MemoryChunkCleaner", while by default "MemoryChunkManager"
Unless I missed something in the big picture, but it seems that using a GC
based approach would make things much inefficient in terms of memory
management and would cause probably quite the confusion for all the systems
out there which rely on refcounting for keeping things in check, I'm not
sure w
Hi Laurent,
Sorry I might describe it unclearly and yes, the purpose is straightforward: to
discard (in another strategy rather than default) the usage of Java buffer ref
counting and rely on GC to do cleanup for the buffers. Though some design
changes to Allocator APIs maybe required to ach
Hi,
I gave a quick look at your patches but to be honest, it's not easy to
figure out the problem you're trying to solve in the first place. Or is it
simply to discard the use of ref counting to track buffer usages and
relying on Java references to keep track of buffers at the expense of
creating
Hi,
I would like to discuss on the potential of introducing a GC-based reference
management strategy to Arrow Java, and we
have already been working on an implementation in our own project. I have put
the related codes in following branch and
if it makes sense to upstream Apache Arrow I can open
12 matches
Mail list logo
