I think you should probably use an allocator that lives long enough, yes.
Regards
Antoine.
Le 30/06/2023 à 10:23, Wenbo Hu a écrit :
Hi,
But gc is not predictable (though python's is more progressive than
that of jvm), the following code fails randomly if `gc.collect` is not
called.
```
#
Hi,
But gc is not predictable (though python's is more progressive than
that of jvm), the following code fails randomly if `gc.collect` is not
called.
```
# ENV
# Python 3.8.16 (default, Jun 12 2023, 12:55:15)
# [Clang 14.0.6 ] :: Anaconda, Inc. on darwin
# Type "help", "copyright", "credits" or
Hi,
To answer precisely:
1) The exported record batch will live as long as the Python RecordBatch
object is kept alive. If your script keeps the Python RecordBatch object
alive until the end, then the exported record batch is kept alive until
the end.
2) The rest is standard Python semant
Thanks for your explanation, Antoine.
I figured out why I'm facing the memory leak and need to call delete explicit.
my example code may mislead the situation. The key problem is when I
wrap the code of convert java stream to RecordBatchReader, I generate
a child allocator from current context (li
Le 29/06/2023 à 09:50, Wenbo Hu a écrit :
Hi,
I'm using Jpype to pass streams between java and python back and forth.
For follow code works fine with its release callback
```python
with child_allocator("test-allocator") as allocator:
r = some_package.InMemoryArrowReader.create(
1. For weakref types, cython API raise TypeError.
2. All related references need to explicit delete before the allocator close
For following code, works fine.
```
with child_allocator("test-allocator") as allocator:
r = some_package.InMemoryArrowReader.create(allocator)
c_stream
Hi,
I'm using Jpype to pass streams between java and python back and forth.
For follow code works fine with its release callback
```python
with child_allocator("test-allocator") as allocator:
r = some_package.InMemoryArrowReader.create(allocator)
c_stream = arrow_c.new("struc
