Hi Steve,
RocksDB does not contribute to the JVM direct memory. RocksDB off-heap
memory consumption is part of managed memory [1].
You got `OutOfMemoryError: Direct buffer memory` which is related to
the JVM direct memory, also off-heap but managed by JVM.
The JVM direct memory limit depends on t
Thanks Stephan, any pointers on how managed memory is used in streaming
application will really help.
Regards,
Govind
> On Aug 24, 2017, at 1:53 AM, Stephan Ewen wrote:
>
> Hi!
>
> RocksDB will be used when it is selected as the state backend, independent of
> the checkpointing configuration
Hi!
RocksDB will be used when it is selected as the state backend, independent
of the checkpointing configuration.
Using RocksDB as the state backend, Flink will have some objects on the
heap, like timers (we will move them to RocksDB as well in the near future)
but the majority will be off heap.
I have couple more questions regarding flink's jvm memory.
In a streaming application what is managed memory used for? I read from a
blog that all objects created inside the user function will go into
unmanaged memory. Where does the managed key/ operator state state reside?
Also when does the st
One would need to look at your code and possible on some heap statistics. Maybe
something wrong happens when you cache them (do you use a 3rd party library or
your own implementation?). Do you use a stable version of your protobuf library
(not necessarily the most recent). You also may want to l
