Ok, thanks. I've said this in another thread but everything seems to go
completely idle during checkpoints while waiting on 1 operator, there's no
CPU usage, hardly any disk usage. I'll assume it's something else then.
On Wed, Dec 16, 2020 at 10:42 AM Robert Metzger wrote:
> I don't think the di
I don't think the direct memory is causing any performance bottlenecks. The
backpressure is probably caused by something else (high CPU load, slow
external system, data skew)
On Wed, Dec 16, 2020 at 7:23 PM Steven Wu wrote:
> if you are running out of direct buffer, you will see
> "java.lang.Ou
if you are running out of direct buffer, you will see
"java.lang.OutOfMemoryError:
Direct buffer memory"
On Wed, Dec 16, 2020 at 9:47 AM Rex Fenley wrote:
> Thanks for the reply. If what I'm understanding is correct there's no
> chance of an OOM, but since direct memory is for I/O, it being comp
Thanks for the reply. If what I'm understanding is correct there's no
chance of an OOM, but since direct memory is for I/O, it being completely
filled may be a sign of backpressure? Currently one of our operators takes
a tremendous amount of time to align during a checkpoint. Could increasing
direc
Hey Rex,
the direct memory is used for IO. There is no concept of direct memory
being "full". The only thing that can happen is that you have something in
place (Kubernetes, YARN) that limits / enforces the memory use of a Flink
process, and you run out of your memory allowance. The direct memory
Hello,
Our job consistently shows
Outside JVM
Type
Count
Used
Capacity
*Direct* 32,839 1.03 GB 1.03 GB
for direct memory.
Is it typical for it to be full? What are the consequences that we may not
be noticing of direct memory being full?
Thanks!
--
Rex Fenley | Software Engineer - Mobile an
