I just saw that your other email is about the same issue. Since you've done a heapdump already, did you see any pattern in the allocated objects? Ideally none of the classes from your user code should stick around when no job is running. What's the size of the heap dump? I'm happy to take a look at it if it's reasonably small.
On Wed, Aug 30, 2017 at 10:27 AM, Robert Metzger <rmetz...@apache.org> wrote: > Hi Javier, > > I'm not aware of such issues with Flink, but if you could give us some > more details on your setup, I might get some more ideas on what to look for. > > are you using the RocksDBStateBackend? (RocksDB is doing some JNI > allocations, that could potentially leak memory) > Also, are you passing any special garbage collector options? (Maybe some > classes are not unloaded) > Are you using anything else that is special (such as protobuf or avro > formats, or any other big library)? > > Regards, > Robert > > > > On Mon, Aug 28, 2017 at 5:04 PM, Javier Lopez <javier.lo...@zalando.de> > wrote: > >> Hi all, >> >> we are starting a lot of Flink jobs (streaming), and after we have >> started 200 or more jobs we see that the non-heap memory in the >> taskmanagers increases a lot, to the point of killing the instances. We >> found out that every time we start a new job, the committed non-heap memory >> increases by 5 to 10MB. Is this an expected behavior? Are there ways to >> prevent this? >> > >
