Re: heap dump shows StoppableSourceStreamTask retained by java.lang.finalizer

2017-09-19 Thread Steven Wu
> How did you kill the TaskManagers? I assume you didn't kill the JVM process because otherwise you wouldn't see the finalizer objects piling up. Till, I configure Chao Monkey to always kill the newest/same TaskManager. So other N-1 TaskManagers stayed up during the whole process. Each of them exp

Re: heap dump shows StoppableSourceStreamTask retained by java.lang.finalizer

2017-09-19 Thread Steven Wu
Stephan, agree that it is not a real memory leak. I haven't found it affecting the system. so it is sth odd for now. but if it is not really necessary, why do we want to defer memory release with unpredictable behavior? can StreamTask stop() method take care of the cleanup work and don't need to r

Re: heap dump shows StoppableSourceStreamTask retained by java.lang.finalizer

2017-09-19 Thread Stephan Ewen
Hi! >From my understanding, overriding finalize() still has some use cases and is valid if done correctly, (although PhantomReference has more control over the cleanup process). finalize() is still used in JDK classes as well. Whenever one overrides finalize(), the object cannot be immediately ga

Re: heap dump shows StoppableSourceStreamTask retained by java.lang.finalizer

2017-09-18 Thread Till Rohrmann
Hi Steven, the finalize method in StreamTask acts as a safety net in case the services of the StreamTask haven't been properly shut down. In the code, however, it looks as if the TimerService, for example, is always being stopped in the finally block of the invoke method. Thus, it might not be nec

Re: heap dump shows StoppableSourceStreamTask retained by java.lang.finalizer

2017-09-14 Thread Fabian Hueske
Hi Steven, thanks for reporting this issue. Looping in Till who's more familiar with the task lifecycles. Thanks, Fabian 2017-09-12 7:08 GMT+02:00 Steven Wu : > Hi , > > I was using Chaos Monkey to test Flink's behavior against frequent killing > of task manager nodes. I found that stopped/disp