Hi Steven, thanks for reporting this issue. Looping in Till who's more familiar with the task lifecycles.
Thanks, Fabian 2017-09-12 7:08 GMT+02:00 Steven Wu <stevenz...@gmail.com>: > Hi , > > I was using Chaos Monkey to test Flink's behavior against frequent killing > of task manager nodes. I found that stopped/disposed StreamTask got > retained by java finalizer. It is kind like a memory leak. Since each > StreamTask retains 2.6 MB memory. With 20 kills (and job restarts) for > 8-CPU container, there are 2.6 * 20 * 8 MB retained in heap. > > [image: Inline image 1] > > finalize() is generally not recommended for cleanup, because "*Finalizers > are unpredictable, often dangerous, and generally unnecessary*", quoted > from Joshua Bloch's book. > http://www.informit.com/articles/article.aspx?p=1216151&seqNum=7 > > This code from StreamTask.java seems to be the cause. Is it necessary? can > it be removed? We are using flink-1.2 release branch. But I see the same > code in flink-1.3 and master branch > > /** > * The finalize method shuts down the timer. This is a fail-safe shutdown, > in case the original > * shutdown method was never called. > * > * <p> > * This should not be relied upon! It will cause shutdown to happen much > later than if manual > * shutdown is attempted, and cause threads to linger for longer than > needed. > */ > @Override > protected void finalize() throws Throwable { > super.finalize(); > if (timerService != null) { > if (!timerService.isTerminated()) { > LOG.info("Timer service is shutting down."); > timerService.shutdownService(); > } > } > > cancelables.close(); > } > > Thanks, > Steven >
