Thanks Yingjie for driving.
It is very useful to have this check list.
I think we can list all problematic third-party libraries.
Including hadoop jar:
org.apache.hadoop.fs.FileSystem.StatisticsDataReferenceCleaner.
Because there are too many libraries with this problem. And our Yarn mode
perJob
I'd like to do that.
Best,
Yingjie
Till Rohrmann 于2019年12月18日周三 下午4:48写道：
> I think we should add this check list to the coding guidelines and continue
> extending it there. Do you wanna update the coding guidelines accordingly
> Yingjie?
>
> Cheers,
> Till
>
> On Wed, Dec 18, 2019 at 8:21 AM Y
I think we should add this check list to the coding guidelines and continue
extending it there. Do you wanna update the coding guidelines accordingly
Yingjie?
Cheers,
Till
On Wed, Dec 18, 2019 at 8:21 AM Yingjie Cao wrote:
> Hi Till & Biao,
>
> Thanks for the reply.
>
> I agree that supplying s
Hi Till & Biao,
Thanks for the reply.
I agree that supplying some stress or stability tests can really help,
except for the jvm resource leak mentioned above, there may be other type
of resource leak like slot or network buffer leak. In addition, other tests
like triggering failover in various di
Hi Yingjie,
Thanks for figuring out the impressive bug and bringing this discussion.
I'm afraid there is no such a silver bullet for isolation from third-party
library. However I agree that resource checking utils might help.
It seems that you and Till have already raised some feasible ideas.
Res
Hi Yingjie,
thanks for reporting this issue and starting this discussion. If we are
dealing with third party libraries I believe there is always the risk that
one overlooks closing resources. Ideally we make it as hard from Flink's
perspective as possible but realistically it is hard to completely
