Re: Flink distribution housekeeping for YARN sessions

2020-01-29 Thread Till Rohrmann
Here is the corresponding JIRA ticket: https://issues.apache.org/jira/browse/FLINK-15806 On Wed, Jan 29, 2020 at 3:16 PM Till Rohrmann wrote: > Hi Theo, > > your assumption is correct that Flink won't clean up its files when using > `yarn application -kill ID`. This should also hold true for oth

Re: Flink distribution housekeeping for YARN sessions

2020-01-29 Thread Till Rohrmann
Hi Theo, your assumption is correct that Flink won't clean up its files when using `yarn application -kill ID`. This should also hold true for other temporary files generated by Flink's Blob service, shuffle service and io manager. These files are usually stored under /tmp and should be cleaned up

Flink distribution housekeeping for YARN sessions

2020-01-28 Thread Theo Diefenthal
Hi there, Today I realized that we currently have a lot of not housekept flink distribution jar files and would like to know what to do about this, i.e. how to proper housekeep them. In the job submitting HDFS home directory, I find a subdirectory called `.flink` with hundreds of subfolders