Re: checkpoints not being removed from HDFS

2016-05-13 Thread Maciek Próchniak
Hi Ufuk, It seems I messed it up a bit :) I cannot comment on jira, since it's temporarily locked... 1. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: `/flink/checkpoints_test/570d6e67d571c109daab468e5678402b/chk-62 is non empty': Directory is not empty - this seems to be expected behavi

Re: checkpoints not being removed from HDFS

2016-05-12 Thread Maciek Próchniak
thanks, I'll try to reproduce it in some test by myself... maciek On 12/05/2016 18:39, Ufuk Celebi wrote: The issue is here: https://issues.apache.org/jira/browse/FLINK-3902 (My "explanation" before dosn't make sense actually and I don't see a reason why this should be related to having many s

Re: checkpoints not being removed from HDFS

2016-05-12 Thread Ufuk Celebi
The issue is here: https://issues.apache.org/jira/browse/FLINK-3902 (My "explanation" before dosn't make sense actually and I don't see a reason why this should be related to having many state handles.) On Thu, May 12, 2016 at 3:54 PM, Ufuk Celebi wrote: > Hey Maciek, > > thanks for reporting th

Re: checkpoints not being removed from HDFS

2016-05-12 Thread Ufuk Celebi
Hey Maciek, thanks for reporting this. Having files linger around looks like a bug to me. The idea behind having the recursive flag set to false in the AbstractFileStateHandle.discardState() call is that the FileStateHandle is actually just a single file and not a directory. The second call tryin