Hi all, I have a long running , streaming app saving checkpoints to the file system.
What is the layout of the checkpoint directory ? My current checkpoint directory has >2000 directories in it , similar to this: chk-4645 Also, the directory has grown to >3GB I have a small cluster, and all were started at the same time, nothing has been restarted, but this is occurring one of the nodes, the others have about the same number of directories in the checkpoint directory, but not nearly as large. Why are there so many chk-xxxx directories ? And why can they become so large ? Is there something I should be setting in the yaml file ? I was going to just remove them , but it just struck me as odd that there are so many … Thanks Jim
