Re: cleaning yarn logs for long-running applications

2018-01-15 Thread Julio Biason
Hi Soheil, That's something you probably want to take a look at logrotate: https://linux.die.net/man/8/logrotate (We didn't reach the point of generate massive number of logs yet here, but that's the default route we take to solve this kind of issue.) On Sun, Jan 14, 2018 at 8:12 AM, Soheil Pour

cleaning yarn logs for long-running applications

2018-01-14 Thread Soheil Pourbafrani
Hi, I want to use Yarn as cluster manager for running Flink applications, but I'm worried about how Flink or Yarn handle local logs in each machine. Does they clean aged logs for a long-running application? If not, it's possible the local storage get full!!!