Re: Ability to partition logs per pipeline

2016-07-31 Thread Aljoscha Krettek
Hi, I'm afraid that's not possible right now. The preferred way of running would be to have a Yarn cluster per job, that way you can isolate the logs. Cheers, Aljoscha On Thu, 14 Jul 2016 at 09:49 Chawla,Sumit wrote: > Hi Robert > > I actually mean both. Scenarios where multiple jobs are runni

Re: Ability to partition logs per pipeline

2016-07-14 Thread Chawla,Sumit
Hi Robert I actually mean both. Scenarios where multiple jobs are running on cluster, and same job could be running on multiple task managers. How can we make sure that each job logs to a different file so that Logs are not mixed, and its easy to debug a particular job. Something like Hadoop Y

Re: Ability to partition logs per pipeline

2016-07-14 Thread Robert Metzger
Hi Sumit, What exactly do you mean by pipeline? Are you talking about cases were multiple jobs are running concurrently on the same TaskManager, or are you referring to parallel instances of a Flink job? On Wed, Jul 13, 2016 at 9:49 PM, Chawla,Sumit wrote: > Hi All > > Does flink provide any ab