Re: Monitoring single-run job statistics

2016-01-06 Thread Stephan Ewen
Good luck with the Thesis! On Wed, Jan 6, 2016 at 6:18 PM, Filip Łęczycki wrote: > Hi Stephan, > > Thank you for you answer. I would love to contribute but currently I have > no capacity as I am buried with my thesis. > > I will reach out after graduating :) > > Bestr regards. > Filip > > Pozdra

Re: Monitoring single-run job statistics

2016-01-06 Thread Filip Łęczycki
Hi Stephan, Thank you for you answer. I would love to contribute but currently I have no capacity as I am buried with my thesis. I will reach out after graduating :) Bestr regards. Filip Pozdrawiam, Filip Łęczycki 2016-01-05 10:35 GMT+01:00 Stephan Ewen : > Hi Filip! > > There are thoughts an

Re: Monitoring single-run job statistics

2016-01-05 Thread Stephan Ewen
Hi Filip! There are thoughts and efforts to extend Flink to push the result statistics of Flink jobs to the YARN timeline server. That way, you can explore jobs that are completed. Since the whole web dashboard in Flink has a pure REST design, this is a quite straightforward fix. >From the capac

Re: Monitoring single-run job statistics

2016-01-04 Thread Filip Łęczycki
Hi Till, Thank you for you answer however I am sorry to hear that. I was reluctant to execute jobs with long running Flink cluster due to the fact that multiple jobs would cloud yarn statistics regarding cpu and memory time as well as Flink's garbage collector statistics in log, as they would be s

Re: Monitoring single-run job statistics

2016-01-04 Thread Till Rohrmann
Hi Filip, at the moment it is not possible to retrieve the job statistics after the job has finished with flink run -m yarn-cluster. The reason is that the YARN cluster is only alive as long as the job is executed. Thus, I would recommend you to execute your jobs with a long running Flink cluster

Monitoring single-run job statistics

2016-01-01 Thread Filip Łęczycki
Hi all, I am running filnk aps on YARN cluster and I am trying to get some benchmarks. When I start a long-running flink cluster on my YARN cluster I have an access to web UI and rest API that provide me statistics of the deployed jobs (as desribed here: https://ci.apache.org/projects/flink/flink-