Hi,
I tried to view directly from the web interface but I could not find any
other information about the completed jobs. I have the list, but when I
open it, no further information is provided. Is this correct?
2015-06-18 15:10 GMT-03:00 Jean Bez <jeanluca...@gmail.com>:
> Hello Max,
>
> I will try to do that! Do you know if I could obtain data about the I/O
> and communication as well? From what I could understand I can get the
> runtime and the accumulator results only. Is that right?
>
> 2015-06-18 11:37 GMT-03:00 Maximilian Michels <m...@apache.org>:
>
>> Hi Jean,
>>
>> As I said, there is currently only the run time available. You can print
>> the run time and accumulators results to std out by retrieving the
>> JobExecutionResult from the ExecutionEnvironment:
>>
>> JobExecutionResult result = env.execute();
>> System.out.println("runtime: " result.getNetRuntime());
>> for (Map.Entry<String, Object> entry :
>> result.getAllAccumulatorResults().entrySet()) {
>> System.out.println(entry.getKey() + ": " entry.getValue());
>> }
>>
>> You would do that in your Flink program. You could also store metrics in
>> the accumulators. However, since you're trying to compare different systems
>> I'd advise you to use some external tools for monitoring resource usage
>> like Ganglia or collectd.
>>
>> Best,
>> Max
>>
>> On Thu, Jun 18, 2015 at 4:11 PM, Jean Bez <jeanluca...@gmail.com> wrote:
>>
>>> Hi Maximilian,
>>>
>>> The metrics am interested in are I/O, run time and communication. Could
>>> you please provide an example of how to obtain such results?
>>>
>>> Thank you!!
>>>
>>> 2015-06-18 10:45 GMT-03:00 Maximilian Michels <m...@apache.org>:
>>>
>>>> Hi Jean,
>>>>
>>>> I think it would be a nice to have feature to display some metrics on
>>>> the command line after a job has completed. We already have the run time
>>>> and the accumulator results available at the CLI and printing those would
>>>> be easy. What metrics in particular are you looking for?
>>>>
>>>> Best,
>>>> Max
>>>>
>>>> On Thu, Jun 18, 2015 at 3:41 PM, Jean Bez <jeanluca...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Fabian,
>>>>>
>>>>> I am trying to compare some examples on Hadoop, Spark and Flink. If
>>>>> possible I would like to see the job statistics like the report given by
>>>>> Hadoop. Since I am running these examples on a large cluster it would be
>>>>> much better if I could obtain such data directly from the console.
>>>>>
>>>>> Thanks!
>>>>> Jean
>>>>> Em 18/06/2015 04:55, "Fabian Hueske" <fhue...@gmail.com> escreveu:
>>>>>
>>>>>> Hi Jean,
>>>>>>
>>>>>> what kind of job execution stats are you interested in?
>>>>>>
>>>>>> Cheers, Fabian
>>>>>>
>>>>>> 2015-06-18 9:01 GMT+02:00 Matthias J. Sax <
>>>>>> mj...@informatik.hu-berlin.de>:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> the CLI cannot show any job statistics. However, you can use the
>>>>>>> JobManager web interface that is accessible at port 8081 from a
>>>>>>> browser.
>>>>>>>
>>>>>>> -Matthias
>>>>>>>
>>>>>>>
>>>>>>> On 06/17/2015 10:13 PM, Jean Bez wrote:
>>>>>>> > Hello,
>>>>>>> >
>>>>>>> > Is it possible to view job statistics after it finished to execute
>>>>>>> > directly in the command line? If so, could you please explain how?
>>>>>>> I
>>>>>>> > could not find any mentions about this in the docs. I also tried
>>>>>>> to set
>>>>>>> > the logs to debug mode, but no other information was presented.
>>>>>>> >
>>>>>>> > Thank you!
>>>>>>> >
>>>>>>> > Regards,
>>>>>>> > Jean
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>
