Hi Jean,
As I said, there is currently only the run time available. You can print
the run time and accumulators results to std out by retrieving the
JobExecutionResult from the ExecutionEnvironment:
JobExecutionResult result = env.execute();
System.out.println("runtime: " result.getNetRuntime());
for (Map.Entry<String, Object> entry :
result.getAllAccumulatorResults().entrySet()) {
System.out.println(entry.getKey() + ": " entry.getValue());
}
You would do that in your Flink program. You could also store metrics in
the accumulators. However, since you're trying to compare different systems
I'd advise you to use some external tools for monitoring resource usage
like Ganglia or collectd.
Best,
Max
On Thu, Jun 18, 2015 at 4:11 PM, Jean Bez <jeanluca...@gmail.com> wrote:
> Hi Maximilian,
>
> The metrics am interested in are I/O, run time and communication. Could
> you please provide an example of how to obtain such results?
>
> Thank you!!
>
> 2015-06-18 10:45 GMT-03:00 Maximilian Michels <m...@apache.org>:
>
>> Hi Jean,
>>
>> I think it would be a nice to have feature to display some metrics on the
>> command line after a job has completed. We already have the run time and
>> the accumulator results available at the CLI and printing those would be
>> easy. What metrics in particular are you looking for?
>>
>> Best,
>> Max
>>
>> On Thu, Jun 18, 2015 at 3:41 PM, Jean Bez <jeanluca...@gmail.com> wrote:
>>
>>> Hi Fabian,
>>>
>>> I am trying to compare some examples on Hadoop, Spark and Flink. If
>>> possible I would like to see the job statistics like the report given by
>>> Hadoop. Since I am running these examples on a large cluster it would be
>>> much better if I could obtain such data directly from the console.
>>>
>>> Thanks!
>>> Jean
>>> Em 18/06/2015 04:55, "Fabian Hueske" <fhue...@gmail.com> escreveu:
>>>
>>>> Hi Jean,
>>>>
>>>> what kind of job execution stats are you interested in?
>>>>
>>>> Cheers, Fabian
>>>>
>>>> 2015-06-18 9:01 GMT+02:00 Matthias J. Sax <
>>>> mj...@informatik.hu-berlin.de>:
>>>>
>>>>> Hi,
>>>>>
>>>>> the CLI cannot show any job statistics. However, you can use the
>>>>> JobManager web interface that is accessible at port 8081 from a
>>>>> browser.
>>>>>
>>>>> -Matthias
>>>>>
>>>>>
>>>>> On 06/17/2015 10:13 PM, Jean Bez wrote:
>>>>> > Hello,
>>>>> >
>>>>> > Is it possible to view job statistics after it finished to execute
>>>>> > directly in the command line? If so, could you please explain how? I
>>>>> > could not find any mentions about this in the docs. I also tried to
>>>>> set
>>>>> > the logs to debug mode, but no other information was presented.
>>>>> >
>>>>> > Thank you!
>>>>> >
>>>>> > Regards,
>>>>> > Jean
>>>>>
>>>>>
>>>>
>>
>
