Re: continous time triger

[Till Rohrmann]

Hi Radu,

you can register processing and event time time triggers using the
TriggerContext which is given to the onElement, onProcessingTime and
onEventTime methods of Trigger. In case you register a processing time
timer, the onProcessingTime method will be called once the system clock has
passed the timer time. In case of an event time timer, the onEventTime
method is called once a watermark has been received which has a higher
watermark than the timer.

I hope this helps you to solve your problem.

Cheers,
Till
​

On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran 
wrote:

> Re-Hi,
>
>
>
> I have another question regarding the triggering of the processing of a
> window. Can this be done in some way at specific time intervals,
> independent of whether  an event has been received or not, via a trigger?
>
>
>
> The reason why I am considering a trigger rather than timeWindow(All) is
> that timeWindow will end up generating multiple windows and duplicating
> data, while having the option from the trigger to actually fire the
> processing at certain times, independent of when the events arrived) would
> enable to operate with a single window.
>
>
>
> Regards,
>
>
>
> Dr. Radu Tudoran
>
> Research Engineer - Big Data Expert
>
> IT R&D Division
>
>
>
> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>
> European Research Center
>
> Riesstrasse 25, 80992 München
>
>
>
> E-mail: *radu.tudo...@huawei.com *
>
> Mobile: +49 15209084330
>
> Telephone: +49 891588344173
>
>
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>
> This e-mail and its attachments contain confidential information from
> HUAWEI, which is intended only for the person or entity whose address is
> listed above. Any use of the information contained herein in any way
> (including, but not limited to, total or partial disclosure, reproduction,
> or dissemination) by persons other than the intended recipient(s) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!
>
>
>

Re: Redeployements and state

[Ufuk Celebi]

Hey Niels!

Stephan gave a very good summary of the current state of things. What do you 
think of the outlined stop with savepoint method?

Regarding the broken links: I’ve fixed various broken links in the master docs 
yesterday. If you encounter something again, feel free to post it to the ML or 
open a JIRA for it.

– Ufuk

> On 25 Jan 2016, at 16:21, Stephan Ewen  wrote:
> 
> Hi Niels!
> 
> There is a slight mismatch between your thoughts and the current design, but 
> not much.
> 
> What you describe (at the start of the job, the latest checkpoint is 
> automatically loaded) is basically what the high-availability setup does if 
> the master dies. The new master loads all jobs and continues them from the 
> latest checkpoint.
> If you run an HA setup, and you stop/restart your jobs not by stopping the 
> jobs, but by killing the cluster, you should get that behavior.
> 
> Once a job is properly stopped, and you start a new job, there is no way for 
> Flink to tell that this is in fact the same job and it should resume from 
> where the recently stopped. Also, "same" should be a fuzzy "same", to allow 
> for slight changes in the job (bug fixes). Safepoints let you put the 
> persistent part of the job somewhere, to tell a new job where to pick up from.
>   - Makes it work in non-HA setups
>   - Allows you to keep multiple savepoint (like "versions", say one per day 
> or so) to roll back to
>   - Can have multiple versions of the same jobs resuming from one savepoint 
> (what-if or A/B tests, or seamless version upgrades)
> 
> 
> There is something on the roadmap that would make your use case very easy: 
> "StopWithSavepoint"
> 
> There is an open pull request to cleanly stop() a streaming program. The next 
> enhancement is to stop it and let it draw a savepoint as part of that. Then 
> you can simply script a stop/start like that:
> 
> # stop with savepoint
> bin/flink stop -s  jobid
> 
> # resume
> bin/flink run -s  job
> 
> 
> Hope that helps,
> Stephan
> 
> 
> On Fri, Jan 22, 2016 at 3:06 PM, Niels Basjes  wrote:
> Hi,
> 
> @Max: Thanks for the new URL. I noticed that a lot (in fact almost all) of 
> links in the new manuals lead to 404 errors. Maybe you should run an 
> automated test to find them all.
> 
> I did a bit of reading about the savepoints and that in fact they are written 
> as "Allow to trigger checkpoints manually".
> 
> Let me sketch what I think I need:
> 1) I need recovery of the topology in case of partial failure (i.e. a single 
> node dies).
> 2) I need recovery of the topology in case of full topology failure (i.e. 
> Hadoop security tokens cause the entire thing to die, or I need to deploy a 
> fixed version of my software).
> 
> Now what I understand is that the checkpoints are managed by Flink and as 
> such allow me to run the topology without any manual actions. These are 
> cleaned automatically when no longer needed.
> These savepoints however appear to need external 'intervention'; they are 
> intended as 'manual'. So in addition to my topology I need something extra 
> that periodically (i.e. every minute) fires a command to persist a checkpoint 
> into a savepoint and to cleanup the 'old' ones.
> 
> What I want is something that works roughly as follows:
> 1) I configure everything (i.e. assign Ids configure the checkpoint 
> directory, etc.)
> 2) The framework saves and cleans the checkpoints automatically when the 
> topology is running.
> 3) I simply start the topology without any special options.
> 
> My idea is essentially that at the startup of a topology the system looks at 
> the configured checkpoint persistance and recovers the most recent one.
> 
> Apparently there is a mismatch between what I think is useful and what has 
> been implemented so far. 
> Am I missing something or should I submit this as a Jira ticket for a later 
> version?
> 
> Niels Basjes
> 
> 
> 
> 
> 
> 
> On Mon, Jan 18, 2016 at 12:13 PM, Maximilian Michels  wrote:
> The documentation layout changed in the master. Then new URL:
> https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/savepoints.html
> 
> On Thu, Jan 14, 2016 at 2:21 PM, Niels Basjes  wrote:
> > Yes, that is exactly the type of solution I was looking for.
> >
> > I'll dive into this.
> > Thanks guys!
> >
> > Niels
> >
> > On Thu, Jan 14, 2016 at 11:55 AM, Ufuk Celebi  wrote:
> >>
> >> Hey Niels,
> >>
> >> as Gabor wrote, this feature has been merged to the master branch
> >> recently.
> >>
> >> The docs are online here:
> >> https://ci.apache.org/projects/flink/flink-docs-master/apis/savepoints.html
> >>
> >> Feel free to report back your experience with it if you give it a try.
> >>
> >> – Ufuk
> >>
> >> > On 14 Jan 2016, at 11:09, Gábor Gévay  wrote:
> >> >
> >> > Hello,
> >> >
> >> > You are probably looking for this feature:
> >> > https://issues.apache.org/jira/browse/FLINK-2976
> >> >
> >> > Best,
> >> > Gábor
> >> >
> >> >
> >> >
> >> >
> >> > 2016-01-14 11:05 GMT+01:00 Niels Basjes :
> >> >> Hi

Re: maxtime / watermark for GlobaWindow

[Till Rohrmann]

Hi Radu,

If I’m not mistaken, then it’s not possible with the current GlobalWindow
implementation. However, you could extend the GlobalWindow which adds a new
field into which the timestamp of the triggering element is stored. This
field can then be read from within the WindowFunction to retrieve the
latest timestamp.

Cheers,
Till
​

On Mon, Jan 25, 2016 at 8:36 PM, Radu Tudoran 
wrote:

> Hi,
>
>
>
> I am using a global window to collect some events. I use a trigger to fire
> the processing.
>
> Is there any way to get the time of the event that has triggered the
> processing.
>
>
>
> I am asking this as the getMaxTime() field of the GlobalWindow returns
> MaxLong.
>
>
>
>
>
> The code skeleton is:
>
>
>
> stream
>
>.windowAll(GlobalWindows.create())
>
>.trigger( *new* MyTrigger())
>
>.*apply*( *new** AllWindowFunction, Tuple1,
> GlobalWindow>()* {
>
>   @Override
>
>   *public* *void* *apply(GlobalWindow
> arg0,*
>
> *Iterable< Tuple1, >
> arg1,*
>
> *Collector>
> arg2) **throws** Exception* {
>
>
>
> // - get the even timestamp
>
>
>
>
>
>
>
>   }
>
>}*)*
>
>
>
>
>
>
>
> Dr. Radu Tudoran
>
> Research Engineer - Big Data Expert
>
> IT R&D Division
>
>
>
> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>
> European Research Center
>
> Riesstrasse 25, 80992 München
>
>
>
> E-mail: *radu.tudo...@huawei.com *
>
> Mobile: +49 15209084330
>
> Telephone: +49 891588344173
>
>
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>
> This e-mail and its attachments contain confidential information from
> HUAWEI, which is intended only for the person or entity whose address is
> listed above. Any use of the information contained herein in any way
> (including, but not limited to, total or partial disclosure, reproduction,
> or dissemination) by persons other than the intended recipient(s) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!
>
>
>

Re: continous time triger

[Brian Chhun]

For what it's worth, we have a trigger that fires once a day for a
recurring calculation. When an element comes in, we set the trigger
context's processing time timer to the exact millisecond of the desired
time. The predefined triggers were useful to look at to achieve this:
https://github.com/apache/flink/tree/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers

Some things I discovered along the way, particularly using processing time,
which may be useful:
- registering a time that's already passed will cause the timer callback to
be called
- when the system shuts down, the window is fired even though the trigger
has not gone off (this sounds subject to change though)

On Tue, Jan 26, 2016 at 3:47 AM, Till Rohrmann  wrote:

> Hi Radu,
>
> you can register processing and event time time triggers using the
> TriggerContext which is given to the onElement, onProcessingTime and
> onEventTime methods of Trigger. In case you register a processing time
> timer, the onProcessingTime method will be called once the system clock
> has passed the timer time. In case of an event time timer, the onEventTime
> method is called once a watermark has been received which has a higher
> watermark than the timer.
>
> I hope this helps you to solve your problem.
>
> Cheers,
> Till
> ​
>
> On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran 
> wrote:
>
>> Re-Hi,
>>
>>
>>
>> I have another question regarding the triggering of the processing of a
>> window. Can this be done in some way at specific time intervals,
>> independent of whether  an event has been received or not, via a trigger?
>>
>>
>>
>> The reason why I am considering a trigger rather than timeWindow(All) is
>> that timeWindow will end up generating multiple windows and duplicating
>> data, while having the option from the trigger to actually fire the
>> processing at certain times, independent of when the events arrived) would
>> enable to operate with a single window.
>>
>>
>>
>> Regards,
>>
>>
>>
>> Dr. Radu Tudoran
>>
>> Research Engineer - Big Data Expert
>>
>> IT R&D Division
>>
>>
>>
>> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>>
>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>
>> European Research Center
>>
>> Riesstrasse 25, 80992 München
>>
>>
>>
>> E-mail: *radu.tudo...@huawei.com *
>>
>> Mobile: +49 15209084330
>>
>> Telephone: +49 891588344173
>>
>>
>>
>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
>> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
>> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
>> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
>> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>>
>> This e-mail and its attachments contain confidential information from
>> HUAWEI, which is intended only for the person or entity whose address is
>> listed above. Any use of the information contained herein in any way
>> (including, but not limited to, total or partial disclosure, reproduction,
>> or dissemination) by persons other than the intended recipient(s) is
>> prohibited. If you receive this e-mail in error, please notify the sender
>> by phone or email immediately and delete it!
>>
>>
>>
>
>

RE: continous time triger

[Radu Tudoran]

Hi,

Thank you for sharing your experience and also to Till for the advice.
What I would like to do is to be able to fire the window potentially multiple 
times, even if an event did not arrive. I will look more about how dealing with 
the processing time could help in this

Dr. Radu Tudoran
Research Engineer - Big Data Expert
IT R&D Division

[cid:image007.jpg@01CD52EB.AD060EE0]
HUAWEI TECHNOLOGIES Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München

E-mail: radu.tudo...@huawei.com
Mobile: +49 15209084330
Telephone: +49 891588344173

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, 
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
This e-mail and its attachments contain confidential information from HUAWEI, 
which is intended only for the person or entity whose address is listed above. 
Any use of the information contained herein in any way (including, but not 
limited to, total or partial disclosure, reproduction, or dissemination) by 
persons other than the intended recipient(s) is prohibited. If you receive this 
e-mail in error, please notify the sender by phone or email immediately and 
delete it!

From: Brian Chhun [mailto:brian.ch...@getbraintree.com]
Sent: Tuesday, January 26, 2016 5:28 PM
To: user@flink.apache.org
Subject: Re: continous time triger

For what it's worth, we have a trigger that fires once a day for a recurring 
calculation. When an element comes in, we set the trigger context's processing 
time timer to the exact millisecond of the desired time. The predefined 
triggers were useful to look at to achieve this: 
https://github.com/apache/flink/tree/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers

Some things I discovered along the way, particularly using processing time, 
which may be useful:
- registering a time that's already passed will cause the timer callback to be 
called
- when the system shuts down, the window is fired even though the trigger has 
not gone off (this sounds subject to change though)

On Tue, Jan 26, 2016 at 3:47 AM, Till Rohrmann 
mailto:trohrm...@apache.org>> wrote:

Hi Radu,

you can register processing and event time time triggers using the 
TriggerContext which is given to the onElement, onProcessingTime and 
onEventTime methods of Trigger. In case you register a processing time timer, 
the onProcessingTime method will be called once the system clock has passed the 
timer time. In case of an event time timer, the onEventTime method is called 
once a watermark has been received which has a higher watermark than the timer.

I hope this helps you to solve your problem.

Cheers,
Till
​

On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran 
mailto:radu.tudo...@huawei.com>> wrote:
Re-Hi,

I have another question regarding the triggering of the processing of a window. 
Can this be done in some way at specific time intervals, independent of whether 
 an event has been received or not, via a trigger?

The reason why I am considering a trigger rather than timeWindow(All) is that 
timeWindow will end up generating multiple windows and duplicating data, while 
having the option from the trigger to actually fire the processing at certain 
times, independent of when the events arrived) would enable to operate with a 
single window.

Regards,

Dr. Radu Tudoran
Research Engineer - Big Data Expert
IT R&D Division

[cid:image007.jpg@01CD52EB.AD060EE0]
HUAWEI TECHNOLOGIES Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München

E-mail: radu.tudo...@huawei.com
Mobile: +49 15209084330
Telephone: +49 891588344173

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, 
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
This e-mail and its attachments contain confidential information from HUAWEI, 
which is intended only for the person or entity whose address is listed above. 
Any use of the information contained herein in any way (including, but not 
limited to, total or partial disclosure, reproduction, or dissemination) by 
persons other than the intended recipient(s) is prohibited. If you receive this 
e-mail in error, please notify the sender by phone or email immediately and 
delete it!

Re: rowmatrix equivalent

[Lydia Ickler]

Hi Till,

maybe I will do that :) 
If I have some other questions I will let you know!

Best regards,
Lydia


> Am 24.01.2016 um 17:33 schrieb Till Rohrmann :
> 
> Hi Lydia,
> 
> Flink does not come with a distributed matrix implementation as Spark does it 
> with the RowMatrix, yet. However, you can easily implement it yourself. This 
> would also be a good contribution to the project if you want to tackle the 
> problem
> 
> Cheers,
> Till
> 
> On Sun, Jan 24, 2016 at 4:03 PM, Lydia Ickler  > wrote:
> Hi all,
> 
> this is maybe a stupid question but what within Flink is the equivalent to 
> Sparks’ RowMatrix ?
> 
> Thanks in advance,
> Lydia
> 

Task Manager metrics per job on Flink 0.9.1

[Pieter Hameete]

Hi people!

A lot of metrics are gathered for each TaskManager every few seconds. The
web UI shows nice graphs for some of these metrics too.

I would like to make graphs of the memory and cpu usage, and the time spent
on garbage collection for each job. Because of this I am wondering if the
metrics are also stored somewhere, or if there is an option to enable
storing the metrics per job.

In the configuration documentation I could not find such an option. Is this
possible in version 0.9.1 of Flink? If not: is it possible in Flink 0.10.1
or is it possible to request or develop such a feature?

Thank you for your help and kind regards,

Pieter

Streaming left outer join

[Alexander Gryzlov]

Hello,

I'm trying to implement a left outer join of two Kafka streams within a
sliding window. So far I have the following code:

foos
  .coGroup(bars)
  .where(_.baz).equalTo(_.baz)
  .window(SlidingTimeWindows.of(Time.of(1, TimeUnit.MINUTES), Time.of(1,
TimeUnit.SECONDS)))
  .apply((fs: Iterator[Foo], bs: Iterator[Bar], o: Collector[FooBar]) =>
   fs.foreach(f =>
if (bs.isEmpty)
  o.collect(FooBar(f, None))
else
  bs.foreach(b => o.collect(FooBar(f, Some(b
   )
  )

However, this results in the pair being emitted from every window slide,
regardless of the match. The desired behaviour would be:
* emit the the match as soon as it's found, don't emit any more pairs for
it,
* otherwise, emit the empty match, when the left side element leaves the
last of its windows

What would be the idiomatic/efficient way to implement such behaviour? Is
it possible at all with the coGroup/window mechanism, or some other way is
necessary?

Alex

Re: Task Manager metrics per job on Flink 0.9.1

[Ritesh Kumar Singh]

Going by the list in the latest documentation

for Flink 0.10.1 release, memory and cpu stats are not stored. Neither is
the time spent on garbage collection stored anywhere.

In my opinion, trying to store these metrics will degrade the performance
of jobs too. And so its basically a trade off between performance and
computation cost. For me, the web ui hangs even for the current set of
parameters :(

*Ritesh Kumar Singh,*
*https://riteshtoday.wordpress.com/* 

On Tue, Jan 26, 2016 at 7:16 PM, Pieter Hameete  wrote:

> Hi people!
>
> A lot of metrics are gathered for each TaskManager every few seconds. The
> web UI shows nice graphs for some of these metrics too.
>
> I would like to make graphs of the memory and cpu usage, and the time
> spent on garbage collection for each job. Because of this I am wondering if
> the metrics are also stored somewhere, or if there is an option to enable
> storing the metrics per job.
>
> In the configuration documentation I could not find such an option. Is
> this possible in version 0.9.1 of Flink? If not: is it possible in Flink
> 0.10.1 or is it possible to request or develop such a feature?
>
> Thank you for your help and kind regards,
>
> Pieter
>
>
>

Re: Task Manager metrics per job on Flink 0.9.1

[Pieter Hameete]

Hi Ritesh,

thanks for the response! The metrics are already being gathered though, so
I think it would be nice to have a configuration/option to log them
somewhere. It doesnt have to be enabled by default, and I dont think it
should degrade the performance very much. It looks like the metrics are
currently sent with each heartbeat by default already. Your Web UI probably
hangs because it has to update all the graphs on every heartbeat, when you
have many task managers that will be heavy on your computer :-)

- Pieter

2016-01-26 20:17 GMT+01:00 Ritesh Kumar Singh :

> Going by the list in the latest documentation
> 
> for Flink 0.10.1 release, memory and cpu stats are not stored. Neither is
> the time spent on garbage collection stored anywhere.
>
> In my opinion, trying to store these metrics will degrade the performance
> of jobs too. And so its basically a trade off between performance and
> computation cost. For me, the web ui hangs even for the current set of
> parameters :(
>
> *Ritesh Kumar Singh,*
> *https://riteshtoday.wordpress.com/* 
>
> On Tue, Jan 26, 2016 at 7:16 PM, Pieter Hameete 
> wrote:
>
>> Hi people!
>>
>> A lot of metrics are gathered for each TaskManager every few seconds. The
>> web UI shows nice graphs for some of these metrics too.
>>
>> I would like to make graphs of the memory and cpu usage, and the time
>> spent on garbage collection for each job. Because of this I am wondering if
>> the metrics are also stored somewhere, or if there is an option to enable
>> storing the metrics per job.
>>
>> In the configuration documentation I could not find such an option. Is
>> this possible in version 0.9.1 of Flink? If not: is it possible in Flink
>> 0.10.1 or is it possible to request or develop such a feature?
>>
>> Thank you for your help and kind regards,
>>
>> Pieter
>>
>>
>>
>

Re: Task Manager metrics per job on Flink 0.9.1

[Ritesh Kumar Singh]

I didn't know these stats were collected. Thanks for telling :)
In that case, it should definitely be a feature which can be enabled via
config files.

*Ritesh Kumar Singh,*
*https://riteshtoday.wordpress.com/* 

On Tue, Jan 26, 2016 at 8:22 PM, Pieter Hameete  wrote:

> Hi Ritesh,
>
> thanks for the response! The metrics are already being gathered though, so
> I think it would be nice to have a configuration/option to log them
> somewhere. It doesnt have to be enabled by default, and I dont think it
> should degrade the performance very much. It looks like the metrics are
> currently sent with each heartbeat by default already. Your Web UI probably
> hangs because it has to update all the graphs on every heartbeat, when you
> have many task managers that will be heavy on your computer :-)
>
> - Pieter
>
> 2016-01-26 20:17 GMT+01:00 Ritesh Kumar Singh <
> riteshoneinamill...@gmail.com>:
>
>> Going by the list in the latest documentation
>> 
>> for Flink 0.10.1 release, memory and cpu stats are not stored. Neither is
>> the time spent on garbage collection stored anywhere.
>>
>> In my opinion, trying to store these metrics will degrade the performance
>> of jobs too. And so its basically a trade off between performance and
>> computation cost. For me, the web ui hangs even for the current set of
>> parameters :(
>>
>> *Ritesh Kumar Singh,*
>> *https://riteshtoday.wordpress.com/* 
>>
>> On Tue, Jan 26, 2016 at 7:16 PM, Pieter Hameete 
>> wrote:
>>
>>> Hi people!
>>>
>>> A lot of metrics are gathered for each TaskManager every few seconds.
>>> The web UI shows nice graphs for some of these metrics too.
>>>
>>> I would like to make graphs of the memory and cpu usage, and the time
>>> spent on garbage collection for each job. Because of this I am wondering if
>>> the metrics are also stored somewhere, or if there is an option to enable
>>> storing the metrics per job.
>>>
>>> In the configuration documentation I could not find such an option. Is
>>> this possible in version 0.9.1 of Flink? If not: is it possible in Flink
>>> 0.10.1 or is it possible to request or develop such a feature?
>>>
>>> Thank you for your help and kind regards,
>>>
>>> Pieter
>>>
>>>
>>>
>>
>