Re: Some questions about storm metrics

2015-09-21 Thread huangwei (G)
@Matthias Thanks for the advice. I opened a JIRA and a PR for the first step about the storm metrics. If you think this is OK, I'll do a integrated test and add all the storm metrics in future. Greetings, Wei Huang = Hi, there is no need to apol

Re: Some questions about storm metrics

2015-09-21 Thread Matthias J. Sax
Hi, there is no need to apologize :) If I understand you correctly, you need access to RuntimeContext within FlinkTopologyContext. FlinkTopologyContext in created within StormWrapperSetupHelper.convertToTopologyContext which has already access to StreamingRuntimeContext (which is just a special R

Re: Some questions about storm metrics

2015-09-19 Thread huangwei (G)
Hi Matthias, Very sorry for the email in weekend. I try to add a wrapper(FlinkCountMetric) for the Storm-CountMetric and it's also easy to do it. But I get a trouble in `FlinkTopologyContext.registerMetric` method that I don't know how to use the `getRuntimeContext` in this class since I want to

答复: Some questions about storm metrics

2015-09-08 Thread huangwei (G)
@Matthias Thanks for advice :-) == I personally never used Metrics in Storm. It would be best, if you design an own example. I guess you are familiar with https://storm.apache.org/documentation/Metrics.html As a first step, I would try to include the predefi

Re: Some questions about storm metrics

2015-09-07 Thread Matthias J. Sax
I personally never used Metrics in Storm. It would be best, if you design an own example. I guess you are familiar with https://storm.apache.org/documentation/Metrics.html As a first step, I would try to include the predefined Storm metric and work on custom metrics later. As the Storm documentat

Re: Some questions about storm metrics

2015-09-07 Thread huangwei (G)
Hi Kostas, Thanks for your support! I created the JIRA and pushed a PR for it. @Matthias Could you give me a storm example in using the cust metrics? Since it is needed for follow-up development. Many thanks! Greetings, Huang Wei = Sounds great. It is easy t

Re: 答复: Some questions about storm metrics

2015-09-02 Thread Kostas Tzoumas
Sounds great. It is easy to create an AverageAccumulator by implementing the org.apache.flink.api.common.accumulators.Accumulator interface, accumulating the sum and the count internally, and dividing in the end. This would be something useful also outside of Storm compatibility. Then, MultiReduc

答复: Some questions about storm metrics

2015-09-01 Thread huangwei (G)
Dear all, I focused on the Flink-accumulator and found that there is not any feature to count the average of values. And there is a MultiReducedMetric class in storm that calculates the average. This class can be used internally to get the latency in storm. So as the first step, I`d like to add a

Re: Some questions about storm metrics

2015-09-01 Thread huangwei (G)
Hi Kostas, Thank you for the information. I`ll take a look. And I`m comparing the storm metrics with Flink accumulators. I`ll let you know if I have some ideas. Flink accumulators are documented here: https://ci.apache.org/projects/flink/flink-docs-master/

Re: 答复: Some questions about storm metrics

2015-09-01 Thread Kostas Tzoumas
Flink accumulators are documented here: https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#accumulators--counters I think Aljoscha's idea is to implement Storm metrics on top of those. Huangwei, perhaps you can look whether the interfaces match (basically if Flink

Re: 答复: Some questions about storm metrics

2015-08-31 Thread Matthias J. Sax
I did not look into it and I am not super familiar with Flink's accumulators... Can not provide much help right now. Sorry :( -Matthias On 08/31/2015 01:01 PM, huangwei (G) wrote: > Hi, > I think I can take a simple design for it. > Do you have any idea? > > =

答复: Some questions about storm metrics

2015-08-31 Thread huangwei (G)
Hi, I think I can take a simple design for it. Do you have any idea? Maybe. I did not put any thoughts into this... On 08/31/2015 12:25 PM, Aljoscha Krettek wrote: > Hi, > could the metrics not be implemented on top of Flink accumulators? > They would ev

Re: Some questions about storm metrics

2015-08-31 Thread Matthias J. Sax
Maybe. I did not put any thoughts into this... On 08/31/2015 12:25 PM, Aljoscha Krettek wrote: > Hi, > could the metrics not be implemented on top of Flink accumulators? They > would even be live-updated in the new web front-end. > > Cheers, > Aljoscha > > On Mon, 31 Aug 2015 at 11:32 Matthias J

Re: Some questions about storm metrics

2015-08-31 Thread Aljoscha Krettek
Hi, could the metrics not be implemented on top of Flink accumulators? They would even be live-updated in the new web front-end. Cheers, Aljoscha On Mon, 31 Aug 2015 at 11:32 Matthias J. Sax wrote: > Hi, > > custom metrics are currently not supported by the compatibility layer. > See README her

Re: Some questions about storm metrics

2015-08-31 Thread Matthias J. Sax
Hi, custom metrics are currently not supported by the compatibility layer. See README here: https://github.com/apache/flink/tree/master/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core It is an open question, if/how Flink can support this feature. -Matthias On 08/31/2015 0

Some questions about storm metrics

2015-08-31 Thread huangwei (G)
Hi Matthias and all, I`d like to do the custom metrics in flink-storm-compatibility. And there are some questions following: Is there anything like storm metrics in Flink? If not, does Flink need to have metrics mechanism in internal first? Or just let the metrics use in flink-storm-compatibility?