Hi:
I need to collect application metrics which are counts (per unit of time eg: 
minute)  for certain events.  There are two ways of doing this:
1. Create separate streams (using split stream etc) in the application 
explicitly, then aggregate the counts in a window and save them.  This mixes 
metrics collection with application logic and making the application logic 
complex.2. Use Flink metrics framework (counter, guage, etc) to save metrics
I have a very small test with 2 events but when I run the application the 
counters are not getting saved (they show value 0) even though that part of the 
code is being executed.  I do see the numRecordsIn counters being updated from 
the source operator.  I've also tried incrementing the count by 10 (instead of 
1) every time the function gets execute but still the counts remain 0.
Here is snippet of the code:
dataStream.map(new RichMapFunction<String, String>() {
            protected Counter counter;
            public void open(Configuration parameters) {                counter 
= getRuntimeContext().getMetricGroup().addGroup("test", 
"split").counter("success");            }            @Override            
public String map(String value) throws Exception {                
counter.inc();                return value;            }        });

As I mentioned, I do get the success metric count but the value is always 0, 
even though the above map function was executed.  
My questions are:
1. Are there any issues regarding counters being approximate ?2. If I want to 
collect accurate counts, is it recommended to use counters or should I do it 
explicitly (which is making the code too complex) ?3. Do counters participate 
in flink's failure/checkpointing/recovery ?4. Is there any better way of 
collecting application metric counts ?
Thanks
Mans

Reply via email to