Yea, that was I was thinking..
Batch can be quick
Can I report metric on "added" action ? (i should override the
notifyOnAddedMetric to report ?)
------------------
Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io
Le mer. 28 nov. 2018 à 15:54, Chesnay Schepler <ches...@apache.org> a
écrit :
> How quick does job batch terminate? Metrics are unregistered once the job
> ends; if the job duration is shorter than the report interval they may
> never be exposed.
>
> On 28.11.2018 15:18, bastien dine wrote:
>
> Hello Chesnay,
>
> Thanks for your response !
> I have logs enable (info), slf4jReporter is working, I can see :
>
> 15:16:00.112 [Flink-MetricRegistry-thread-1] INFO
> org.apache.flink.metrics.slf4j.Slf4jReporter -
> =========================== Starting metrics report
> ===========================
>
> -- Counters
> -------------------------------------------------------------------
>
> -- Gauges
> ---------------------------------------------------------------------
> ....
>
> On both jobmanager & taskmanager,
> *BUT* i see only system metrics, not my custom one..
> Am i missing something in declaration in my topology ?
>
> *Note* : I am using DataSet API (so my program is batch, and not
> continuous)
>
> Regards,
> Bastien
>
> ------------------
>
> Bastien DINE
> Data Architect / Software Engineer / Sysadmin
> bastiendine.io
>
>
> Le mar. 27 nov. 2018 à 17:07, Chesnay Schepler <ches...@apache.org> a
> écrit :
>
>> Please enable WARN logging and check for warnings by the SLF4JReporter
>> and/or MetricQueryService.
>>
>> On 27.11.2018 17:00, bastien dine wrote:
>>
>> Hello everyone,
>>
>> Once again I require your help !
>> I am trying to report custom metric (see my code below)
>> Yet, I do not see them anywhere.. nor in the metric tab from my tasks,
>> nor in the rest API, nor in the declared slf4j reporter..
>> Can someone help me to debug this ..
>>
>> Here is my RichMap function :
>>
>> public class MetricGaugeRichMap<T, E extends MetricTuple<T>> extends
>> RichMapFunction<E, E> {
>>
>> private transient T valueToExpose;
>> private final String metricGroup;
>> private final String metricName;
>>
>>
>> public MetricGaugeRichMap(String metricGroup, String metricName) {
>> this.metricGroup = metricGroup;
>> this.metricName = metricName;
>> }
>>
>> @Override public void open(Configuration config) {
>> getRuntimeContext()
>> .getMetricGroup()
>> .addGroup(metricGroup)
>> .gauge(metricName, (Gauge<T>) () -> valueToExpose);
>> }
>>
>> @Override public E map(E metricTuple) throws Exception {
>> valueToExpose = metricTuple.getMetricValue();
>> return metricTuple;
>> }
>> }
>>
>> calling from topology :
>>
>> env.fromElements(new MetricTuple<>(metricGroup, metricName, metricValue))
>> .map(new MetricGaugeRichMap<>(metricGroup, metricName))
>> .output(new MetricGaugeOutputFormat<>()); // dummy output
>>
>> ------------------
>>
>>
>> Bastien DINE
>> Data Architect / Software Engineer / Sysadmin
>> bastiendine.io
>>
>>
>>
>
