Yes, you can override notifyOnAddedMetric/notifyOnRemovedMetric to
ensure that a metric is logged at least once.
On 28.11.2018 16:01, bastien dine wrote:
Yea, that was I was thinking..
Batch can be quick
Can I report metric on "added" action ? (i should override the
notifyOnAddedMetric to report ?)
------------------
Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io <http://bastiendine.io>
Le mer. 28 nov. 2018 à 15:54, Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> a écrit :
How quick does job batch terminate? Metrics are unregistered once
the job ends; if the job duration is shorter than the report
interval they may never be exposed.
On 28.11.2018 15:18, bastien dine wrote:
Hello Chesnay,
Thanks for your response !
I have logs enable (info), slf4jReporter is working, I can see :
15:16:00.112 [Flink-MetricRegistry-thread-1] INFO
org.apache.flink.metrics.slf4j.Slf4jReporter -
=========================== Starting metrics report
===========================
-- Counters
-------------------------------------------------------------------
-- Gauges
---------------------------------------------------------------------
....
On both jobmanager & taskmanager,
*BUT* i see only system metrics, not my custom one..
Am i missing something in declaration in my topology ?
_Note_ : I am using DataSet API (so my program is batch, and not
continuous)
Regards,
Bastien
------------------
Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io <http://bastiendine.io>
Le mar. 27 nov. 2018 à 17:07, Chesnay Schepler
<ches...@apache.org <mailto:ches...@apache.org>> a écrit :
Please enable WARN logging and check for warnings by the
SLF4JReporter and/or MetricQueryService.
On 27.11.2018 17:00, bastien dine wrote:
Hello everyone,
Once again I require your help !
I am trying to report custom metric (see my code below)
Yet, I do not see them anywhere.. nor in the metric tab from
my tasks, nor in the rest API, nor in the declared slf4j
reporter..
Can someone help me to debug this ..
Here is my RichMap function :
public class MetricGaugeRichMap<T,E extends MetricTuple<T>> extends
RichMapFunction<E,E> {
private transient T valueToExpose;
private final StringmetricGroup;
private final StringmetricName;
public MetricGaugeRichMap(String metricGroup, String metricName) {
this.metricGroup = metricGroup;
this.metricName = metricName;
}
@Override public void open(Configuration config) {
getRuntimeContext()
.getMetricGroup()
.addGroup(metricGroup)
.gauge(metricName, (Gauge<T>) () -> valueToExpose);
}
@Override public E map(E metricTuple)throws Exception {
valueToExpose = metricTuple.getMetricValue();
return metricTuple;
}
}
calling from topology :
env.fromElements(new MetricTuple<>(metricGroup,metricName,metricValue))
.map(new MetricGaugeRichMap<>(metricGroup,metricName))
.output(new MetricGaugeOutputFormat<>()); // dummy output
------------------
Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io <http://bastiendine.io>
