Hi Chesnay and Dawid,
I see multiple entries as following in Log:
2020-08-26 23:46:19,105 WARN
org.apache.flink.runtime.metrics.MetricRegistryImpl - Error while
registering metric: numRecordsIn.
java.lang.IllegalArgumentException: A metric named
ip-99-9999-99-99.taskmanager.container_1596056409708_1570_01_000006.vdcs-kafka-flink-test.Map.0.numRecordsIn
already exists
at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91)
2020-08-26 23:46:19,094 WARN
org.apache.flink.runtime.metrics.MetricRegistryImpl - Error while
registering metric: numRecordsOut.
java.lang.IllegalArgumentException: A metric named
ip-99-9999-99-999.taskmanager.container_1596056409708_1570_01_000005.vdcs-kafka-flink-test.Map.2.numRecordsOut
already exists
at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91)
at
org.apache.flink.dropwizard.ScheduledDropwizardReporter.notifyOfAddedMetric(ScheduledDropwizardReporter.java:131)
at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:91)
at
org.apache.flink.dropwizard.ScheduledDropwizardReporter.notifyOfAddedMetric(ScheduledDropwizardReporter.java:131)
at
org.apache.flink.runtime.metrics.MetricRegistryImpl.register(MetricRegistryImpl.java:343)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.addMetric(AbstractMetricGroup.java:426)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:359)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.counter(AbstractMetricGroup.java:349)
at
org.apache.flink.runtime.metrics.groups.OperatorIOMetricGroup.<init>(OperatorIOMetricGroup.java:41)
at
org.apache.flink.runtime.metrics.groups.OperatorMetricGroup.<init>(OperatorMetricGroup.java:48)
at
org.apache.flink.runtime.metrics.groups.TaskMetricGroup.lambda$getOrAddOperator$0(TaskMetricGroup.java:154)
at java.util.HashMap.computeIfAbsent(HashMap.java:1127)
at
org.apache.flink.runtime.metrics.groups.TaskMetricGroup.getOrAddOperator(TaskMetricGroup.java:154)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.setup(AbstractStreamOperator.java:180)
at
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.setup(AbstractUdfStreamOperator.java:82)
at
org.apache.flink.streaming.api.operators.SimpleOperatorFactory.createStreamOperator(SimpleOperatorFactory.java:75)
at
org.apache.flink.streaming.api.operators.StreamOperatorFactoryUtil.createOperator(StreamOperatorFactoryUtil.java:48)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.createChainedOperator(OperatorChain.java:429)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:353)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:144)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:433)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:461)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
at java.lang.Thread.run(Thread.java:748)
Regards,
Vijay
On Wed, Aug 26, 2020 at 7:53 AM Chesnay Schepler <ches...@apache.org> wrote:
> metrics.reporter.grph.class:
> org.apache.flink.metrics.graphite.GraphiteReporter
>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.10/monitoring/metrics.html#graphite-orgapacheflinkmetricsgraphitegraphitereporter
>
> On 26/08/2020 16:40, Vijayendra Yadav wrote:
>
> Hi Dawid,
>
> I have 1.10.0 version of flink. What is alternative for this version ?
>
> Regards,
> Vijay
>
>
> On Aug 25, 2020, at 11:44 PM, Dawid Wysakowicz <dwysakow...@apache.org>
> <dwysakow...@apache.org> wrote:
>
>
>
> Hi Vijay,
>
> I think the problem might be that you are using a wrong version of the
> reporter.
>
> You say you used flink-metrics-graphite-1.10.0.jar from 1.10 as a plugin,
> but it was migrated to plugins in 1.11 only[1].
>
> I'd recommend trying it out with the same 1.11 version of Flink and
> Graphite reporter.
>
> Best,
>
> Dawid
>
> [1] https://issues.apache.org/jira/browse/FLINK-16965
> On 26/08/2020 08:04, Vijayendra Yadav wrote:
>
> Hi Nikola,
>
> To rule out any other cluster issues, I have tried it in my local now.
> Steps as follows, but don't see any metrics yet.
>
> 1) Set up local Graphite
>
> docker run -d\
> --name graphite\
> --restart=always\
> -p 80:80\
> -p 2003-2004:2003-2004\
> -p 2023-2024:2023-2024\
> -p 8125:8125/udp\
> -p 8126:8126\
> graphiteapp/graphite-statsd
>
> Mapped Ports
> Host Container Service
> 80 80 nginx <https://www.nginx.com/resources/admin-guide/>
> 2003 2003 carbon receiver - plaintext
> <http://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol>
> 2004 2004 carbon receiver - pickle
> <http://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-pickle-protocol>
> 2023 2023 carbon aggregator - plaintext
> <http://graphite.readthedocs.io/en/latest/carbon-daemons.html#carbon-aggregator-py>
> 2024 2024 carbon aggregator - pickle
> <http://graphite.readthedocs.io/en/latest/carbon-daemons.html#carbon-aggregator-py>
> 8080 8080 Graphite internal gunicorn port (without Nginx proxying).
> 8125 8125 statsd
> <https://github.com/etsy/statsd/blob/master/docs/server.md>
> 8126 8126 statsd admin
> <https://github.com/etsy/statsd/blob/master/docs/admin_interface.md>
> 2) WebUI:
>
> <image.png>
>
>
>
> 3) Run Flink example Job.
> ./bin/flink run
> ./examples/flink-examples-streaming_2.11-1.11-SNAPSHOT-SocketWindowWordCount.jar
> --port 9999
>
> with conf/flink-conf.yaml set as:
>
> metrics.reporter.grph.factory.class:
> org.apache.flink.metrics.graphite.GraphiteReporterFactory
> metrics.reporter.grph.host: localhost
> metrics.reporter.grph.port: 2003
> metrics.reporter.grph.protocol: TCP
> metrics.reporter.grph.interval: 1 SECONDS
>
> and graphite jar:
>
> plugins/flink-metrics-graphite/flink-metrics-graphite-1.10.0.jar
>
>
> 4) Can't see any activity in webui graphite.
>
>
> Could you review and let me know what is wrong here ? any other way you
> suggest to be able to view the raw metrics data ?
> Also, do you have sample metrics raw format, you can share from any other
> project.
>
> Regards,
> Vijay
>
>
>
>
> On Sun, Aug 23, 2020 at 9:26 PM Nikola Hrusov <n.hru...@gmail.com> wrote:
>
>> Hi Vijay,
>>
>> Your steps look correct to me.
>> Perhaps you can double check that the graphite port you are sending is
>> correct? THe default carbon port is 2003 and if you use the aggregator it
>> is 2023.
>>
>> You should be able to see in both flink jobmanager and taskmanager that
>> the metrics have been initialized with the config you have pasted.
>>
>> Regards
>> ,
>> Nikola Hrusov
>>
>>
>> On Mon, Aug 24, 2020 at 5:00 AM Vijayendra Yadav <contact....@gmail.com>
>> wrote:
>>
>>> Hi Team,
>>>
>>> I am trying to export Flink stream default metrics using Graphite, but
>>> I can't find it in the Graphite metrics console. Could you confirm the
>>> steps below are correct?
>>>
>>> *1) Updated flink-conf.yaml*
>>>
>>> metrics.reporter.grph.factory.class:
>>> org.apache.flink.metrics.graphite.GraphiteReporterFactory
>>> metrics.reporter.grph.host: port
>>> metrics.reporter.grph.port: 9109
>>> metrics.reporter.grph.protocol: TCP
>>> metrics.reporter.grph.interval: 30 SECONDS
>>>
>>> 2) Added Graphite jar in plugin folder :
>>>
>>> ll */usr/lib/flink/plugins/metric/*
>>> *flink-metrics-graphite-1.10.0.jar*
>>>
>>> 3) Looking metrics in graphite server:
>>>
>>> http://port:8080/metrics <http://10.108.58.63:8080/metrics>
>>>
>>> Note: No code change is done.
>>>
>>> Regards,
>>> Vijay
>>>
>>>
>>>
>
