Doh - stupid mistake. Thanks for pointing that out.
On Sun, Jun 2, 2019 at 12:52 PM Bowen Li <bowenl...@gmail.com> wrote:
> To answer your question on your debugging code, your reporter has a bug:
>
> log.info("STATSD SENDING: ", name, value);
>
> should be ->
>
> log.info("STATSD SENDING: {} {}", name, value);
>
>
> -
>
>
>
>
> On Sat, Jun 1, 2019 at 7:30 PM Padarn Wilson <pad...@gmail.com> wrote:
>
>> Thanks both: Using the the inbuilt Slf4j reporter is a great idea, I
>> will do this.
>>
>> @Peter.Groesbeck - appreciate the config. This looks very similar to
>> what I had, but if it is working for you perhaps there is something else
>> missing from our EMR setup. Will go back and double check the connectivity
>> from all the instances.
>>
>>
>>
>> On Thu, May 30, 2019 at 9:42 PM Peter Groesbeck <
>> peter.groesb...@gmail.com> wrote:
>>
>>> Hi Padarn for what it's worth I am using DataDog metrics on EMR with
>>> Flink 1.7.1 and this here my flink-conf configuration:
>>>
>>> - Classification: flink-conf
>>> ConfigurationProperties:
>>> metrics.reporter.dghttp.class:
>>> org.apache.flink.metrics.datadog.DatadogHttpReporter
>>> metrics.reporter.dghttp.apikey: <your-datadog-key>
>>> metrics.reporter.dghttp.tags:
>>> 'service:myservice,env:prod,region:us-east-1'
>>> metrics.scope.jm: 'jobmanager'
>>> metrics.scope.jm.job: 'jobmanager'
>>> metrics.scope.operator: 'taskmanager.<job_name>.<operator_name>'
>>> metrics.scope.task: 'taskmanager.<job_name>.<task_name>'
>>> metrics.scope.tm: 'taskmanager'
>>> metrics.scope.tm.job: 'taskmanager'
>>> Configurations: []
>>>
>>>
>>> On Thu, May 30, 2019 at 6:46 AM Yun Tang <myas...@live.com> wrote:
>>>
>>>> Hi Padarn
>>>>
>>>> If you want to verify why no metrics sending out, how about using the
>>>> built-in Slf4j reporter [1] which would record metrics in logs.
>>>> If you could view the metrics after enabled slf4j-reporter, you could
>>>> then compare the configurations.
>>>>
>>>> Best
>>>> Yun Tang
>>>>
>>>> [1]
>>>> https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#slf4j-orgapacheflinkmetricsslf4jslf4jreporter
>>>>
>>>> ------------------------------
>>>> *From:* Padarn Wilson <padarn.wil...@grab.com>
>>>> *Sent:* Thursday, May 30, 2019 18:20
>>>> *To:* user
>>>> *Subject:* [External] Flink 1.7.1 on EMR metrics
>>>>
>>>> Hello all,
>>>>
>>>> I am trying to run Flink 1.7.1 on EMR and having some trouble with
>>>> metric reporting.
>>>>
>>>> I was using the DataDogHttpReporter, but have also tried the
>>>> StatsDReporter, but with both was seeing no metrics being collected.
>>>>
>>>> To debug this I implemented my own reporter (based on StatsDReporter)
>>>> and logged the name of the metric being sent:
>>>>
>>>> private void send(final String name, final String value) {
>>>> log.info("STATSD SENDING: ", name, value);
>>>> try {
>>>> String formatted = String.format("%s:%s|g", name, value);
>>>> byte[] data = formatted.getBytes(StandardCharsets.UTF_8);
>>>> socket.send(new DatagramPacket(data, data.length, this.address));
>>>> }
>>>> catch (IOException e) {
>>>> LOG.error("unable to send packet to statsd at '{}:{}'",
>>>> address.getHostName(), address.getPort());
>>>> }
>>>> }
>>>>
>>>>
>>>> This code is certainly reached, because in my log I see a lot of this:
>>>>
>>>> 2019-05-30 10:18:40,351 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>> 2019-05-30 10:18:40,351 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>> 2019-05-30 10:18:40,351 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>> 2019-05-30 10:18:40,351 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>> 2019-05-30 10:18:40,351 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>> 2019-05-30 10:18:40,351 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>> 2019-05-30 10:18:40,352 INFO com.grab.statsd.StatsDReporter
>>>> - STATSD SENDING:
>>>>
>>>> As you can see, the name and value for the metric being reported is empty.
>>>>
>>>>
>>>> And the logs show everything initialized fine with no error:
>>>>
>>>> 2019-05-30 10:18:30,342 INFO
>>>> org.apache.flink.runtime.metrics.MetricRegistryImpl -
>>>> Configuring stsd with {port=8125, host=127.0.0.1,
>>>> class=com.grab.statsd.StatsDReporter}.
>>>> 2019-05-30 10:18:30,344 INFO com.grab.statsd.StatsDReporter
>>>> - Configured StatsDReporter with {host:127.0.0.1,
>>>> port:8125}
>>>> 2019-05-30 10:18:30,344 INFO
>>>> org.apache.flink.runtime.metrics.MetricRegistryImpl -
>>>> Periodically reporting metrics in intervals of 10 SECONDS for reporter
>>>> stsd of type com.grab.statsd.StatsDReporter.
>>>>
>>>>
>>>>
>>>>
>>>> Has anyone else tried to work with Flink and metrics on EMR 1.7.1 (latest
>>>> version on EMR). If so, any pointers as to what could be set up
>>>> incorrectly?
>>>>
>>>>
>>>>
>>>>
>>>> *Grab is hiring. Learn more at https://grab.careers
>>>> <https://grab.careers/>*
>>>>
>>>> By communicating with Grab Inc and/or its subsidiaries, associate
>>>> companies and jointly controlled entities (“Grab Group”), you are deemed to
>>>> have consented to the processing of your personal data as set out in the
>>>> Privacy Notice which can be viewed at https://grab.com/privacy/
>>>>
>>>> This email contains confidential information and is only for the
>>>> intended recipient(s). If you are not the intended recipient(s), please do
>>>> not disseminate, distribute or copy this email Please notify Grab Group
>>>> immediately if you have received this by mistake and delete this email from
>>>> your system. Email transmission cannot be guaranteed to be secure or
>>>> error-free as any information therein could be intercepted, corrupted,
>>>> lost, destroyed, delayed or incomplete, or contain viruses. Grab Group do
>>>> not accept liability for any errors or omissions in the contents of this
>>>> email arises as a result of email transmission. All intellectual property
>>>> rights in this email and attachments therein shall remain vested in Grab
>>>> Group, unless otherwise provided by law.
>>>>
>>>
