I just did the same test as you had with SocketWindowWordCount, and the
counter showed up all right.
You should probably connect Jconsole to localhost:28781 (or whatever port
you have your JMX server listened on)
That's how I setup the env, perhaps there is other better ways to do it.
On Wed, Sep 20, 2017 at 9:15 AM, Jiewen Shao <jiewens...@gmail.com> wrote:
> Still got stuck, here are my steps (on my laptop)
>
> for example:
> Step1:
>
> public class MetricsTest<T> extends RichMapFunction<T, T> {
>
>
> private static final long serialVersionUID = 1L;
>
> private org.apache.flink.metrics.Meter meter;
>
> private Counter counter;
>
>
> @Override
>
> public void open(Configuration config) {
>
> this.counter = getRuntimeContext()
>
> .getMetricGroup()
>
> .counter("my-counter");
>
>
>
> this.meter = getRuntimeContext()
>
> .getMetricGroup()
>
> .meter("my-meter", new DropwizardMeterWrapper(new
> com.codahale.metrics.Meter()));
>
> }
>
>
> @Override
>
> public T map(T item) throws Exception {
>
> this.counter.inc();
>
> this.meter.markEvent();
>
> return item;
>
> }
>
> }
>
>
>
>
> And I did followings in one of the Flink sample
> (SocketWindowWordCount.java):
> Step2:
>
> DataStream<String> text = env.socketTextStream("localhost", 12345, "\n");
>
> text.map(new MetricsTest()); //<-- added this line
>
>
> Step3:
>
> mvn clean install
>
>
> step4: nc -l 12345
>
>
> step5:
>
> flink run -c [my_class_name] my.jar
>
>
> step6: (type something under nc terminal)
>
> run jconsole, and pick the local process for this "flink run", and click
> the tab "MBeans" (I don't see my metrics other than system ones, is that
> the right place to look at?)
>
>
> and flink-conf.yaml has:
>
> # metrics
>
> metrics.reporters: jmx
>
> metrics.reporter.jmx.class: org.apache.flink.metrics.jmx.JMXReporter
>
> metrics.reporter.jmx.port: 28780-28790
>
>
> and taskmanager log looks ok regarding JMX
>
>
> did I miss steps or configurations? Thanks a lot!
>
>
>
>
> On Mon, Sep 18, 2017 at 12:30 AM, Michael Fong <mcfong.o...@gmail.com>
> wrote:
>
>> Hi,
>>
>> There are several possibilities:
>> 1. Please check if reporter is set up ( guide
>> <https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html#jmx-orgapacheflinkmetricsjmxjmxreporter>
>> )
>> For example, I would make sure my local JMXReporter service is up and
>> running by checking taskmanager.log and search for the line:
>>
>> 2017-09-18 15:18:57,174 INFO org.apache.flink.metrics.jmx.JMXReporter
>> - Started JMX server on port 28781.
>> 2017-09-18 15:18:57,175 INFO org.apache.flink.metrics.jmx.JMXReporter
>> - Configured JMXReporter with {port:28780-28790}
>>
>> If for any reason the JMX server does not start up, your might see some
>> errors:
>>
>> 2017-09-18 15:26:04,743 INFO org.apache.flink.runtime.metrics.MetricRegistry
>> - Configuring JMXReporter with {port=28781, class=org.apac
>> he.flink.metrics.jmx.JMXReporter}.
>> 2017-09-18 15:26:04,760 ERROR org.apache.flink.runtime.metrics.MetricRegistry
>> - Could not instantiate metrics reporter jmx. Metrics migh
>> t not be exposed/reported.
>> java.lang.RuntimeException: Could not start JMX server on any configured
>> port. Ports: 28781
>> at org.apache.flink.metrics.jmx.JMXReporter.open(JMXReporter.ja
>> va:126)
>> at org.apache.flink.runtime.metrics.MetricRegistry.<init>(Metri
>> cRegistry.java:131)
>> at org.apache.flink.runtime.taskexecutor.TaskManagerServices.fr
>> omConfiguration(TaskManagerServices.java:188)
>> at org.apache.flink.runtime.taskmanager.TaskManager$.startTaskM
>> anagerComponentsAndActor(TaskManager.scala:1984)
>> at org.apache.flink.runtime.taskmanager.TaskManager$.runTaskMan
>> ager(TaskManager.scala:1823)
>> at org.apache.flink.runtime.taskmanager.TaskManager$$anonfun$1.
>> apply$mcV$sp(TaskManager.scala:1926)
>> at org.apache.flink.runtime.taskmanager.TaskManager$$anonfun$1.
>> apply(TaskManager.scala:1904)
>> at org.apache.flink.runtime.taskmanager.TaskManager$$anonfun$1.
>> apply(TaskManager.scala:1904)
>> at scala.util.Try$.apply(Try.scala:192)
>>
>>
>> Here is my local setup for conf/flink-conf.yaml for example:
>> metrics.reporters: jmx
>> metrics.reporter.jmx.class: org.apache.flink.metrics.jmx.JMXReporter
>> metrics.reporter.jmx.port: 28780-28790
>>
>> 2. You might want to try a real streaming example which could execute
>> continuously. If I remember correctly, when the task is completed, the
>> manager would seem to release the associated resource and object. In your
>> example, it is only processing a few strings, which would finish in matter
>> of milliseconds, before bringing up jconsole manually.
>>
>> Hope some of these help,
>>
>>
>>
>> On Mon, Sep 18, 2017 at 12:22 PM, Jiewen Shao <jiewens...@gmail.com>
>> wrote:
>>
>>> Thanks, When I started jconsole, it listed com.apache.flink.runtime.
>>> jobmanager..:[port] as one of the Local Process, i was able to connect
>>> to it with insecure connection, but i was not able to locate the Counter
>>> metrics, I only saw some system metrics.
>>>
>>> On Sun, Sep 17, 2017 at 7:39 PM, Michael Fong <mcfong.o...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> You may enable metrics reporter to see the output of your metrics;
>>>> counter in your example.
>>>>
>>>> There is a brief documentation regarding to metrics and reporter setup
>>>> at link
>>>> <https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html>.
>>>> The easiest approach, in my opinion, is to set up a JMX reporter so that
>>>> you may see your metrics via JConsole.
>>>>
>>>> Hope this helps.
>>>>
>>>> Regrads,
>>>>
>>>>
>>>> On Mon, Sep 18, 2017 at 10:27 AM, Jiewen Shao <jiewens...@gmail.com>
>>>> wrote:
>>>>
>>>>> I'm new to flink and I have read https://ci.apache.org/pro
>>>>> jects/flink/flink-docs-release-1.3/monitoring/metrics.html, I am
>>>>> still unclear where do I read the metrics I added.
>>>>>
>>>>> for example,
>>>>>
>>>>> public static void main(String[] args) throws Exception {
>>>>>
>>>>>
>>>>> StreamExecutionEnvironment env =
>>>>> StreamExecutionEnvironment.getExecutionEnvironment();
>>>>>
>>>>> env.setParallelism(2);
>>>>>
>>>>>
>>>>>
>>>>> List<String> wordList = Arrays.asList("Hive", "Presto",
>>>>> "Impala", "Parquet","ORC","Hadoop", "Flink", "Spark", "Storm", "Tez",
>>>>> "Flink");
>>>>>
>>>>> DataStreamSource<String> source = env.fromCollection(wordList
>>>>> );
>>>>>
>>>>> DataStream<Tuple2<String, Integer>> dataStream = env
>>>>> .fromCollection(wordList).map(new WordLengthCounter());
>>>>>
>>>>> dataStream.print();
>>>>>
>>>>> env.execute();
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> and
>>>>>
>>>>>
>>>>> public class WordLengthCounter extends RichMapFunction<String,
>>>>> Tuple2<String, Integer>> {
>>>>>
>>>>>
>>>>> private static final long serialVersionUID = 1L;
>>>>>
>>>>> private Counter counter;
>>>>>
>>>>>
>>>>> @Override
>>>>>
>>>>> public void open(Configuration config) {
>>>>>
>>>>> this.counter = getRuntimeContext()
>>>>>
>>>>> .getMetricGroup()
>>>>>
>>>>> .counter("myCounter");
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> @Override
>>>>>
>>>>> public Tuple2<String, Integer> map(String value) throws Exception {
>>>>>
>>>>> this.counter.inc();
>>>>>
>>>>> return new Tuple2<String, Integer>(value, value.length());
>>>>>
>>>>> }
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>> Now, where do I see the counter? Sorry for the naive question
>>>>>
>>>>> can anyone point me to any good end-to-end "hello world" example for
>>>>> flink metrics.
>>>>>
>>>>
>>>>
>>>
>>
>
