Hi, Etienne! The actual command line is “mvn clean install”, because I set the
arguments in the code:
public void testFlinkRunner() throws Exception {
String[] args = new String[]{"--runner=FlinkRunner", "--streaming=true",
"--inputTopic="+topic};
run(args);
}
I’m using Beam 2.8.0, and Flink 1.5.4.
Because I wasn’t sure whether I was in detached mode, I called the toString()
method on the Flink runner result to see if I would get
FlinkDetachedRunnerResult. This is what was returned:
FlinkRunnerResult{accumulators={__metricscontainers=org.apache.beam.runners.core.metrics.MetricsContainerStepMap@b799384b},
runtime=33126}
I set the options for MetricsHttpSink:
options.setMetricsHttpSinkUrl("http://localhost:3000";);
options.setMetricsSink(MetricsHttpSink.class);
This works when I test SparkRunner, so I believe I have it set up correctly for
MetricsPusher to capture the metrics from Flink as well.
-Phil
On 2018/12/07 14:46:06, Etienne Chauchot <[email protected]> wrote:
> Hi Phil,>
> MetricsPusher is tested on all the runners in both batch and streaming mode.
> I just ran this test in Flink in streaming>
> mode and it works.>
> What is the command line you are using and which version of Beam?>
>
> Please also remember that, as discussed, metrics (other flink features ) do
> not work if flink is used in detached mode.>
>
> Etienne>
>
>
> Le mardi 04 décembre 2018 à 12:49 -0600, Phil Franklin a écrit :>
> > I’m having difficulty accessing Beam metrics when using FlinkRunner in
> > streaming mode. I don’t get any metrics from MetricsPusher, though the same
> > setup delivered metrics from SparkRunner. Probably for the same reason
> > that MetricsPusher doesn’t work, I also don’t get any output when I call an
> > instance of MetricsHttpSink directly. The problem seems to be that Flink
> > never returns from pipeline.run(), an issue that others have referred to as
> > FlinkRunner hanging. >
> > >
> > Is there a solution for getting metrics in this case that I’m missing?>
> > >
> > Thanks!>
> > -Phil>
>
