I don't know how/whether you can provide different flink-conf.yaml files
when using YARN.
But this is the only way to map specific ports to a specific JM/TM process.
On 31.08.2018 08:15, vino yang wrote:
Hi Deirdre,
Sorry, I thought you were using a Standalone cluster environment.
If you are based on YARN, then it seems that the way I said does not
work. Maybe you specify a larger port range and will not cause conflicts.
I am curious, why do you want to fix the run port of the reporter?
Maybe you can expect Chesnay to give you advice.
Thanks, vino.
Deirdre Kong <[email protected]
<mailto:[email protected]>> 于2018年8月31日周五 下午1:26写道：
Hi Vino/Chesnay,
Thank you for the info.
I am actually using Yarn for deployment. Flink is installed in
AWS EMR, so sometimes jm and tm processes are deployed in the same
container, sometimes they are deployed in different containers. I
would need to configure Prometheus to listen on a specify port
for JM and TM reporter. So even if JM is bounced, I can get the
JM metrics on the same port each time.
Can you elaborate what do you mean by specify a separate Flink
installation package for it?
Chesnay, do you have any insights on this?
Thanks,
Deirdre
On Thu, Aug 30, 2018 at 7:18 PM vino yang <[email protected]
<mailto:[email protected]>> wrote:
Hi Deirdre,
If you run multiple Flink component (jm/tm) processes on one
physical node, it is recommended to specify the port range to
avoid conflicts[1], I guess this is based on the same Flink
binary installation package.
If you want to always have the TM reporter running on the same
port, you can specify a separate Flink installation package
for it, explicitly specifying this port in the configuration
file for this installation package.
However, you still need to pay attention to port conflicts
with other components.
The issue you provided is handled by Chesnay, so maybe
Chesnay opinion is more appropriate. Ping Chesnay for you.
[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.6/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter
Thanks, vino.
Deirdre Kong <[email protected]
<mailto:[email protected]>> 于2018年8月31日周五
上午5:25写道：
Hi,
I am running Flink in Amazon EMR. In flink-conf.yaml, I
have `metrics.reporter.prom.port: 9249-9250`
Depending whether the job manager and task manager are
running in the same node, the task manager metrics are
reported on port 9250 (if running on same node as job
manager), or on port 9249 (if running on a different node).
Is there a way to configure so that the task manager
metrics are always reported on port 9250?
I saw a post
<https://issues.apache.org/jira/browse/FLINK-8284> that we
can "provide each *Manager with a separate
configuration." How to do that?
Thanks
