@Chesnay, can you elaborate on how to map specific ports to a specific JM/TM process?
@Vino, I can only update Prometheus configuration once. Say I set my port to be 9249-9250 in flink-conf.yml, and configure Prometheus to listen on <JM-IP>:9249 for JM metrics and <TM-IP>:9250 for TM metrics. If JM and TM are deployed in the same container, then I have no issue. But if YARN deployed them in different containers then the TM metrics will expose on port 9249 instead. Thanks, Deirdre On Thu, Aug 30, 2018 at 11:47 PM Chesnay Schepler <ches...@apache.org> wrote: > I don't know how/whether you can provide different flink-conf.yaml files > when using YARN. > > But this is the only way to map specific ports to a specific JM/TM process. > > On 31.08.2018 08:15, vino yang wrote: > > Hi Deirdre, > > Sorry, I thought you were using a Standalone cluster environment. > If you are based on YARN, then it seems that the way I said does not work. > Maybe you specify a larger port range and will not cause conflicts. > I am curious, why do you want to fix the run port of the reporter? > > Maybe you can expect Chesnay to give you advice. > > Thanks, vino. > > Deirdre Kong <deirdre.y.k...@gmail.com> 于2018年8月31日周五 下午1:26写道： > >> Hi Vino/Chesnay, >> >> Thank you for the info. >> >> I am actually using Yarn for deployment. Flink is installed in AWS EMR, >> so sometimes jm and tm processes are deployed in the same container, >> sometimes they are deployed in different containers. I would need to >> configure Prometheus to listen on a specify port for JM and TM reporter. >> So even if JM is bounced, I can get the JM metrics on the same port each >> time. >> >> Can you elaborate what do you mean by specify a separate Flink >> installation package for it? >> >> Chesnay, do you have any insights on this? >> >> Thanks, >> Deirdre >> >> On Thu, Aug 30, 2018 at 7:18 PM vino yang <yanghua1...@gmail.com> wrote: >> >>> Hi Deirdre, >>> >>> If you run multiple Flink component (jm/tm) processes on one physical >>> node, it is recommended to specify the port range to avoid conflicts[1], I >>> guess this is based on the same Flink binary installation package. >>> If you want to always have the TM reporter running on the same port, you >>> can specify a separate Flink installation package for it, explicitly >>> specifying this port in the configuration file for this installation >>> package. >>> However, you still need to pay attention to port conflicts with other >>> components. >>> >>> The issue you provided is handled by Chesnay, so maybe Chesnay opinion >>> is more appropriate. Ping Chesnay for you. >>> >>> [1]: >>> https://ci.apache.org/projects/flink/flink-docs-release-1.6/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter >>> >>> Thanks, vino. >>> >>> Deirdre Kong <deirdre.y.k...@gmail.com> 于2018年8月31日周五 上午5:25写道： >>> >>>> Hi, >>>> >>>> I am running Flink in Amazon EMR. In flink-conf.yaml, I have >>>> `metrics.reporter.prom.port: >>>> 9249-9250` >>>> Depending whether the job manager and task manager are running in the >>>> same node, the task manager metrics are reported on port 9250 (if running >>>> on same node as job manager), or on port 9249 (if running on a different >>>> node). >>>> >>>> Is there a way to configure so that the task manager metrics are always >>>> reported on port 9250? >>>> I saw a post <https://issues.apache.org/jira/browse/FLINK-8284> that >>>> we can "provide each *Manager with a separate configuration." How to do >>>> that? >>>> >>>> Thanks >>>> >>> >
