>From my understanding, what you want is actually a management system
for Flink jobs. I think it might be good to submit the job(with `flink
run`) and retrieve the WebUI in another process.
Best,
Yangze Guo
On Mon, Aug 2, 2021 at 10:39 PM Hailu, Andreas [Engineering]
<andreas.ha...@gs.com> wrote:
>
> Hi Yangze, sure!
>
> After a submitted Flink app is complete, our client app polls the RESTful
> interface to pull job metrics -- operator start/end times, duration, records
> + bytes read/written etc... All of these metrics are all published to a
> database for analytical purposes, again both programmatic and ad-hoc.
>
> There was no clear exposure of ClusterClient, so we had originally worked
> around this by extending the CliFrontend class with a bit of a façade class
> that grabbed the ClusterClient from the executeProgram() method:
>
> @Override
> protected void executeProgram(PackagedProgram program, ClusterClient client,
> int parallelism) throws ProgramMissingJobException,
> ProgramInvocationException {
> logAndSysout("Starting execution of program");
> System.setProperty(JOB_MANAGER_WEB_INTERFACE_PROPERTY,
> client.getWebInterfaceURL()); // <- Used elsewhere in application
> ...
> }
>
> These metrics prove immensely valuable as they help us optimize performance,
> diagnose issues, as well as predict resource requirements for applications.
>
> // ah
>
> -----Original Message-----
> From: Yangze Guo <karma...@gmail.com>
> Sent: Sunday, August 1, 2021 10:38 PM
> To: Hailu, Andreas [Engineering] <andreas.ha...@ny.email.gs.com>
> Cc: user@flink.apache.org
> Subject: Re: Obtain JobManager Web Interface URL
>
> AFAIK, the ClusterClient should not be exposed through the public API.
> Would you like to explain your use case and why you need to get the web UI
> programmatically?
>
> Best,
> Yangze Guo
>
> On Fri, Jul 30, 2021 at 9:54 PM Hailu, Andreas [Engineering]
> <andreas.ha...@gs.com> wrote:
> >
> > Hello Yangze, thanks for responding.
> >
> > I'm attempting to perform this programmatically on YARN, so looking at a
> > log just won't do :) What's the appropriate way to get an instance of a
> > ClusterClient? Do you know of any examples I can look at?
> >
> > // ah
> >
> > -----Original Message-----
> > From: Yangze Guo <karma...@gmail.com>
> > Sent: Thursday, July 29, 2021 11:17 PM
> > To: Hailu, Andreas [Engineering] <andreas.ha...@ny.email.gs.com>
> > Cc: user@flink.apache.org
> > Subject: Re: Obtain JobManager Web Interface URL
> >
> > Hi, Hailu
> >
> > AFAIK, the ClusterClient#getWebInterfaceURL has been available since 1.10.
> >
> > Regarding the JobManager web interface, it will be print in the logs when
> > staring a native Kubernetes or Yarn cluster. In standalone mode, it is
> > configured by yourself[1].
> >
> > [1]
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__ci.apache.org_pro
> > jects_flink_flink-2Ddocs-2Dmaster_docs_deployment_resource-2Dproviders
> > _standalone_overview_-23starting-2Dand-2Dstopping-2Da-2Dcluster&d=DwIF
> > aQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=hRr4SA7BtUvKoMBP6VD
> > hfisy2OJ1ZAzai-pcCC6TFXM&m=FVv2XIIuWzaAGdj6tz9whXTJ5GQ_xgAqIgesdgtEjG4
> > &s=Cu-w4-hIu8MGtvnq2Ob8StpWCZhbFmwN4knnt35NqOM&e=
> >
> > Best,
> > Yangze Guo
> >
> > On Fri, Jul 30, 2021 at 1:41 AM Hailu, Andreas [Engineering]
> > <andreas.ha...@gs.com> wrote:
> > >
> > > Hi team,
> > >
> > >
> > >
> > > Is there a method available to obtain the JobManager’s REST url? We
> > > originally overloaded CliFrontend#executeProgram and nabbed it from the
> > > ClusterClient#getWebInterfaceUrl method, but it seems this method’s
> > > signature has been changed and no longer available as of 1.10.0.
> > >
> > >
> > >
> > > Best,
> > >
> > > Andreas
> > >
> > >
> > >
> > >
> > > ________________________________
> > >
> > > Your Personal Data: We may collect and process information about you
> > > that may be subject to data protection laws. For more information
> > > about how we use and disclose your personal data, how we protect
> > > your information, our legal basis to use your information, your
> > > rights and who you can contact, please refer to:
> > > http://www.gs.com/privacy-notices
> >
> > ________________________________
> >
> > Your Personal Data: We may collect and process information about you
> > that may be subject to data protection laws. For more information
> > about how we use and disclose your personal data, how we protect your
> > information, our legal basis to use your information, your rights and
> > who you can contact, please refer to:
> > www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>
>
> ________________________________
>
> Your Personal Data: We may collect and process information about you that may
> be subject to data protection laws. For more information about how we use and
> disclose your personal data, how we protect your information, our legal basis
> to use your information, your rights and who you can contact, please refer
> to: www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>
