Hi Sai, could you check that the dashboard you are seeing is really running on Yarn and not a standalone Flink cluster which you have running locally?
Cheers, Till On Mon, Dec 31, 2018 at 7:40 PM Sai Inampudi <sai.inamp...@gmail.com> wrote: > Hey Gary, thanks for reaching out. > > Executing "yarn application -list" does not return my flink cluster so I > assume like in my initial post that the application must be terminated. My > config when I ran the job did not have log aggregation enabled and that > might be why when I try to look at the logs via "yarn logs -applicationId > <YOUR_APP_ID>", I get back nothing: > (e.g. Unable to get ApplicationState. Attempting to fetch logs > directly from the filesystem. > /tmp/logs/si022833/logs/application_1545041832015_73428 does not > exist.) > > > (My previous reply was formatted incorrectly so I am replying back with > proper formatting. Apologies for the mistake) > > > On 2018/12/31 18:13:05, Sai Inampudi <sai.inamp...@gmail.com> wrote: > > > > > > On 2018/12/31 10:53:58, Gary Yao <g...@da-platform.com> wrote: > > > Hi, > > > > > > You can use the YARN client to list all applications on your YARN > cluster: > > > > > > yarn application -list > > > > > > If this does not show any running applications, the Flink cluster must > have > > > somehow terminated. If you have YARN's log aggregation enabled, you > should > > > be > > > able to view the Flink logs by running: > > > > > > yarn logs -applicationId <YOUR_APP_ID> > > > > > > Best, > > > Gary > > > > > > On Fri, Dec 28, 2018 at 9:42 PM Sai Inampudi <sai.inamp...@gmail.com> > wrote: > > > > > > > Hi everyone, > > > > > > > > I recently attempted to create a Flink cluster on YARN by executing > the > > > > following: > > > > ~/flink-1.5.4/bin/yarn-session.sh -n 5 -tm 2048 -s 4 -d -nm > flink_yarn > > > > > > > > The resulting command was not completely successful but it did end up > > > > creating a Apache Flink Dashboard with 1 Task Manager, 1 Task Slot, > and 1 > > > > Job Manager. > > > > > > > > When I look at my Yarn Resource Manager, I don't see my application > > > > running. CLI calls for the application id also returned nothing. > > > > > > > > I would like to kill the existing web dashboard as well as the other > > > > lingering task manager/job manager so that I can try recreating the > yarn > > > > session successfully. > > > > > > > > Has anyone encountered this before and has any suggestion? I looked > > > > through documentation [1] which says to stop a yarn session, you > will want > > > > to use the YARN utilities (yarn application -kill <appId>) to stop > the YARN > > > > session. However, the application id in my logs is not found in the > > > > Resource Manager so it seems to already have been killed (due to the > > > > original yarn session command not properly executing?). > > > > > > > > > > > > > > > > > > > > [1] > > > > > https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/deployment/yarn_setup.html#detached-yarn-session > > > > > > > Hey Gary, thanks for reaching out. > > Executing "yarn application -list" does not return my flink cluster so I > assume like in my initial post that the application must be terminated. My > config when I ran the job did not have log aggregation enabled and that > might be why when I try to look at the logs via "yarn logs -applicationId > <YOUR_APP_ID>", I get back nothing (e.g. Unable to get ApplicationState. > Attempting to fetch logs directly from the filesystem. > > /tmp/logs/si022833/logs/application_1545041832015_73428 does not exist.) > > >
