Re: Discover SparkUI port for spark streaming job running in cluster mode

2015-12-14 Thread Jonathan Kelly
Oh, nice, I did not know about that property. Thanks! On Mon, Dec 14, 2015 at 4:28 PM, Ted Yu wrote: > w.r.t. getting application Id, please take a look at the following > in SparkContext : > > /** >* A unique identifier for the Spark application. >* Its format depends on the scheduler

Re: Discover SparkUI port for spark streaming job running in cluster mode

2015-12-14 Thread Ted Yu
w.r.t. getting application Id, please take a look at the following in SparkContext : /** * A unique identifier for the Spark application. * Its format depends on the scheduler implementation. * (i.e. * in case of local spark app something like 'local-1433865536131' * in case of

Re: Discover SparkUI port for spark streaming job running in cluster mode

2015-12-14 Thread Jonathan Kelly
Are you running Spark on YARN? If so, you can get to the Spark UI via the YARN ResourceManager. Each running Spark application will have a link on the YARN ResourceManager labeled "ApplicationMaster". If you click that, it will take you to the Spark UI, even if it is running on a slave node in the