Hello Guru,
Many thanks for your reply.
I am new to this who thing. So pardon me for my naiivety at times.
I am not sure if I am using Spark standalone or Spark on Yarn because when
I check the port number of Spark it shows it as 18088 and like you have
mentioned maybe it is then Spark on Yarn.
Al
Hi Ashish,
If you are not using Spark on YARN and instead using Spark Standalone, you
don’t need Spark history server. More on the Web Interfaces is provided in the
following link. Since are using standalone mode, you should be able to access
the web UI for the master and workers at ports that
Hello Guru,
Thank you for your quick response.
This is what i get when I try executing spark-shell :port number
C:\spark-1.4.0\bin>spark-shell :18088
log4j:WARN No appenders could be found for logger
(org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j sy
Hi Ashish,
Are you running Spark-on-YARN on the cluster with an instance of Spark History
server?
Also if you are using Cloudera Manager and using Spark on YARN, spark on yarn
service has a link for the history server web UI.
Can you paste the command and the output you are seeing in the thr
Thank you Ayan for your response.. But I have just realised that the Spark
is configured to be a history server.
Please, can somebody suggest to me how can I convert Spark history server
to be a Master server?
Thank you
Sincerely,
Ashish Dutt
On Wed, Jul 8, 2015 at 12:28 PM, ayan guha wrote:
On UI?
Master: http://:8080
Worker: http://:8081
On Wed, Jul 8, 2015 at 1:42 PM, Ashish Dutt wrote:
> Hi,
> I have CDH 5.4 installed on a linux server. It has 1 cluster in which
> spark is deployed as a history server.
> I am trying to connect my laptop to the spark history server.
> When I run
Hi,
I have CDH 5.4 installed on a linux server. It has 1 cluster in which spark
is deployed as a history server.
I am trying to connect my laptop to the spark history server.
When I run spark-shell master ip: port number I get the following output
How can I verify that the worker is connected to th
