Hi Mark,
I said I've only managed to develop a limited understanding of how
Spark works in the different deploy modes ;-)
But somehow I thought that cluster in spark standalone is not
supported. I think I've seen a JIRA with a change quite recently where
it was said or something similar. Can't fi
Standalone mode also supports running the driver on a cluster node. See
"cluster" mode in
http://spark.apache.org/docs/latest/spark-standalone.html#launching-spark-applications
. Also,
http://spark.apache.org/docs/latest/spark-standalone.html#high-availability
On Mon, Nov 30, 2015 at 9:47 AM, Ja
Hi,
My understanding of Spark on YARN and even Spark in general is very
limited so keep that in mind.
I'm not sure why you compare yarn-cluster and spark standalone? In
yarn-cluster a driver runs on a node in the YARN cluster while spark
standalone keeps the driver on the machine you launched a S
If your cluster is a dedicated spark cluster (only running spark job, no
other jobs like hive/pig/mr), then spark standalone would be fine.
Otherwise I think yarn would be a better option.
On Fri, Nov 27, 2015 at 3:36 PM, cs user wrote:
> Hi All,
>
> Apologies if this question has been asked bef
