Hi Mark,
I said I've only managed to develop a limited understanding of how
Spark works in the different deploy modes ;-)
But somehow I thought that cluster in spark standalone is not
supported. I think I've seen a JIRA with a change quite recently where
it was said or something similar. Can't fi
Standalone mode also supports running the driver on a cluster node. See
"cluster" mode in
http://spark.apache.org/docs/latest/spark-standalone.html#launching-spark-applications
. Also,
http://spark.apache.org/docs/latest/spark-standalone.html#high-availability
On Mon, Nov 30, 2015 at 9:47 AM, Ja
Hi,
My understanding of Spark on YARN and even Spark in general is very
limited so keep that in mind.
I'm not sure why you compare yarn-cluster and spark standalone? In
yarn-cluster a driver runs on a node in the YARN cluster while spark
standalone keeps the driver on the machine you launched a S
If your cluster is a dedicated spark cluster (only running spark job, no
other jobs like hive/pig/mr), then spark standalone would be fine.
Otherwise I think yarn would be a better option.
On Fri, Nov 27, 2015 at 3:36 PM, cs user wrote:
> Hi All,
>
> Apologies if this question has been asked bef
Hi All,
Apologies if this question has been asked before. I'd like to know if there
are any downsides to running spark over yarn with the --master yarn-cluster
option vs having a separate spark standalone cluster to execute jobs?
We're looking at installing a hdfs/hadoop cluster with Ambari and
s
I have been analyzing Storm performance and there's no significant overhead
added to the processing nodes. I'm interested in those results over Spark
as well.
Thanks in advance,
Otávio Carvalho.
Undergrad. CompSci Student at UFRGS
Porto Alegre, Brazil.
2014-05-20 18:46 GMT-03:00 anishs...@yah
Hi All
I need to analyse performance of Spark YARN vs Spark Standalone
Please suggest if we have some pre-published comparison statistics available.
TIA
--
Anish Sneh
http://in.linkedin.com/in/anishsneh
