What are ideas around Spark cluster for streaming purposes ?
What is better standalone / Mesos / YARN ?
Please share cluster details and size of data and type of processing.
(multiple processing points) (architecture or similar)
I see folks using YARN cluster for streaming purposes.
Regards,
Dee
I am looking to decide what is best for my production grade spark
application(s).
YARN
=
1. YARN supports security. When Spark is run over YARN the communication
between processes can use secure authentication through Kerberos.
2. Spark standalone cluster can only run Spark jobs and
I'm not sure what you're looking for, since you can't really compare
Standalone with YARN or Mesos, as Standalone is assuming the Spark
workers/master owns the cluster, and YARN/Mesos is trying to share the
cluster among different applications/frameworks.
And when you refer to resource utilization
Do we have any comparisons in terms of resource utilization, scheduling of
running Spark in the below three modes
1) Standalone
2) over YARN
3) over Mesos
Can some one share resources (thoughts/URLs) on this area.
--
Deepak
est hadoop versions too. It depends on
> the fact that if you are already using yarn and don't want the hassle of
> setting up another cluster manager you can probably prefer yarn.
>
>
>> Thanks,
>> Ishaaq
>>
>>
>>
>> --
>> View this message in con
don't want the hassle of
setting up another cluster manager you can probably prefer yarn.
> Thanks,
> Ishaaq
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/standalone-vs-YARN-tp4271.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
s: YARN or standalone? Which is the more
stable option currently? Which is the more future-proof option?
Thanks,
Ishaaq
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/standalone-vs-YARN-tp4271.html
Sent from the Apache Spark User List mailing list
