Hi Andreas, Yes, seems I can't avoid Zookeeper right now. It would be really nice if we can achieve HA via gossip protocol like Cassandra/Spark DSE does ?
Is this possible ? Cheers On Wed, Nov 25, 2015 at 4:12 PM, Andreas Fritzler < andreas.fritz...@gmail.com> wrote: > Hi Welly, > > you will need Zookeeper if you want to setup the standalone cluster in HA > mode. > http://spark.apache.org/docs/latest/spark-standalone.html#high-availability > > In the YARN case you probably have already Zookeeper in place if you are > running YARN in HA mode. > > Regards, > Andreas > > On Wed, Nov 25, 2015 at 10:02 AM, Welly Tambunan <if05...@gmail.com> > wrote: > >> Hi Ufuk >> >> >In failure cases I find YARN more convenient, because it takes care of >> restarting failed task manager processes/containers for you. >> >> So this mean that we don't need zookeeper ? >> >> >> Cheers >> >> On Wed, Nov 25, 2015 at 3:46 PM, Ufuk Celebi <u...@apache.org> wrote: >> >>> > On 25 Nov 2015, at 02:35, Welly Tambunan <if05...@gmail.com> wrote: >>> > >>> > Hi All, >>> > >>> > I would like to know if there any feature differences between using >>> Standalone Cluster vs YARN ? >>> > >>> > Until now we are using Standalone cluster for our jobs. >>> > Is there any added value for using YARN ? >>> > >>> > We don't have any hadoop infrastructure in place right now but we can >>> provide that if there's some value to that. >>> >>> There are no features, which only work on YARN or in standalone >>> clusters. YARN mode is essentially starting a standalone cluster in YARN >>> containers. >>> >>> In failure cases I find YARN more convenient, because it takes care of >>> restarting failed task manager processes/containers for you. >>> >>> – Ufuk >>> >>> >> >> >> -- >> Welly Tambunan >> Triplelands >> >> http://weltam.wordpress.com >> http://www.triplelands.com <http://www.triplelands.com/blog/> >> > > -- Welly Tambunan Triplelands http://weltam.wordpress.com http://www.triplelands.com <http://www.triplelands.com/blog/>
