Honestly, HA requires more than just active stand by. It should be able to scale without major surgeries, which is not possible right now. For example, if you start too many interpreters, zeppelin box will simply run out of memory.
Alex On Thu, Jun 29, 2017 at 10:59 PM, wenxing zheng <wenxing.zh...@gmail.com> wrote: > at first, I would think GIT storage is a good option and we can push and > pull the changes regularly. > > With multiple zeppelin instances, maybe we need a new component or service > to act as a distributed scheduler: dispatch the Job to and manage the Jobs > on the Zeppelin instances. > > On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla <vinayshu...@gmail.com> > wrote: > >> Here is what I think should be part of HA consideration: >> >> 1. Have multiple Zeppelin Instances >> 2. Have the notebooks storage backed by something like an NFS so all >> notebooks are visible across all Zeppelin instances >> 3. Put multiple load balancers infront of Zeppelin to route requests. >> >> Consider that HA needs scalability, which depends on which interpreter >> you plan to use. So you might need to consider HA at both Zeppelin and >> interpreter level. For example if you were using Z + Livy + Spark, you will >> need to consider scalability + HA needs of Z + Livy interpreter + Livy >> Server + Spark (on Cluster manager). >> >> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng <wenxing.zh...@gmail.com> >> wrote: >> >>> and do we have any architecture doc for reference? Because we need to >>> add the HA capability as soon as possible, hope we can figure it out. >>> >>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng <wenxing.zh...@gmail.com >>> > wrote: >>> >>>> Thanks to Jeff and Moon. >>>> >>>> So currently the active-active model doesn't work on GIT storage, am I >>>> right? >>>> >>>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> wrote: >>>> >>>>> Basically active-(hot)standby model would work. >>>>> Two or more Zeppelin instance can be started and pointing the same >>>>> notebook storage, if only one Zeppelin instance (active) change notebook >>>>> at >>>>> any given time. >>>>> >>>>> In case of the active instance fails, one of rest instance can take >>>>> over the role by refreshing notebook list and start make change. >>>>> >>>>> But all these fail over is not provided by Zeppelin and need to >>>>> depends on external script or HA software (like Heartbeat). >>>>> >>>>> Like Jeff mentioned, community does not have concrete plan for having >>>>> HA built-in at this moment. >>>>> >>>>> Hope this helps, >>>>> >>>>> Thanks, >>>>> moon >>>>> >>>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >>>>> >>>>>> >>>>>> No concrete plan for that. There're other higher priority things need >>>>>> to be done. At least it would not be available in 0.8, maybe after 1.0 >>>>>> >>>>>> >>>>>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>>>>> >>>>>>> Thanks to Jianfeng. >>>>>>> >>>>>>> Do you know any plan on this? >>>>>>> >>>>>>> >>>>>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>>>>> jzh...@hortonworks.com> wrote: >>>>>>> >>>>>>>> HA is not supported, there’s still lots of configuration files >>>>>>>> stored in local file system. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Best Regard, >>>>>>>> Jeff Zhang >>>>>>>> >>>>>>>> >>>>>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>>>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>> Subject: Query about the high availability of Zeppelin >>>>>>>> >>>>>>>> Hello all, >>>>>>>> >>>>>>>> I still didn't find any docs on this topic? Appreciated if anyone >>>>>>>> can shed some lights on how to get the Zeppelin into a cluster with >>>>>>>> shared/centralized storage >>>>>>>> >>>>>>>> Regards, Wenxing >>>>>>>> >>>>>>> >>>>>>> >>>> >>> >> >
