I think if you have a shared storage for notebooks (for example, NFS mounted from a third server), and a load-balancer that supports sticky sessions (like F5) on top, it should be possible to have HA without any code change in Zeppelin. Am I missing something?
-- Ruslan Dautkhanov On Fri, Jun 30, 2017 at 5:54 PM, Alexander Filipchik <afilipc...@gmail.com> wrote: > Honestly, HA requires more than just active stand by. > It should be able to scale without major surgeries, which is not possible > right now. For example, if you start too many interpreters, zeppelin box > will simply run out of memory. > > Alex > > On Thu, Jun 29, 2017 at 10:59 PM, wenxing zheng <wenxing.zh...@gmail.com> > wrote: > >> at first, I would think GIT storage is a good option and we can push and >> pull the changes regularly. >> >> With multiple zeppelin instances, maybe we need a new component or >> service to act as a distributed scheduler: dispatch the Job to and manage >> the Jobs on the Zeppelin instances. >> >> On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla <vinayshu...@gmail.com> >> wrote: >> >>> Here is what I think should be part of HA consideration: >>> >>> 1. Have multiple Zeppelin Instances >>> 2. Have the notebooks storage backed by something like an NFS so all >>> notebooks are visible across all Zeppelin instances >>> 3. Put multiple load balancers infront of Zeppelin to route requests. >>> >>> Consider that HA needs scalability, which depends on which interpreter >>> you plan to use. So you might need to consider HA at both Zeppelin and >>> interpreter level. For example if you were using Z + Livy + Spark, you will >>> need to consider scalability + HA needs of Z + Livy interpreter + Livy >>> Server + Spark (on Cluster manager). >>> >>> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng <wenxing.zh...@gmail.com >>> > wrote: >>> >>>> and do we have any architecture doc for reference? Because we need to >>>> add the HA capability as soon as possible, hope we can figure it out. >>>> >>>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng < >>>> wenxing.zh...@gmail.com> wrote: >>>> >>>>> Thanks to Jeff and Moon. >>>>> >>>>> So currently the active-active model doesn't work on GIT storage, am I >>>>> right? >>>>> >>>>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> >>>>> wrote: >>>>> >>>>>> Basically active-(hot)standby model would work. >>>>>> Two or more Zeppelin instance can be started and pointing the same >>>>>> notebook storage, if only one Zeppelin instance (active) change notebook >>>>>> at >>>>>> any given time. >>>>>> >>>>>> In case of the active instance fails, one of rest instance can take >>>>>> over the role by refreshing notebook list and start make change. >>>>>> >>>>>> But all these fail over is not provided by Zeppelin and need to >>>>>> depends on external script or HA software (like Heartbeat). >>>>>> >>>>>> Like Jeff mentioned, community does not have concrete plan for having >>>>>> HA built-in at this moment. >>>>>> >>>>>> Hope this helps, >>>>>> >>>>>> Thanks, >>>>>> moon >>>>>> >>>>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >>>>>> >>>>>>> >>>>>>> No concrete plan for that. There're other higher priority things >>>>>>> need to be done. At least it would not be available in 0.8, maybe after >>>>>>> 1.0 >>>>>>> >>>>>>> >>>>>>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>>>>>> >>>>>>>> Thanks to Jianfeng. >>>>>>>> >>>>>>>> Do you know any plan on this? >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>>>>>> jzh...@hortonworks.com> wrote: >>>>>>>> >>>>>>>>> HA is not supported, there’s still lots of configuration files >>>>>>>>> stored in local file system. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Best Regard, >>>>>>>>> Jeff Zhang >>>>>>>>> >>>>>>>>> >>>>>>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>>>>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>>>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>>> Subject: Query about the high availability of Zeppelin >>>>>>>>> >>>>>>>>> Hello all, >>>>>>>>> >>>>>>>>> I still didn't find any docs on this topic? Appreciated if anyone >>>>>>>>> can shed some lights on how to get the Zeppelin into a cluster with >>>>>>>>> shared/centralized storage >>>>>>>>> >>>>>>>>> Regards, Wenxing >>>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>>> >>> >> >
