Spark will typically give you that resource (CPU, memory...) sharing and reuse. However, the way you configure your Spark REPL (number of cores, memory size...) is not my way, just because your data and needs are not mine.
There is still a place to design and develop and additional layer of abstraction which would allow Zeppelin interpreters to run on shared and distributed resources (think to a Hadoop Slider application). In that case, a single interpreter configuration could run on any node of a distributed cluster (Slider would distributed the job for you). Still, I believe that it would be beneficial to Zeppelin to allow, not oblige, each user to configure their own separate interpreter settings. On Mon, Sep 26, 2016 at 2:52 PM, darren <dar...@ontrenet.com> wrote: > Just throwing my $0.02 at the prior comment by York. In my opinion users > should not "have their own processing". In cloud environments everything is > shared because resources are only utilized on demand. Thus users request > processing and occupy it while using it then it is released for others. If > there are enough resources to go around then it looks like everyone has > their own. > > > > Sent from my Verizon, Samsung Galaxy smartphone > > -------- Original message -------- > From: Eric Charles <e...@apache.org> > Date: 9/26/16 8:21 AM (GMT-05:00) > To: users@zeppelin.apache.org > Subject: Re: zeppelin architecture with multiple users > > > > On Fri, Sep 23, 2016 at 1:37 AM, York Huang <yorkhuang.d...@gmail.com> > wrote: > >> I think Vincent's option 1 is the way to go at this stage. Basically, in >> a multi-users environment, every users should have their own storage and >> processing. So far Zeppelin has not seen to be able to isolate users >> totally. >> > > > https://github.com/apache/zeppelin/pull/1390 is an attempt to fully > isolate notes and services (including interpreters) per user. > > >> >> My question is, >> 1. I am using windows for client which means Zeppelin is installed on >> windows. Is their any downside or limitation? >> 2. Even installed on individual desktop, I still like to have >> authentication for login as users may access their desktop remotely from >> browser. Any idea how to set this up? >> 3. I am using MapR cluster. Does anyone have similar experiences on how >> to config secure cluster with zeppelin in windows? Also is it possible to >> set up an admin user for every desktop so that only I can change the >> configuration? >> >> Thanks, >> >> >> On 22 September 2016 at 18:16, vincent gromakowski < >> vincent.gromakow...@gmail.com> wrote: >> >>> It seems credentials are saved per user as described in Shiro, can you >>> confirm ? I don't find anything in credential API that links the datasource >>> user to zeppelin user... Or username on datasource should be the same as >>> Zeppelin username ? >>> >>> Can an admin user set credentials for other users ? >>> >>> 2016-09-22 10:12 GMT+02:00 vincent gromakowski < >>> vincent.gromakow...@gmail.com>: >>> >>>> Hi, >>>> From my point of view you have 3 options: >>>> >>>> 1. Use a dedicated zeppelin instance per user. Solution I currently >>>> use. Mesos/marathon launch an instance by user with it's linux UID. A >>>> service discovery is routing each user based on HTTPS basic auth to his >>>> instance. Because the configuration file is dedicated per user, it's easy >>>> to setup credentials for backend. Because the UID is also setup, spark jobs >>>> are running under each user permissions. This way is totally secure, but no >>>> possible sharing between users except sending notebooks by mail or git repo >>>> >>>> 2. Use a shared instance and configure Shiro permissions which allow to >>>> manage multi tenancy in Zeppelin (notebooks access) but not in backend as >>>> all users notebooks will run under the same UID and have the same >>>> credentials for accessing backend >>>> >>>> 3. Use a shared instance and a backend that allows impersonation like >>>> Livy server. Livy server will execute Spark sessions per user. What is >>>> unclear is how to deal with backend credentials ? How to configure multiple >>>> Cassandra credentials and attach each one to a user ? Same thing for Spark >>>> Livy, How can we configure each Livy session with users cassandra >>>> credentials ? And finally how credentials are secured in Zeppelin ? >>>> >>>> 2016-09-22 8:59 GMT+02:00 York Huang <yorkhuang.d...@gmail.com>: >>>> >>>>> Hi DuyHai, >>>>> >>>>> I would like to know how to set up security (authentication and >>>>> authorization), the architecture, etc. >>>>> >>>>> The users are using windows. I am ok to set up individual zeppelin on >>>>> their desktop or a central zeppelin server. But I want to know the >>>>> complexity, limitation, details, etc. >>>>> >>>>> Many thanks! >>>>> >>>>> On 16 September 2016 at 03:51, DuyHai Doan <doanduy...@gmail.com> >>>>> wrote: >>>>> >>>>>> Right now, you have some options to isolate the notes. Look at the >>>>>> doc about interpreter binding mode here : >>>>>> http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/inte >>>>>> rpreters.html#interpreter-binding-mode >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Sep 15, 2016 at 7:15 AM, York Huang <yorkhuang.d...@gmail.com >>>>>> > wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I want to set up a environment for a group of users so that they can >>>>>>> access zeppelin. Each of them should have their own space, should not >>>>>>> interfere each other. >>>>>>> >>>>>>> I install zeppelin on the MapR sandbox. If I access it from >>>>>>> different computers, even I access different notebooks, the data are >>>>>>> still >>>>>>> shared. >>>>>>> >>>>>>> What I want is the data should be totally seperate between users and >>>>>>> notebooks. >>>>>>> >>>>>>> How do I set it up like this? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> York Huang >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
