On a side note… Has anyone got the Livy interpreter to be added as an interpreter in the latest build of Zeppelin 0.6.0? By the way, I have Shiro authentication on. Could this interfere?
Thanks, Ben > On Jun 29, 2016, at 11:18 AM, moon soo Lee <m...@apache.org> wrote: > > Livy interpreter internally creates multiple sessions for each user, > independently from 3 binding modes supported in Zeppelin. > Therefore, 'shared' mode, Livy interpreter will create sessions per each > user, 'scoped' or 'isolated' mode will result create sessions per notebook, > per user. > > Notebook is shared among users, they always use the same interpreter > instance/process, for now. I think supporting per user interpreter > instance/process would be future work. > > Thanks, > moon > > On Wed, Jun 29, 2016 at 7:57 AM Chen Song <chen.song...@gmail.com > <mailto:chen.song...@gmail.com>> wrote: > Thanks for your explanation, Moon. > > Following up on this, I can see the difference in terms of single or multiple > interpreter processes. > > With respect to spark drivers, since each interpreter spawns a separate Spark > driver in regular Spark interpreter setting, it is clear to me the different > implications of the 3 binding modes. > > However, when it comes to Livy server with impersonation turned on, I am a > bit confused. Will Livy interpreter always create a new Spark driver (along > with a Spark Context instance) for each user session, regardless of the > binding mode of Livy interpreter? I am not very familiar with Livy, but from > what I could tell, I see no difference between different binding modes for > Livy on as far as how Spark drivers are concerned. > > Last question, when a notebook is shared among users, will they always use > the same interpreter instance/process already created? > > Thanks > Chen > > > > On Fri, Jun 24, 2016 at 11:51 AM moon soo Lee <m...@apache.org > <mailto:m...@apache.org>> wrote: > Hi, > > Thanks for asking question. It's not dumb question at all, Zeppelin docs does > not explain very well. > > Spark Interpreter, > > 'shared' mode, a spark interpreter setting spawn a interpreter process to > serve all notebooks which binded to this interpreter setting. > 'scoped' mode, a spark interpreter setting spawn multiple interpreter > processes per notebook which binded to this interpreter setting. > > Using Livy interpreter, > > Zeppelin propagate current user information to Livy interpreter. And Livy > interpreter creates different session per user via Livy Server. > > > Hope this helps. > > Thanks, > moon > > > On Tue, Jun 21, 2016 at 6:41 PM Chen Song <chen.song...@gmail.com > <mailto:chen.song...@gmail.com>> wrote: > Zeppelin provides 3 binding modes for each interpreter. With `scoped` or > `shared` Spark interpreter, every user share the same SparkContext. Sorry for > the dumb question, how does it differ from Spark via Ivy Server? > > > -- > Chen Song >
