hadoop.security.auth_to_local

2018-07-06 Thread Ruslan Dautkhanov
I assume some users are connecting to Spark in Zeppelin through Livy. It seems Livy doesn't support `hadoop.security.auth_to_local` - filed https://issues.apache.org/jira/browse/LIVY-481 Has anyone ran into this issue? It seems Livy's `livy.server.auth.kerberos.name-rules` config was trying to w

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Ruslan Dautkhanov
That interpreting binding UI was also used to restart interpreters that our users were using form time to time. If we remove interpreter binding, there should still be a way to restart them quickly without going deep into menus. And also there should be a way to show interpreter status, like for e

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Ruslan Dautkhanov
+1 to remove it Setting default interpreter is not very useful anyway (for example, we can't make %pyspark default without manually editing xml files in Zeppelin distro). https://issues.apache.org/jira/browse/ZEPPELIN-3282 -- Ruslan Dautkhanov On Fri, Jul 6, 2018 at 7:27 AM Paul Brenner wrote

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Paul Brenner
I agree with Partridge. We have different interpreters defined with different queues and settings. So we need a way to quickly change the default interpreter and can’t rely on typing the desired interpreter at the start of each paragraph.  ( https://share.polymail.io/v1/z/b/NWIzZjZkNTc0YWE0/kV

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Jongyoul Lee
It looks like a good idea to show the default interpreter group - and it should be also changeable - and store it to `note.json`. It could remove interpreter-binding information, keep default interpreter when extracting note. Moreover it also helps to reduce our server code as well. What do you gu

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Sanjay Dasgupta
If there is no per-interpreter overhead of binding all the interpreters from the beginning, we should definitely do it. This will simplify the GUI somewhat. Regards, - Sanjay On Fri, Jul 6, 2018 at 1:49 PM, Partridge, Lucas (GE Aviation) < lucas.partri...@ge.com> wrote: > “So usually we would r

Re: org.apache.hadoop.fs.FileSystem$Statistics.getThreadStatistics()

2018-07-06 Thread Andrea Santurbano
Thanks Adamantios, I created a Dockerfile in order to aoutomate the process, feel free to use it: https://gist.github.com/conker84/4ffc9a2f0125c808b4dfcf3b7d70b043 Il giorno gio 5 lug 2018 alle ore 13:00 Adamantios Corais < adamantios.cor...@gmail.com> ha scritto: > Hi Andrea, > > The followin

[DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Partridge, Lucas (GE Aviation)
“So usually we would recommend user to specify the full qualified interpreter name.” - I usually recommend the exact opposite to our users. We frequently change interpreter groups to allow for different Spark cluster settings (number of executors, memory, etc). Users with more demanding requirem

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Jeff Zhang
We already allow setting default interpreter when creating note. Another way to set default interpreter is to reorder the interpreter setting binding in note page. But personally I don't recommend user to use short interpreter name because of default interpreter. 2 Reaons: 1. It introduce in-accur

Re: [DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Jongyoul Lee
There are two purposes of interpreter binding. One is what you mentioned and another one is to manage a default interpreter. If we provide a new way to set default interpreter, I think we can remove them :-) We could set permissions in other ways. Overall, +1 On Fri, Jul 6, 2018 at 4:24 PM, Jeff

[DISCUSS] Is interpreter binding necessary ?

2018-07-06 Thread Jeff Zhang
Hi Folks, I raise this thread to discuss whether we need the interpreter binding. Currently when user create notes, they have to bind interpreters to their notes in note page. Otherwise they will hit interpreter not found issue. Besides that in zeppelin server side, we maintain the interpreter bin