Re: Passing variables from %pyspark to %sh

2017-01-13 Thread Ruslan Dautkhanov
Created https://issues.apache.org/jira/browse/ZEPPELIN-1967 (JIRA had some issues.. https://twitter.com/infrabot - had to wait a couple of days.) Great ideas. Thank you everyone. -- Ruslan Dautkhanov On Thu, Jan 12, 2017 at 8:55 AM, t p wrote: > Is something like feasible from the front

Re: Passing variables from %pyspark to %sh

2017-01-12 Thread t p
Is something like feasible from the front end perspective - i.e the web UI (Angular?) - i.e. not matter which process/JVM runs the interpreter, I’d assume that a book is executed in the context of a we browser which unifies all the pages of the book... > On Jan 12, 2017, at 9:56 AM, Jeff Zhang

Re: Passing variables from %pyspark to %sh

2017-01-12 Thread Jeff Zhang
Agree to share variables between interpreters. Currently zeppelin launch one JVM for each interpreter group. So it is not possible to share variables between spark and sh. But for some interpreters like sh, md, it is not necessary to create separate JVM for them. We can embed them in spark interpre

Re: Passing variables from %pyspark to %sh

2017-01-12 Thread Jongyoul Lee
Yes, many users suggest that feature to share results between paragraphs and different interpreters. I think this would be one of major features in a next release. On Thu, Jan 12, 2017 at 10:30 PM, t p wrote: > Is it possible to have similar support to exchange checkbox/dropdown > variables and

Re: Passing variables from %pyspark to %sh

2017-01-12 Thread t p
Is it possible to have similar support to exchange checkbox/dropdown variables and can variables be exchanged with other interpreters like PSQL (e.g. variable set by spark/pyspark and accessible in another para which is running PSQL interpreter). I’m interested in doing this and I’d like to kn

Re: Passing variables from %pyspark to %sh

2017-01-11 Thread Jongyoul Lee
There's no way to communicate between spark and sh intepreter. It need to implement it but it doesn't yet. But I agree that it would be helpful for some cases. Can you create issue? On Thu, Jan 12, 2017 at 3:32 PM, Ruslan Dautkhanov wrote: > It's possible to exchange variables between Scala and

Passing variables from %pyspark to %sh

2017-01-11 Thread Ruslan Dautkhanov
It's possible to exchange variables between Scala and Spark through z.put and z.get. How to pass a variable to %sh? In Jupyter it's possible to do for example as > ! hadoop fs -put {localfile} {hdfsfile} where localfile and and hdfsfile are Python variables. Can't find any references for so