Re: Zeppelin - Spark Driver location

spark-submit would only run when you run the first paragraph using spark interpreter. After that, paragraph would send code to the spark app to execute. >>> Also spark standalone cluster moder should work even before this new release, right? I didn't verify that, not sure whether other people very

Re: Zeppelin - Spark Driver location

Also spark standalone cluster moder should work even before this new release, right? On Wed, Mar 14, 2018 at 8:43 AM, ankit jain wrote: > Hi Jhang, > Not clear on that - I thought spark-submit was done when we run a > paragraph, how does the .sh file come into play? > > Thanks > Ankit > > On Tue

Re: Zeppelin - Spark Driver location

Hi Jhang, Not clear on that - I thought spark-submit was done when we run a paragraph, how does the .sh file come into play? Thanks Ankit On Tue, Mar 13, 2018 at 5:43 PM, Jeff Zhang wrote: > > spark-submit is called in bin/interpreter.sh, I didn't try standalone > cluster mode. It is expected

Re: Zeppelin - Spark Driver location

Hello Jhon, Conceptually this makes sense, since Zeppelin creates a spark application for the execution runtime underneath its frontend process. Having said this, depending on how Zeppelin is implemented, it might be required for the driver to be collocated with the zeppelin process on the same

Re: Zeppelin - Spark Driver location

> Zeppelin version: 0.8.0 (merged at September 2017 version) https://issues.apache.org/jira/browse/ZEPPELIN-2898 was merged end of September so not sure if you have that. Check out https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235 how to set this up. -- Ruslan Dautkhanov O

Re: Zeppelin - Spark Driver location

Hi Jhang, What is the expected behavior with standalone cluster mode? Should we see separate driver processes in the cluster(one per user) or multiple SparkSubmit processes? I was trying to dig in Zeppelin code & didn’t see where Zeppelin does the Spark-submit to the cluster? Can you please poi

Re: Zeppelin - Spark Driver location

spark-submit is called in bin/interpreter.sh, I didn't try standalone cluster mode. It is expected to run driver in separate host, but didn't guaranteed zeppelin support this. Ankit Jain 于2018年3月14日周三 上午8:34写道： > Hi Jhang, > What is the expected behavior with standalone cluster mode? Should we s

Re: Zeppelin - Spark Driver location

ZEPPELIN-2898 is for yarn cluster model. And Zeppelin have integration test for yarn mode, so guaranteed it would work. But don't' have test for standalone, so not sure the behavior of standalone mode. Ruslan Dautkhanov 于2018年3月14日周三 上午8:06写道

Re: Zeppelin - Spark Driver location

Does this new feature work only for yarn-cluster ?. Or for spark standalone too ? El mar., 13 de mar. de 2018 18:34, Ruslan Dautkhanov escribió: > > Zeppelin version: 0.8.0 (merged at September 2017 version) > > https://issues.apache.org/jira/browse/ZEPPELIN-2898 was merged end of > September so