Re: Multiple concurrent spark notebooks

2016-10-06 Thread Mark Libucha
That was it! Thanks so much Andreas. Can't believe I had overlooked that drop down in the interpreter settings. Mohit and Mich probably assumed I had tried that already. Thanks everyone. Mark On Thu, Oct 6, 2016 at 8:35 AM, Andreas Lang wrote: > Hi Mark, > > you may want to check the spark int

Re: Multiple concurrent spark notebooks

2016-10-06 Thread Andreas Lang
Hi Mark, you may want to check the spark interpreter settings. In the most recent version of zeppelin you can set it to shared, isolated or scoped. Shared: single interpreter and spark context (and the queuing you see) Isolated: every notebook has its own interpreter and spark context Scoped: eve

Re: Multiple concurrent spark notebooks

2016-10-06 Thread Mark Libucha
Mich, thanks for the suggestion. I tried your settings, but they did not solve the problem. I'm running in yarn-client mode, not local or standalone, so the resources in the Spark cluster (which is very large) should not be an issue. Right? The problem seems to be that Zeppelin is not submitting

Re: Multiple concurrent spark notebooks

2016-10-05 Thread Mich Talebzadeh
Hi Mark, Zeppelin on Park uses Spark interpreter Edit the interpreter. By default Zeppelin uses local mode as seen below [image: Inline images 1] You can of course change that to standalone mode by specifying master spark://:7077 and increase cores.max and spark.executor.memory as shown above

Re: Multiple concurrent spark notebooks

2016-10-05 Thread Mohit Jaggi
change your spark settings so that the REPL does not get the whole cluster. e.g. by reducing the executor memory and cpu allocation. Mohit Jaggi Founder, Data Orchard LLC www.dataorchardllc.com > On Oct 5, 2016, at 11:02 AM, Mark Libucha wrote: > > Hi everyone, > > I've got Zeppelin runnin

Multiple concurrent spark notebooks

2016-10-05 Thread Mark Libucha
Hi everyone, I've got Zeppelin running against a Cloudera/Yarn/Spark cluster and everything seems to be working fine. Very cool. One minor issue, though. When one notebook is running, others queue up behind it. Is there a way to run multiple notebooks concurrently? Both notebooks are running the