Re: Monitoring a Notebook in Spark UI

2020-07-21 Thread Jeff Zhang
ame time, but each > user can run only one job at one time. How is it possible to run multiple > sql jobs concurrently in one spark app? > > > > Thanks, > > > > Stéphane > > > > > > *From:* Jeff Zhang [mailto:zjf...@gmail.com] > *Sent:* Tuesday,

RE: Monitoring a Notebook in Spark UI

2020-07-21 Thread stephane.davy
time, but each user can run only one job at one time. How is it possible to run multiple sql jobs concurrently in one spark app? Thanks, Stéphane From: Jeff Zhang [mailto:zjf...@gmail.com] Sent: Tuesday, July 21, 2020 17:54 To: users Subject: Re: Monitoring a Notebook in Spark UI

Re: Monitoring a Notebook in Spark UI

2020-07-21 Thread Jeff Zhang
Regarding how many spark apps, it depends on the interpreter binding mode, you can refer to this document. http://zeppelin.apache.org/docs/0.9.0-preview1/usage/interpreter/interpreter_binding_mode.html Internally, each spark app run a scala shell to execute scala code and python shell to execute py

Monitoring a Notebook in Spark UI

2020-07-21 Thread Joshua Conlin
Hello, I'm looking for documentation to better understand pyspark/scala notebook execution in Spark. I typically see application runtimes that can be very long, is there always a spark "application" running for a notebook or zeppelin session? Those that are not actually being run in zeppelin typ