I read some of the notes here. I have been away from Zeppelin for a while
but have extensive experience with Spark on Kubernetes (k8s).
First of all I assume that the zeppelin server is just the client that you
are running to submit a job to Spark on k8s.
If the Spark on k8s is offered as a servi
Thanks for the sharing, it would be nice if you can write a blog to share
it with more wide zeppelin users.
Fabrizio Fab 于2021年10月28日周四 下午4:29写道：
>
> Yeah ! Thank you very much Philipp: tonight I explored carefully the
> source code and discovered the 2 thrift servers stuff.
>
> Therefore I sol
Yeah ! Thank you very much Philipp: tonight I explored carefully the source
code and discovered the 2 thrift servers stuff.
Therefore I solved my problem: here the solution adopted, which can be useful
for other people.
CONTEXT
I have my Zeppelin Server installation located into a LAN, where
Hi Fabrizio,
We have two connections. First, the Zeppelin interpreter opens a
connection to the Zeppelin server to register and to send back the
interpreter output. The Zeppelin server is the CALLBACK_HOST and the
PORT indicates where the Zeppelin server opened the Thrift service for
the Zepp
Hi Philipp, okay, I realized just now of my HUGE misunderstanding !
The "double-spark-submit" patter is just the standard spark-on-k8s way of
running spark applications in cluster mode:
the 1st "spark-submit" in "cluster mode" is started from the client (in the
zeppelin host, in our case), then
Thank you Philipp, for your answer.
interpreter.sh is the shell script which is run by the Zeppelin Server and, in
particular, the following Line that you highlighted starts the interpreter in
CLUSTER MODE in my case:
INTERPRETER_RUN_COMMAND+=("${SPARK_SUBMIT}" "--class" "${ZEPPELIN_SERVER}"
"
Hi Fabrizio,
At the moment I think zeppelin does not support running spark jobs in
cluster mode. But in fact K8s mode simulates cluster mode. Because the
Zeppelin interpreter is already started as a pod in K8s, as a manual
Spark submit execution would do in cluster mode.
Spark-submit is call
