Jhon Cardenas created ZEPPELIN-3542:
---------------------------------------
Summary: Stop a pyspark paragraph cancels all the pyspark jobs of
other users
Key: ZEPPELIN-3542
URL: https://issues.apache.org/jira/browse/ZEPPELIN-3542
Project: Zeppelin
Issue Type: Bug
Components: pySpark
Affects Versions: 0.8.0
Environment: This happen when the spark context is shared between
users (scoped).
Reporter: Jhon Cardenas
Stop a pyspark paragraph cancels all the pyspark jobs of other users.
Cancel button in pyspark paragraph cancels Spark jobs for all users.
This happen when the spark context is shared between users (scoped).
It seems to be related with [the
solution|https://github.com/apache/zeppelin/commit/9f22db91c279b7daf6a13b2d805a874074b070fd]
for the task
[ZEPPELIN-2075|https://issues.apache.org/jira/browse/ZEPPELIN-2075].
This solution is causing that when one particular user cancels his py-spark
job, the py-spark jobs from all the users are being canceled !!.
When a py-spark job is cancelled, the method PySparkInterpreter interrupt() is
invoked, and then the SIGINT event is called, causing that all the jobs in the
same spark context be cancelled:
context.py:
# create a signal handler which would be invoked on receiving SIGINT
def signal_handler(signal, frame):
self.cancelAllJobs()
raise KeyboardInterrupt()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
