Hello, I was wondering if there's a way to kill stages in spark streaming via a StreamingListener. Sometimes stages will hang, and simply killing it via the UI:
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n24478/Screen_Shot_2015-08-27_at_11.png> is enough to let the streaming job continue. Ideally we wouldn't have hanging issues, but we would rather troubleshoot later via logs instead of terminating the entire streaming job. Is there any way to access something that is equivalent to SparkContext.cancelStage, or is this generally not advised? Thanks! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Listener-to-Kill-Stages-tp24478.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
