I'm managing Spark Streaming applications which run on Cloud Dataproc (https://cloud.google.com/dataproc/). Spark Streaming applications running on a Cloud Dataproc cluster seem to run in client mode on YARN.
Some of my applications sometimes stop due to the application failure. I'd like YARN to restart the stopped Spark Streaming applications automatically but I don't how to achieve it. I tried to set a value to "yarn.resourcemanager.am.max-attempts" and restarted YARN, but it didn't restart the failed application automatically. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-restart-a-failed-Spark-Streaming-Application-automatically-in-client-mode-on-YARN-tp25172.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
