Re: restart spark streaming app

2016-08-12 Thread Jacek Laskowski
Hi, I think it's cluster deploy mode. spark-submit --deploy-mode cluster --master yarn myStreamingApp.jar Pozdrawiam, Jacek Laskowski https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Fri,

restart spark streaming app

2016-08-12 Thread Shifeng Xiao
Hi folks, I am using Spark streaming, and I am not clear if there is smart way to restart the app once it fails, currently we just have one cron job to check if the job is running every 2 or 5 minutes and restart the app when necessary. According to spark streaming guide: - *YARN* - Yarn sup