Hello. I have a Spark streaming job running on a cluster managed by Yarn. The spark streaming job starts and receives data from Kafka. It is processing well and then after several seconds I see the following error:
15/09/22 14:53:49 ERROR yarn.ApplicationMaster: SparkContext did not initialize after waiting for 100000 ms. Please check earlier log output for errors. Failing the application. 15/09/22 14:53:49 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 13, (reason: Timed out waiting for SparkContext.) The spark process is then (obviously) shut down by Yarn. What do I need to change to allow Yarn to initialize Spark streaming (vs. batch) jobs? Thank you, Bryan Jeffrey
