KafkaReceiver executor in spark streaming job on YARN suddenly killed by ResourceManager

2015-01-02 Thread Junki Kim
* YARN : 2.5.0-cdh5.2.1 * CDH : 5.2.1 version Thanks in advance. Have a nice day~ -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/KafkaReceiver-executor-in-spark-streaming-job-on-YARN-suddenly-killed-by-ResourceManager-tp20945.html Sent from the Apache Spark

KafkaReceiver executor in spark streaming job on YARN suddenly killed by ResourceManager

2015-01-02 Thread Jun Ki Kim
Hi, guys I tried to run job of spark streaming with kafka on YARN. My business logic is very simple. Just listen on kafka topic and write dstream to hdfs on each batch iteration. After launching streaming job few hours, it works well. However suddenly died by ResourceManager. ResourceManager sen