replying to my own thread; I realized that this only happens when the replication level is 1.
Regardless of whether setting memory_only or disk or deserialized, I had to make the replication level >= 2 to make the streaming work properly on YARN. I still don't get it why, because intuitively less replication should imply faster computation, and testing on a cloudera VM everything worked fine on YARN. If I am missing something important, please let me know. I am going to settle down to '..._2' variants for now. Jong Wook -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Custom-streaming-receiver-slow-on-YARN-tp21544p21553.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
