I have figured out why I am getting this error: We have a lot of data in kafka and the DStream from Kafka used MEMROY_ONLY_SER, so once the memory is low, spark started to discard data that is needed later ... So once I change to MEMORY_AND_DISK_SER, the error is gone.
Tian -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Could-not-compute-split-block-not-found-tp11186p16084.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
