Hey Gerard,
Spark Streaming should just queue the processing and not delete the block
data. There are reports of this error and I am still unable to reproduce
the problem. One workaround you can try the configuration
"spark.streaming.unpersist = false" . This stops Spark Streaming from
cleaning up
Hello Sparkers,
I'm currently running load tests on a Spark Streaming job. When the task
duration increases beyond the batchDuration the job become unstable. In the
logs I see tasks failed with the following message:
Job aborted due to stage failure: Task 266.0:1 failed 4 times, most recent
failu
