Not really a good idea.
It breaks the paradigm.
If I understand the OP’s idea… they want to halt processing the RDD, but not
the entire job.
So when it hits a certain condition, it will stop that task yet continue on to
the next RDD. (Assuming you have more RDDs or partitions than you have t
You can cause a failure by throwing an exception in the code running on the
executors. The task will be retried (if spark.task.maxFailures > 1), and
then the stage is failed. No further tasks are processed after that, and an
exception is thrown on the driver. You could catch the exception and see i
