As my understanding, OutputCommitCoordinator should only be necessary for ResultStage (especially for ResultStage with hdfs write), but currently it is used for all the stages. Is there any reason for that ?
-- Best Regards Jeff Zhang
- Is OutputCommitCoordinator necessary for all the stages ? Jeff Zhang
