ahshahid commented on PR #50757: URL: https://github.com/apache/spark/pull/50757#issuecomment-2847985465
My knowledge is limited , so pls bear with me, I thought there would be a result stage after every map stage implying that there can be no two consequent Map stages. And a ResultStage is always committed ( i.e cannot be lost or rolled back once completed successfully, IIUC). So should not a ResultStage sort of break the chain of inDeterminancy , even with the current code where RDDS get their determinancy flag from parent RDDs across stages...? An RDD emanating out of a ResultStage should not ever be inDeterminate..if I understand correctly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
