ahshahid commented on code in PR #50033:
URL: https://github.com/apache/spark/pull/50033#discussion_r1982444723
##########
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala:
##########
@@ -3185,16 +3198,101 @@ class DAGSchedulerSuite extends SparkFunSuite with
TempLocalSparkContext with Ti
"Spark can only do this while using the new shuffle block fetching
protocol"))
}
+ test("SPARK-51272: retry all the succeeding stages when the map stage is
indeterminate with" +
+ " concurrent tasks completion") {
+ if (scheduler != null) {
+ this.afterEach()
+ }
Review Comment:
I think it is needed.. Otherwise the test will fail..
The reason for this is
The Test SPARK-512272, needs a fresh initialization of DagScheduler, because
the dagEventProcessLoopTester needs to be recreated, as the test is assigning
a not null ( new value) for the field dagSchedulerInterceptor.
If I dont call afterEach, the context will not stop and hence
dagEventProcessLoopTester in the DagScheduler will be using old ( null ) as
interceptor.
If you notice, after calling afterEach, I am assigning a value to
dagSchedulerInterceptor. Then calling beforeEach, will ensure that new
DagScheduler created will use the interceptor needed by the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
