Github user markhamstra commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10590604
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -123,17 +123,17 @@ class DAGScheduler(
private val nextStageId = new AtomicInteger(0)
- private[scheduler] val jobIdToStageIds = new TimeStampedHashMap[Int,
HashSet[Int]]
+ private[scheduler] val jobIdToStageIds = new HashMap[Int, HashSet[Int]]
- private[scheduler] val stageIdToJobIds = new TimeStampedHashMap[Int,
HashSet[Int]]
+ private[scheduler] val stageIdToJobIds = new HashMap[Int, HashSet[Int]]
- private[scheduler] val stageIdToStage = new TimeStampedHashMap[Int,
Stage]
+ private[scheduler] val stageIdToStage = new HashMap[Int, Stage]
- private[scheduler] val shuffleToMapStage = new TimeStampedHashMap[Int,
Stage]
+ private[scheduler] val shuffleToMapStage = new HashMap[Int, Stage]
- private[spark] val stageToInfos = new TimeStampedHashMap[Stage,
StageInfo]
+ private[spark] val stageToInfos = new HashMap[Stage, StageInfo]
--- End diff --
Yes, these being TimeStamped should all have been belt-and-suspenders,
since they should all be getting fully cleaned up when jobs end and stages are
removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
