Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3009#issuecomment-62497811
One approach, which I'm exploring now, is to show a combined progress bar
that just tracks "total number of tasks" summed across all stages involved in
that job. To prevent weird "progress bar travels back in time" issues, I'm
planning to extend `JobUIData` to track these task counts. I'll add a stage
attempt -> job id mapping to track which job is associated with the current
stage and use this to figure out which counters need to be updated when tasks
finish.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
