Hi,
As you can see in the picture below, the application last job
finished at around 13:45 and I could see the output directory updated with
the results. Yet, the application took a total of 20 min more to change the
status. What could be the reason for this? Is this a known fact? The
application has 3 jobs with many stages inside each having around 10K
tasks. Could the scale be reason for this? What is it exactly spark
framework doing during this time?
[image: Screen Shot 2018-12-25 at 5.14.26 PM.png]
Thanks,
Akshay
