Dear all, I have a question about how to measure the runtime for a Spak application. Here is an example:
- On the Spark UI: the total duration time is 2.0 minutes = 120 seconds as following [image: Screen Shot 2016-07-09 at 11.45.44 PM.png] - However, when I check the jobs launched by the application, the time is 13s + 0.8s + 4s = 17.8 seconds, which is much less than 120 seconds. I am not sure which time I should choose to measure the performance of the Spark application. [image: Screen Shot 2016-07-09 at 11.48.26 PM.png] - I also check the event timeline as following. There is a big gap between the second job and the third job. I do not know what happened during that gap. [image: Screen Shot 2016-07-09 at 11.53.29 PM.png] Is there anyone who can help explain which time is the exact time to measure the performance of a Spark application. Thanks in advance, Fei
