Hi,
I'm running Spark 2.4.4. When I execute a simple query "select * from table
group by col", I found the SparkListenerTaskEnd event in event log reports all
negative time duration for aggregate time total:
{"ID":6,"Name":"aggregate time total (min, med,
max)","Update":"2","Value":"-46","Internal":true,"Count Failed
Values":true,"Metadata":"sql"}
The same thing happens in SparkListenerStageCompleted event:
{"ID":6,"Name":"aggregate time total (min, med,
max)","Value":"-133","Internal":true,"Count Failed
Values":true,"Metadata":"sql"}
Then I checked the history server web UI, but the SQL tab displays positive
numbers for the HashAggregate operator:
aggregate time total (min, med, max): 35 ms (0 ms, 2 ms, 6 ms)
I'm wondering is this a bug in Spark 2.4? If not, how does Spark compute the
"aggregate time total" from those negative numbers?
Thanks
