Surya Hebbar has posted comments on this change. ( http://gerrit.cloudera.org:8080/23154 )
Change subject: IMPALA-9846: Enable Aggregated Runtime Profile by Default ...................................................................... Patch Set 18: (1 comment) http://gerrit.cloudera.org:8080/#/c/23154/18/be/src/util/runtime-profile.cc File be/src/util/runtime-profile.cc: http://gerrit.cloudera.org:8080/#/c/23154/18/be/src/util/runtime-profile.cc@2294 PS18, Line 2294: result.total = std::accumulate(vals.begin(), vals.end(), 0.0L); > Total is not a good metric to present in many cases. We should insead print In most cases 'total' is as helpful and meaningful statistic as 'mean' for that metric. If we deliberate on this... Even the existing aggregates min/max/mean also do not make much sense in many cases. For example, 1. mean of 'NumColumns' does not have much meaning 2. mean of 'AverageThreadTokens' becomes average of average, which looses all meaning, as average of average is not equivalent to average of actual values 3. mean of 'MaxCompressedTextFileLength' or 'MaxPartitionLevel', the mean of only 'max' values does not have much meaning So, in most cases 'total' is as helpful and sensible as 'mean' for that metric. And this is not something to decide during representation, but it is to be deliberated during creation, when choosing between 'AveragedCounter' or 'Counter' for a metric. The only conclusion we can draw from all of the above evidence is that, statistics are statistics, they are not always insightful, they do not always represent something meaningful, nor is it necessary for the statistics to always do so. -- To view, visit http://gerrit.cloudera.org:8080/23154 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If41d6322361fba82c946efd614cc7d28cb1c36e8 Gerrit-Change-Number: 23154 Gerrit-PatchSet: 18 Gerrit-Owner: Surya Hebbar <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Surya Hebbar <[email protected]> Gerrit-Comment-Date: Tue, 11 Nov 2025 06:18:08 +0000 Gerrit-HasComments: Yes
