OrangeFactor [ http://brasstacks.mozilla.com/orangefactor/ ] now displays oranges/push; it used to display oranges/testrun.
The definition of testrun was always a bit fuzzy, but was intended to compensate for the fact that some pushes will naturally have more oranges than others due to manual or periodic retriggers. For example, PGO builds on m-c are periodic, so they may appear 0, once, or multiple times on a single commit depending on the rate of incoming pushes, and how many times they're triggered for a particular commit will influence the total number of oranges that exist for that commit. At the time OF was originally designed, the assumptions behind our definition of "testrun" produced a reasonably smooth graph of oranges over time, but over the past couple of years, these assumptions have broken down, with the result that the OF has become pretty arbitrary, although it still is useful as a relative yardstick. Because of these issues, we've changed OrangeFactor to display the actual numbers of oranges/push. This means that the data is more precise, but also more variable. In particular, you will notice significant spikes of the OF on weekends, which reflects both the reduced amount of coalescing that occurs then as well as the fact that we're more likely to get multiple triggers of PGO and other builds against a single commit, because the rate of commits is lower. This is particularly noticeable when looking at multi-week time slices (the default view shows only the most recent week). What this means is that if you're using OrangeFactor to watch trends in intermittents over time, you should be careful to compare weekdays against weekdays, and weekends against weekends, or use the 7-day moving average; comparisons of weekends to weekdays are likely to be misleading. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
