On 2013-04-23 1:17 PM, Ed Morley wrote:
On 23/04/2013 17:28, Kartikaya Gupta wrote:
On 13-04-23 00:39 , Ehsan Akhgari wrote:
How hard would it be to
gather a list of the total number of patches being backed out plus the
amount of time that we spent building/testing those, hopefully in a
style similar to
<http://people.mozilla.org/~catlee/highscores/highscores.html>?
Not trivial, but not too difficult either. Do we have any evidence to
show that the try highscores page has made an impact in reducing
unnecessary try usage? Also I agree with Justin that if we do this it
will be very much a case of sending mixed messages. The try highscores
list says to people "don't land on try" and the backout highscores list
would say to people "always test on try".
It's worth noting that when I've contacted developers in the top 10 of
the tryserver usage leaderboard my message is not "do not use try", but
instead suggestions like:
* "please do not use -p all -u all when you only made an android
specific change"
* "you already did a |-p all -u all| run - on which mochitest-1 failed
on all platforms, so please don't test every testsuite on every platform
for the half dozen iterations you ran on Try thereafter" (at much as
this sounds like an extreme example, there have been cases like this)
Yes, this! ^
The messaging around this should not be to tell people "always test on
try". It should be to help them figure out how to make better judgement
calls on this. This is a skill that people develop and are not born
with, and without data it's hard an an individual to judge how good I'm
at that.
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
