On 08/31/2012 12:33 AM, Rafael Ávila de Espíndola wrote: > I was recently hit by most of the shortcomings you mentioned while > trying to upgrade clang. Fortunately, I found the issue on try, but I > will admit that comparing talos on try is something I only do when I > expect a problem.
I'm a bit late into the game here but I would like to mention that I've never been able to use correctly talos on try. The few times one of my patches were suspected to regress, I sent them to try and every time the results were useless. The delta between min and max from different runs was so huge that I could have been able to use those results to say that my patches were actually improving performances as much as I could have say they were reducing them. As a result, my process has always been to pretend to use try, re-land my patch and hope it was actually not guilty. So far, I never had to try to fix a patch actually regressing something. Anyway, I don't think an indicator like talos is really useful if it's not reliable at all on try and locally. At the end of the day, you might easily end up doing like Justin who had to push a startup regression or simply forget about your patch (depending on the cost/benefit). In my opinion, trying to make talos reliable on try should be the first step if we want developers to care mare about those tests. Cheers, -- Mounir _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
