After getting an e-mail every single time m-c was merged for a day or two, I filtered the e-mails and completely forgot about them. I imagine most other people did the same. If we fix bug 752002, we'd also need to change the e-mails so as to get around everyone's existing filters.
More on topic: I think the essential problem is, you can spend a week chasing down a perf regression when there's a good chance it's not your fault (and also a good chance it's not a regression). So people are making a reasonable trade-off here when they ignore these problems. This is the essential problem that the Signal from Noise project is tackling. But it's a medium-term play at best, since they're rewriting large pieces of infrastructure. So the question is: Are we willing to sit back and wait for SfN, or do we need improvements Right Now? It's my feeling that incremental improvements are often more beneficial than the "stop fixing all but critical bugs in existing code, focus on v2 code" approach that we seem to be taking with SfN, but I understand that there are trade-offs here. Anyway there are simple things we could do to improve the current situation without taking too much of the focus away from SfN, such as unbreaking compare-talos (e.g. [1], but also in general applying real statistics so it's simple to find true regressions and rule out false ones). It's just a question of priorities. -Justin [1] https://bugzilla.mozilla.org/show_bug.cgi?id=696196 On Wed, Aug 29, 2012 at 8:53 PM, Ehsan Akhgari <ehsan.akhg...@gmail.com> wrote: > On 12-08-29 7:32 PM, Nicholas Nethercote wrote: >> >> On Wed, Aug 29, 2012 at 4:03 PM, Ehsan Akhgari <ehsan.akhg...@gmail.com> >> wrote: >>> >>> >>> Some people have noted in the past that some Talos measurements are not >>> representative of something that the users would see, the Talos numbers >>> are >>> noisy, and we don't have good tools to deal with these types of >>> regressions. >>> There might be some truth to all of these, but I believe that the bigger >>> problem is that nobody owns watching over these numbers, and as a result >>> as >>> take regressions in some benchmarks which can actually be representative >>> of >>> what our users experience. >> >> >> In my experience, a lot of those emails say "there was a regression >> caused by one of the following 100 patches", and I will have written 1 >> of those patches. I usually ignore those ones (though it depends on >> the nature of the patch). >> >> But if I get an email saying something like "there was a regression >> caused by one of the following 3 commits", I'll look into it. > > > Yeah, I know that those emails are not perfect, and I know that some people > do look into these problems, but I don't think that is the general trend, as > is evident from the 6-week cycle results on the latest uplift. > > Cheers, > Ehsan > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
