On 12-08-29 9:46 PM, Joshua Cranmer wrote:
On 8/29/2012 6:03 PM, Ehsan Akhgari wrote:
Hi everyone,
The Talos regression detection emails caught a number of regressions
during the Monday uplift (see [1] for Aurora and [2] for Beta
regressions). To put things into perspective, I prepared a
spreadsheet of the most notable performance regressions [3] (and
please do take a look at the spreadsheet!).
The way the current situation happens is that many of the developers
ignore the Talos regression emails that go to dev-tree-management, and
in many cases regressions of a few percents slide in without being
tracked. This trend of relatively big performance regressions becomes
more evident every time we do an uplift, which means that 6 weeks
worth of development get compared to the previous version.
Here's another question worth asking: suppose every commit introduces a
very tiny regression. How big would this regression have to be to add up
to a 10% regression over a 6-week cycle? Assuming 400 commits per cycle,
I calculate that you need to consistently have about 0.023% regression
per commit to add up to that 10% regression. 400 commits is much fewer
than the rate we commit at, but it illustrates that some regressions
might not be because of one big regression no one missed but a death by
a thousand papercuts.
We do see regressions which cannot be attributed to a few changesets,
but I think we can all agree that every single changeset contributing
the same amount of performance regression in the 6 week cycle is highly
unlikely. I'm not sure what you're trying to suggest here.
Ehsan
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
