> So the main thing we'd lose is graph server monitoring of Trace Malloc Leaks. > This is occasionally useful, but in a limited way because the monitoring > process is unowned, and because the current value of the benchmark is high > enough that small changes are ignored by the monitoring system. It has some > chance of sometimes making large regressions easier to detect.
In that case, is consensus that by disabling these tests we're only taking on minimal pain when investigating major leak regressions, and that we think those leaks will still be obvious through other feedback? If so, +1 to the proposal. -Alex On Jul 17, 2013, at 10:54 AM, Matt Brubeck <mbrub...@mozilla.com> wrote: > On 7/17/2013 4:26 AM, Ted Mielczarek wrote: >> The only valuable thing we're losing from shutting this off is >> tracemalloc coverage, which we don't have elsewhere. I don't have any >> evidence to show that anyone has actually looked at the tracemalloc data >> or done anything useful with it in recent history, though. Leaks >> discovered by tracemalloc don't make the test fail, so we're not >> measuring much there. > > Trace Malloc measures three things: leaks, max heap size, and number of > allocations. All of these are reported to the graph server and use the same > monitoring as Talos benchmarks. Like Talos data, this data does not have a > consistent process or owner responsible for watching it or improving it, but > there are some people who watch for alerts from the monitoring script and try > to investigate regressions. > > Trace Malloc Leaks is a fairly stable measurement; our monitoring has not > found regressions in it very often, but it does occasionally (about once a > year?) help alert us to an actionable bug that might have been harder to fix > otherwise. The latest example is: > https://bugzilla.mozilla.org/show_bug.cgi?id=872947 > > Trace Malloc MaxHeap is somewhat useful, but largely overlaps with the > measurements produced by AWSY and Tp5, both of which are probably more useful > and representative. > > Trace Malloc Allocs is not very useful from what I understand; a change in > this benchmark does not necessarily correlate with any user-visible change. > > So the main thing we'd lose is graph server monitoring of Trace Malloc Leaks. > This is occasionally useful, but in a limited way because the monitoring > process is unowned, and because the current value of the benchmark is high > enough that small changes are ignored by the monitoring system. It has some > chance of sometimes making large regressions easier to detect. > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
