On Nov 20, 2015, at 15:51, Andone, Bogdan <bogdan.and...@intel.com> wrote: > > Please ignore the Mandelbrot result. It seems it was a measurement error, as > we were unable to reproduce it on the same commit with additional runs.
Perhaps it would be possible to automatically trigger additional runs in case a result shows a drastic change like that? Just to eliminate false outliers. These benchmarks are a great sanity check which can catch small changes that have inadvertent performance hits, but we need to have confidence in the numbers otherwise people will start ignoring them. -Rasmus
smime.p7s
Description: S/MIME cryptographic signature
