> On May 11, 2017, at 1:33 PM, Pavol Vaskovic via swift-dev
> <swift-dev@swift.org> wrote:
>
> Douglas Gregor suggested in SR-4814 (that was merged as dupe into SR-4669)
> that I get in contact with you because I’m working on issue SR-4669 Add a
> Benchmark_Driver --rerun N option <https://bugs.swift.org/browse/SR-4669> and
> I should make sure that I don’t break the Swift-CI.
>
> The point of that is to deal with false changes (regressions and
> improvements) that are often seen in benchmark reports from CI.
>
> I have run Benchmark_O with --num-iters=100 on my machine for the the whole
> performance test suite, to get a feeling for the distribution of benchmark
> samples, because I also want to move the Benchmark_Driver to use MEAN instead
> of MIN in the analysis.
Using MEAN wasn’t part of the aforementioned SR-4669. The purpose of that task
is to reduce the time CI takes to get useful results (e.g. by using 3 runs as a
baseline). MEAN isn’t useful if you’re only gathering 3 data points.
-Andy
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev
