I think that there’s a lot of room for improvement in how we measure and
analyze perf test results. And I like some of your ideas of adaptive numbers of
runs and what to do with outlier results, etc… that you presented below.
We would very much like to not diverge the API of the CoreLibs XCTest
> Unfortunately our corelibs implementation of XCTest isn’t ready yet for
> performance testing.
That's why I'm here; I'm taking the temperature on implementing it. I'm at the
pain level where I need a solution in the next several months, even if the
solution is to code it up myself. My tests
Hi Drew,
Thanks for the detailed info on your issue. I see you filed a radar, and that
is indeed the best way to make sure an issue on Darwin platforms is addressed.
Unfortunately our corelibs implementation of XCTest isn’t ready yet for
performance testing.
- Tony
> On Dec 10, 2015, at 3:41
Hello folks,
I’m one of the heavy users of XCTest.measureBlock as it exists in Xcode 7.2.
To give some hard numbers, I have ~50 performance tests in an OSX framework
project, occupying about 20m wall clock time total. This occurs on a
per-commit basis.
The current implementation of measureBl
