Re: [swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-29 Thread Brian Gesiak via swift-corelibs-dev
I received some feedback from Paulo Faria that I’d like to share here: Is there a way to change the performance test baselines by code? I can’t find a way to do it today. So my question would be, why can’t we set the baselines by code? Is it because we need a different baseline for each different

Re: [swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-26 Thread Brian Gesiak via swift-corelibs-dev
Hi, it’s me again. :) I figured out how we can do this on Darwin: by using .xctestconfiguration files. SwiftPM performance tests on Darwin, using XCTestConfiguration files Xcode passes all sorts of variables to XCTest by specifying the XCTestConfigurationFilePath=/path/to/an.xctestconfiguration p

Re: [swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-25 Thread Brian Gesiak via swift-corelibs-dev
I received some feedback on this proposal from Ankit Aggarwal, which centered on how developers would edit and update their baseline metrics. Here’s what I’m envisioning specifically: Two new command-line options for swift test 1. swift test --performance-metrics . This is a path to a direct

[swift-corelibs-dev] Performance testing via SwiftPM and XCTest

2016-07-24 Thread Brian Gesiak via swift-corelibs-dev
Hello corelibs-dev and build-dev, Back in May, Brian Croom implemented performance testing in swift-corelibs-xctest: https://github.com/apple/swift-corelibs-xctest/pull/109 I’d love to see Swift developers use this feature to measure the performance of their code. I think we’ll need to add functi