Thanks for the feedback, Dmitri! I think I misstated options #2 and #3: where I typed "move business logic like IsTestCandidate to libIDE", I meant "move a lot of the non-XCTest-specific logic from SourceKit to libIDE, such that implementing IsTestCandidate would be trivial in SourceKit and our test generation tool". The last thing I'd want to do is couple libIDE and XCTest!
On Sun, Apr 3, 2016 at 6:36 PM, Dmitri Gribenko <griboz...@gmail.com> wrote: > On Sun, Apr 3, 2016 at 2:11 PM, Brian Gesiak <modoca...@gmail.com> wrote: > > I think #2 is the best option. It’s less work than both #1 and #3. I > believe > > logic like IsTestCandidate belongs in libIDE anyway—SourceKit should > stick > > to XPC and asynchronous communication with libIDE. > > I like #3 better (an option to swiftc), because that would decouple > the test discovery tool from the Swift compiler. That would allow you > to use the discovery tool with different compilers. And, because we > would avoid statically linking libIDE, it would mean one less copy of > LLVM, Clang and Swift in the toolchain. > > My concern with adding a swiftc option would be deciding upon an interface to it. Would we want something like clang-query--a generic method of finding methods that match a set of criteria? That seems like it would be a lot of work to implement. Perhaps I'm misunderstanding what you have in mind for swiftc. Could you elaborate? > > Not being an expert in many of these components, I have several > questions: > > > > I’m assuming the reflection API to return a list of instance methods on a > > XCTestCase subclass is not ready yet, and won’t be for some time. Is this > > accurate? > > I think so. > > > I’m assuming that SourceKit is intended to be an asynchronous wrapper > over > > libIDE, and that logic like IsTestCandidate should be moved to libIDE. Is > > this accurate? > > SourceKit has a lot of functionality of its own, but moving this > particular piece of logic to libIDE sounds reasonable. > > > I’m assuming that SourceKit is coupled with XPC, and that it would be > more > > work to port it to Linux than it would be to move its logic to libIDE. Is > > this accurate? > > It is not tightly coupled with XPC, there is a portability layer that > you could implement for Linux. You would need to decide on an IPC > mechanism and serialization format though. > > > If you have thoughts/feedback, please reply to this email or comment on > > SR-710. Your input would be greatly appreciated!! > > I'm wondering how feasible is it to change the XCTest API to > accommodate better the Swift language that we have, rather than trying > to add custom tooling to make the existing API work. Adding magic > tooling that adds behavior not present in the language seems unnatural > to me. > > Compare with StdlibUnittest -- by using an API to build tests we get > the following advantages: > > - We completely avoid having the issue of test discovery, executing > the code discovers the tests. No reflection needed! > > - We can add attributes to tests (for example, skip, xfail). In the > current XCTest API this would require adding some kind of user-defined > attributes, which is another language which is a long way from being > designed and implemented. > > - We can define data-parameterized tests. > > - Tests can be dynamically synthesized by control flow. In the > current XCTest API, dynamically generating tests would mean > dynamically generating methods, which is even more far off than > read-only method reflection. > I definitely agree that all of these are fantastic features. However, the corelibs-xctest README spells out the following goal for the project: "This version of XCTest uses the same API as the XCTest you are familiar with from Xcode. Our goal is to enable your project's tests to run on all Swift platforms without having to rewrite them." This goal often conflicts with what would be the best API for a Swift testing framework--we are forced to use mutable reference types, extensive subclassing, include a ton of optional properties--all in order to mirror exactly the Apple XCTest API. I suppose we *could* modify the SDK overlay for XCTest to provide an API more suitable for Swift, but I think that would involve a lot of coordination with Apple XCTest and Xcode teams, some swift-evolution proposals, and a potentially harsh migration path for developers writing their tests in Apple XCTest. So while I think corelibs-xctest and Apple XCTest can (and should) evolve to better take advantage of Swift features over time, I don't think we can migrate both before Swift 3's release. (+CC Mike Ferris, who works on Apple XCTest and corelibs-xctest.) - Brian Gesiak
_______________________________________________ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
