On Apr 6, 2010, at 5:28 PM, Thomas Tuegel wrote:
> On Tue, Apr 6, 2010 at 7:43 PM, Rogan Creswick <cresw...@gmail.com> wrote:
>> test-framework and test-runner both address the second problem, and
>> those solutions can be kept separate, at least for now. Figuring out
>> the best way to specify test commands, dependencies, build/execution
>> order, etc. is going to take some substantial effort, and I think that
>> should be the first goal of the project.
>
> Ok, this is the bottom-line that I didn't understand after our first
> exchange, but I think now I do: I should entirely scrap the second
> aspect of my proposal and focus exclusively on making Cabal build and
> run test programs.
I concur with this conclusion.
>> Cabal allows for multiple executable sections -- are multiple test
>> sections allowed? If so, how are they handled when 'cabal test' is
>> invoked? If not, will there be any support for running multiple test
>> suites? (more on this below).
>>
>> While the test executable could be configured to run different sets of
>> tests (at runtime? hm.. we may need more flags to 'cabal test'), there
>> are some situations it's necessary to build multiple test suites
>> because of odd library dependencies. (for example, testing certain
>> combinations of libraries--imagine supporting multiple versions of
>> ghc.)
>
> I had intended to allow multiple 'Executable' sections; that's what I
> meant by "this is really an 'Executable' stanza by another name". I
> think that takes care of your concern about building multiple test
> suites with different dependencies, also.
>
> My thinking is that 'cabal test' should run all the tests that were
> built, and 'cabal test foo' should run only the test named 'foo'.
This sounds like a good idea to me.
>> The existing Executable sections may serve the need fine, if we could
>> specify how to run the tests in a different way. Perhaps a list of
>> test commands could be specified instead, eg:
>>
>>> TestCommands: foo-test-ghc6.6,
>>> foo-test-ghc6.8,
>>> foo-props --all
>>
>> Anyhow, just food for thought.
>
> One of the reasons I prefer implementing a dedicated 'Test' stanza is
> that it makes it easier to tell which executables to install. There
> may be situations where we want to install test programs, but there
> will _always_ be situations where we _don't_ want to. (Maybe
> '--{en,dis}able-tests' passed to 'cabal install' should control this?)
> Could we take a 'TestCommands' list and parse out the options to get
> the executable names? Yes, but relying on that to work makes me
> uneasy. I think the more robust way to specify executable-specific
> options instead would be to add a field to the 'Test' section
> ('run-options' seems to be consistent with the naming scheme) that
> doesn't exist in 'Executable'.
>
> (I've snipped some of the comments here; if forget about test
> frameworks for this proposal, it's all tangential.)
I agree with your reasoning here about having a separate Test stanza so that
tests can be kept separate from the rest of the package.
- Greg
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
