On Mon Dec 17 08:24:49 2007, doughera wrote: > It passed all tests here. I haven't had time to review the actual patch, > but the strategy you describe seems sound. > > However, I do want to worry about one thing: I worry about the growing > time required to run all of these tests. Just running the t/configure/* > tests now takes 8 minutes on my system, and much of it seems redundant. > For example, files t/configure/107-inter_progs-0[23].t are 131 lines > long each, but only differ in 2 lines of code. Further, the output of > the two tests is identical except for the file name. Looking at all 5 > inter_progs-0[1-5].t tests, do we really need to test 5 times whether or > not init::defaults has a description? In fact, looking at the output, > there are apparently 19 separate tests that are run 5 times each. > Is that really necessary? >
It is true that certain tests are duplicated across files and could be eliminated with no loss in test coverage. The test for whether a given step has a description is the most obvious case in point. We need separate test files to test different ways of calling a given Parrot configuration step. The Parrot::Configure object is a singleton, so you can't create more than one instance of it per test file. Hence, to explore how the step executes in the presence of either different command-line options or different OS environments, I needed to create multiple test files per config step. The editing was much quicker when I simply copied the contents of one test file to a new one and then wrote the code for the variant. Having run these configuration tests hundreds of times on my iBook and Linux server, I feel confident in saying that the duplicated tests are not the ones that take up a lot of time. (And I'm surpised that they take 8 minutes on your box. My iBook is pokey but even there they don't take more than 3 minutes.) In any event, where I'm headed with the configuration tests is to make them specialized tests -- much like the build tools tests -- which are not run as part of 'make test' but which people doing revisions in config/ and lib/Parrot/ should definitely run. It is my hope that the amount of time which the average Parrot developer spends on running the configuration tests becomes zero. kid51
