Re: the future of testing

2008-01-11 Thread Allison Randal
[EMAIL PROTECTED] wrote: A possible scheme might be a directory hierarchy matching the OS/CPU combination, e.g. Linux/x_86, Linux/i_64, Solaris/Sparc, containing dummy files whose names match the processes NOT to be run for that environment. (The precise structure would depend on which combinati

Re: the future of testing

2008-01-11 Thread ajr
> I think the data will support the idea that a directory structure based on OS/CPU is probably not the way to start. > Quite possibly. The proposed file could suffer from the same combinatorial explosion, if not properly structured. Does anyone have a good idea of the most economical structure for

Re: the future of testing

2008-01-10 Thread Andy Dougherty
On Thu, 10 Jan 2008, [EMAIL PROTECTED] wrote: > Some hand-waving on the problem of configuration and test selection, (as > the two appear to share the issues, an ideal solution would address both). > > For any usable environment, a large set of common processes have to be > executed, with a small

Re: the future of testing

2008-01-10 Thread ajr
Some hand-waving on the problem of configuration and test selection, (as the two appear to share the issues, an ideal solution would address both). For any usable environment, a large set of common processes have to be executed, with a smaller, OS &&/|| CPU specific set omitted. One way to do this

Re: the future of testing

2008-01-10 Thread Allison Randal
Gabor Szabo wrote: [darwin] t/pmc/foo.t 3 5-7 9 # platform doesn't support libfoo t/pmc/bar.t 1 42 ... This seems to be too obvious to be a real question but what if someone adds a new test in the middle of bar.t ? Will she have to remember to update the numbers in the central config file? T

Re: the future of testing

2008-01-10 Thread Gabor Szabo
On Jan 10, 2008 12:23 PM, Allison Randal <[EMAIL PROTECTED]> wrote: > Aye, I would want to improve on the Python solution. But maintaining a > config file something like: > > [darwin] > t/pmc/foo.t 3 5-7 9 # platform doesn't support libfoo > t/pmc/bar.t 1 42 > ... > > [MSWin32] > t/pmc/foo.t 32 >

Re: the future of testing

2008-01-10 Thread Allison Randal
Rafael Garcia-Suarez wrote: That assumes that tests are skipped per file, which is not always the case (sometimes you want to skip only one test, sometimes even to work around an OS bug that appears only in one specific version). But reorganizing platform-dependent tests might be a good idea.

Re: the future of testing

2008-01-09 Thread Eric Wilhelm
# from Rafael Garcia-Suarez # on Wednesday 09 January 2008 05:36: >Allison Randal wrote in perl.perl6.internals : >> In the Python test suite, there's a single global location to >> declare a list of test files that are expected to be skipped on a >> particular platform. This has a much cleaner fe

Re: the future of testing

2008-01-09 Thread Rafael Garcia-Suarez
Allison Randal wrote in perl.perl6.internals : > In the Python test suite, there's a single global location to declare a > list of test files that are expected to be skipped on a particular > platform. This has a much cleaner feel than our own motley collection of > skip and todo markers in vari