On Sat Feb 23 15:29:22 2008, [EMAIL PROTECTED] wrote: > The patch attached, diff.trunk.tcif.txt, addresses both RT 47391 > (t/configure/1*.t tests incorrectly rely on init::defaults_ and RT 47503 > (Remove config::init::defaults From configure tests). It accomplishes > the following: > > * In the tests now found in t/steps/*.t (formerly in t/configure/1*.t), > the patch removes the need to import particular configuration step > classes (most frequently, config::init::defaults) and run their > respective runstep() methods. > > * The patch revises the harness running the pre-configuration tests such > that the tests in t/steps/ are run in exactly the same order in which > Configure.pl runs the corresponding configuration step classes. > > * In each test, the configuration step is first run just as it will be > by Configure.pl. The results of that step are recorded on disk. The > test file then reads from the disk the results of all previous > configuration steps and uses that in tests for the step currently being > tested. Thus, for the testing of a given configuration step, individual > tests have available to them all the information found in the > Parrot::Configure object at that same point during regular configuration. > > * Since the Parrot::Configure object is a singleton, only one instance > of it can be constructed in any file in t/steps/*.t. So when we want to > run a particular configuration step class for the purpose of recording > its result on disk, we have to construct an object of a different class: > a Parrot::Configure::Parallel object. Parrot::Configure::Parallel > objects are really the same thing as Parrot::Configure objects, so, not > surprisingly, many of the utility subroutines previously found in > Parrot::Configure have now been refactored out into > Parrot::Configure::Base, from which both Parrot::Configure and > Parrot::Configure::Parallel inherit. The manipulation of > Parrot::Configure::Parallel objects over the course of t/steps/ is > conducted by Parrot::Configure::Parallel::Trace. > > * The harness which runs pre- and post-configuration tests, > Parrot::Configuration::Options::Test, has been revised to require an > up-to-date version of the 'prove' utility (from TAP::Parser 3.05 or > higher) so that options passed to Configure.pl on the command line can > be passed through to the test files in t/steps/. > > * The pre-configuration tests (t/configure/*.t and t/steps/*.t) and the > post-configuration tests continue to be run by 'perl Configure.pl > --test' but are no longer run by 'make test'. The reasons are twofold: > > (1) Since we don't (yet) require TAP::Parser in any version for > running 'make test', it would be incorrect to require all Parrot users > to install it just to run some tests. However, since Parrot developers > hacking on the configuration system tend to be advanced users, it should > not be burdensome on them to install an up-to-date version of > TAP::Parser for the purpose of conducting specialized testing. > > (2) As I argued in my presentation at YAPC::NA::2007 in Houston, > running tests of the configuration and build tools during 'make test' is > largely beside the point, since by that point you have both configured > and built Parrot. Configuration tools ought to be tested before you > actually configure (with 'perl Configure.pl --test' or 'perl > Configure.pl --test=configure). Build tools ought to be tested before > you actually build (with 'make buildtools_tests', 'perl Configure.pl > --test' or 'perl Configure.pl --test=build'). Any member of the Parrot > project working on files in the lib/Parrot/ or config/ directories > should run these tests early and often. > > * Revisions consistent with the above have been made in > config/gen/makefiles/root.in. Tests for > configuration-testing-in-parallel have been added in > t/configure/061-parallel.t through 064-parallel.t. Some other revisions > have been made in Parrot::Configure::Messages, > Parrot::Configure::Step::List and t/pharness/*.t. Last but not least, > since I'm constantly having to go into Parrot::Configure and look at its > code and am sick of scrollling down through 300 lines of POD, I moved > the code in this class up to the top of the file. > > Adoption of this patch will require that we think of our test files in a > way to which some of us are not accustomed. Those of us who have > written CPAN distributions are accustomed to writing test files which > are self-contained units that make no use of information passed to the > test harness via command-line options. That's not sufficient here, > because Parrot configuration can go in many different directions > depending on what's passed on the command line. I am grateful to Andy > Armstrong and other contributors to the perl-qa list who responded to my > request for a version of 'prove' that would enable command-line options > to be available to an individual test file's @ARGV. > > Please review. This patch is necessarily large and is likely to give > the 'patch' program fits. But you can play in a sandbox for the SVN > branch in which I developed this approach. To see > configuration-testing-in-parallel in action, call 'svn co > https://svn.perl.org/branches/tcif/' and run 'perl Configure.pl --test' > with whatever command-line options you customarily use when configuring > in trunk. > > Thank you very much. > kid51
A quick test on osx intel carps about my version of TAP::Harness, as you noted. After resolving that dependency and running config with -test: <SNIP> All tests successful. Files=261, Tests=3095, 84 wallclock secs ( 1.27 usr 1.69 sys + 25.98 cusr 13.99 csys = 42.93 CPU) Result: PASS <normal config output> All tests successful. Files=42, Tests=1149, 31 wallclock secs ( 0.31 usr 0.22 sys + 21.59 cusr 2.84 csys = 24.96 CPU) Result: PASS The tests seem to pass. Regards.
