On Sat Dec 02 20:22:14 2006, [EMAIL PROTECTED] wrote: > > Tests of Parrot's build tools are not tests of Parrot itself. Once > Parrot's make call has > succeeded, testing the components of the build tools is irrelevant. > Hence, including tests of > the build tools in the suite of tests called by 'make test' is also > irrelevant. The build tools can > only be validly tested in a 'pre-make' environment. But that raises > the question: How far > 'pre-make': a pre-Configure.pl environment, or a post-Configure.pl, > pre-make environment > (or some combination of the two)? > > Which boils down to a more fundamental question: What exactly are we > trying to accomplish > via testing of Parrot's build tools? >
Continuing my thinking-out-loud on this subject: Perhaps what we need is a 'pre-make' test suite whose tests a developer (but not a regular user) would run before calling 'make'. The purpose of these tests would be to answer the question, "Are the tools we are about to use to build the Parrot installation capable of doing so?" The tests would test the most important features of the various Perl scripts invoked during 'make' without doing a real build. Any files created during these tests would be created in self-eliminating temporary directories. The test suite would begin by doing a spot check of directories and files to see that files that ought to be present after running Configure.pl (e.g., Makefile) *are* present but that files that are only created during 'make' (e.g. src/pmc/*.c) *are not* present at that time. Comments? Thanks. kid51
