On Tue, 12 Nov 2002 10:00:05 +0000, Michael Lazzaro wrote:
> On Tuesday, November 12, 2002, at 10:01 AM, Brent Dax wrote:
>> Why use POD like this instead of a more atomic version of the standard
>> testing format used by Perl 5? We can use the directory structure to
>> organize things. Since most tests are not worthy of inclusion in the
>> docs (do you really want "$foo eq reverse scalar reverse $foo" in the
>> documentation for reverse()?) I don't see why we can't do this.
> Dunno, looking for a way where we can harness the authors for producing
> tests in the least-intimidating way. I'm not really thinking that the
> tests will be a useful, displayable part of the docs, but I figure the
> closer we can tie it into writing each docs section, the more likely
> we'll be to get contributors to actually think about it & do it. I
> could be wrong there, tho...
Advantages of inline tests:
- close to the documentation
- one place to update
- harder for people to update docs without finding code
Disadvantages:
- doc tools must skip tests normally
- pure-doc patches are harder
- some tests (bugs, regressions) don't naturally fit in docs
- makes docs much larger
- adds an extra step to extract tests before running them
- adds weight to installed documents (or they must be extracted)
Advantages of external tests:
- tests can be grouped by purpose/thing tested within tests
- test files can be smaller
- individual tests can be run during development
- tests can be grouped by subsystem/purpose
Disadvantages of external tests:
- proper organization is important
- multiplies files
- doc patchers may find test to patch and vice versa
On the whole, I prefer external tests. Brent's schema looks good.
In Perl 5 land, the culture expects documentation and test patches with the
addition of a new feature. I think we can cultivate that for Perl 6. As
Brent also alluded, there will probably be standalone regression tests anyway.
Most everything else is solvable by the right kinds of community pressure.
"Don't know where to put your test? Look in t/scalar/DWIM/numify.t."
-- c
