On Wed, 7 Nov 2001, Nathan Torkington wrote:
> I want to see a document that says how Parrot implementers should
> write tests for the features they implement. Does *anyone* understand
> how to do this? If so, could you please step forward and write such a
> document and become a hero.
I thought about doing this a couple of days ago, but because we are
using perl5 to run our test suites at the moment such a document will
shortly be pointless. Essentially tests are blocks of assembly code,
and the output we expect the block to produce, but wrapped up in (a
whole hunka burning) perl[0].
We will need to get rid of the perl side of these tests while
retaining the parrot side yet while still being able to use perl to
run and report on them until we've got enough parrot that we can run
parrot tests using parrot. I've a plan for this but will be busy for
a bit, the plan will require some reorganisation of where things live
at the moment, especially as they live somewhere silly at the moment.
All that wasn't very useful, so for now people should write a test by:
Finding an appropriate file in t/op/*.t, or create a new one.
Add a test by:
incrementing the number of planned tests.
putting some code in like this:
output_is(<<'CODE', <<'OUTPUT', "name for test");
*** a big chunk of assembler, eg:
print 1
print "\n" # you can even comment it if it's obscure
end # don't forget this...!
CODE
*** what you expect the output of the chunk to be, eg.
1
OUTPUT
People should continue to write tests which:
o Probe the boundaries (including edge cases, errors thrown etc.) of
whatever code they've just written. These should include potentially
out of band input unless we decide that compilers should check for this
themselves.
o Are small and self contained, so that if their new feature breaks we
can identify where and why quickly.
o Are valid, essentially that they conform to the additonal documentation
that accompanies the feature. You did write that as well, didn't you?
o Are a chunk of assembler and a chunk of expected output.
[0] Someone mentioned something about this confusing the line endings
in win32, which will cause erroneous test failures, I might look at this
as well.
Alex Gough
