Date: Mon, 30 Mar 2020 03:26:23 +0100
From: Roy Marples <r...@marples.name>
Message-ID: <c4c61fb2-3074-8220-913c-72d407654...@marples.name>
| Without ncurses installed I have no idea how to test that.
| Ideas of course welcome.
You can write ATF tests that use programs not normally installed,
they just need an atf_require_prog (or whatever the equiv is for
tests written in C or C++) so that the test will be skipped when the
relevant prog is not installed ... or you can write your own tests for
things existing manually and manually atf_skip if the preconditions
needed are not met.
The b5 ATF tests won't fail in those cases, as they don't get pkgsrc
progs installed, but relying on those to detect problems isn't a good
idea anyway, as the delays before they get run, and the "fail 3 times before
being reported widely" means that it can take ages after changes are
made before anyone notices issues that way (nb: I don't mean they're
useless, they catch lots of problems, just we should be running the
tests ourselves as well, when changes are made - and when you run them
you'll have all your normal pkgsrc progs installed).
So, when either you change something, or someone else has changed something
in which you have some expertise or interest, everyone concerned should be
running the ATF tests that are applicable (there is no need to run them all,
though that also doesn't hurt anything but our patience).
Of course, this only is only useful if there are ATF tests which actually
test things properly - too many of our tests are simply reactions to fixed
bugs (make sure bug X doesn't come back) rather than real tests of expected
functionality (including that which is believed to work, and has never
actually failed that anyone is aware of ... including the most basic and
trivial elements).
kre
