On Mon, Jun 15, 2015 at 07:30:34AM +0200, Martin Pitt wrote:
> Hello Dominic,
Hello,
Thank you for your reply (and indeed your prompt implementation).
> Dominic Hargreaves [2015-06-14 0:18 +0100]:
> > To allow test execution environments to discover packages which provide
> > tests, their source packages should have a ``Testsuite:`` header
> > containing ``autopkgtest`` (which is currently the only defined value).
> > Multiple values get comma separated, as usual in control files.
> >
> > My expectation having read this was that adt-run would only attempt
> > to test packages if the Testsuite header is present; however adt-run
> > actually appears to not look for this header at all;
>
> That's correct. If you run adt-run on a source tree/dsc etc., it will
> only look for debian/tests/control (or autodep8). The above is just a
> recommendation ("should") and states the reason for that.
>
> > it always processes tests for a package, including automagic test
> > suites set up by autodep8. This is probably fine in the case of
> > invoking adt-run with a single package, but not if you want to look
> > for regressions using adt-run; for example I ran it against all
> > packages I'd been rebuilding against perl 5.22, and found quite a
> > few failures which didn't appear to be related to perl 5.22 but were
> > instead tests which just didn't work.
>
> Right, we get the same on Ubuntu CI. Over time the failures pile up, I
> figure it's quite difficult for the Perl team to look at/fix all
> these.
>
> > Please could adt-run have an option to exit without running any tests
> > if the source package in question doesn't define any?
>
> So to clarify, you want an option to disable autodep8, right? If a
> source package doesn't have tests and autodep8 does not generate any
> either, then adt-run will already report "no tests in this package"
> and exit with 8. With a perl package and --no-autodep8 it would then
> do the same.
I thought about 'no autodep8', but the semantics aren't quite correct,
since a package could declare Testsuite: autopkgtest, but rely on the
autodep8 behaviour. This is in fact the common case for pkg-perl packages,
as far as I can tell.
So by 'declare tests', I was really talking about having
Testsuite: autopkgest in the control file. Sorry if that wasn't clear.
Cheers,
Dominic.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
