Hello Paul,
Paul Hardy [2018-08-19 7:44 -0700]:
> It accesses
> environment variables that have to do with the installed package file
> locations. By default, it will expect binaries to be in
> ${DESTDIR}${bindir}. So I guess it wouldn't be necessary to define
> other things, such as $srcdir, but only directories that point to
> /usr/..., plus autopkgtest variables like AUTOPKGTEST_TMP.
But then an absent $DESTDIR should be just right, as you really want to check
things in /usr, not in some <build dir>/usr/.
> By defining a few environment variables (like setting bindir to
> /usr/bin), it will be more straightforward for people running
> autopkgtest with Autotools to use a default "make installcheck"
> target.
Maybe, but I still don't believe this actually makes sense:
* Defining random variables with generic names like $bindir might also
influence other tests or packages, which makes them work differently than
they would in a normal OS install.
* autopkgtest is meant to be neutral to the test frame work that you use, as
there are literally hundreds of them. There is no way it could maintain
support for these.
* This is too magic. Test commands should be very explicit and allow the
developer to reproduce them step by step in a shell.
* For even `make` to work, you need at least a configured tree, and thus all
the build dependencies, and most likely even need to build the package.
autopkgtests should generally avoid that.
* I don't believe that setting static values for these variables even works
for all automake projects. If they would, why are they even variables?
> If we can expect that autopkgtest uses package executables in
> /usr/bin, package data files in /usr/share/<package>, likewise with
> other FHS directories, could you update the tutorial page to mention
> that? Right now there is just an implication on that page from the
> sample script that the binary executable will be in a directory in
> $PATH.
Right, as Debian policy mandates that user-executable programs should be in
$PATH. So it's generally preferrable to call programs in $PATH without an
explicit path, to make sure they are actually in the correct place.
But note that there's also e. g. /sbin, or you might want to test a
Python module, or even the validity of a documentation package.
The spec [1] currently says "However note that the tests must test the
*installed* version of the program.", which is really the only condition. But
indeed this should really say "*installed* version of the package", as
autopkgtests are in no way limited to testing programs. I adjusted that in [2].
Martin
[1]
https://salsa.debian.org/ci-team/autopkgtest/blob/master/doc/README.package-tests.rst
[2]
https://salsa.debian.org/ci-team/autopkgtest/commit/90876a7598aee4408296857b8bf33a63e0355fc3
