(Note crosspost to debian-devel and ubuntu-devel. If your reply does not concern both Debian and Ubuntu, please remove the inappropriate list. And of course, in general, do not CC individual posters unless they ask for it.)
Following discussions at Ubuntu's Montreal meeting, on debian-devel, and elsewhere, I've done an initial implementation of my planned automatic testing framework. In many ways it is still a proof of concept, and it lacks many useful features, but it can already be used to test packages in a chroot in a basic way. The package with the test machinery is `autodebtest'[1] and can be found[2] in Ubuntu dapper and Debian sid. The dependencies are relatively conservative so I think the .deb should install and work with breezy or sarge too. There is currently only one test in one package. I picked `mawk' as an example because it is a package which is (i) very stable (ii) has limited dependencies and (iii) has at least some upstream test suite. The new test runs the provided upstream mawk tests (they are not separated out by the upstream test suite so it looks like one test from the autodebtest point of view). This new version of the mawk package has not been published anywhere yet. So, I have appended the (small) patch to this message. Next on my todo list for this project is to try to integrate piuparts somehow. This will mean inventing some machinery for `any package' tests. Should I file the mawk patch, attached, as a wishlist bug against mawk[3] ? It would be nice to be able to integrate the tests for each package as far upstream as possible; at the moment, that probably means putting it in the Debian packages. I would like to have some idea what people think I should do with the tests that we're hopefully going to have, eventually for lots of packages. Would Debian like those tests as patches in wishlist bug reports, in general ? That would seem to be best to me but before I go down this route I'd like to be clear that that's what Debian developers want. Below you can find a demo transcript. Thanks, Ian. [1] The package will be renamed `autopkgtest' in my next upload. We apologise for the inconvenience. [2] You need 0.5.2 which is currently awaiting archive processing for both sid and dapper. Currently autodebtest is in dapper universe; the renamed package will be in main RSN. You can find a copy here: [3] I have BCC'd the Debian mawk maintainer. http://www.chiark.greenend.org.uk/~ian/autodebtest-tmp/ liberator:~> dchroot Executing shell in 'dapper' chroot. To execute a command as root, run "sudo <command>" and enter your password. [EMAIL PROTECTED]:~$ cd /work/AutomatedTesting/demo [EMAIL PROTECTED]:/work/AutomatedTesting/demo$ mkdir unpatched [EMAIL PROTECTED]:/work/AutomatedTesting/demo$ cd unpatched/ [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched$ dpkg-source -x /work/download/mawk_1.3.3-11.dsc dpkg-source: extracting mawk in mawk-1.3.3 dpkg-source: unpacking mawk_1.3.3.orig.tar.gz dpkg-source: applying /work/download/mawk_1.3.3-11.diff.gz [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched$ cd mawk-1.3.3/ [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ debian/rules build >../build-log 2>&1 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ echo $? 0 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ fakeroot debian/rules binary >../binary-log 2>&1 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ echo $? 0 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched/mawk-1.3.3$ cd ../.. [EMAIL PROTECTED]:/work/AutomatedTesting/demo$ mkdir patched [EMAIL PROTECTED]:/work/AutomatedTesting/demo$ cd patched/ [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched$ dpkg-source -x /work/download/mawk_1.3.3-11.dsc dpkg-source: extracting mawk in mawk-1.3.3 dpkg-source: unpacking mawk_1.3.3.orig.tar.gz dpkg-source: applying /work/download/mawk_1.3.3-11.diff.gz [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched$ cd mawk-1.3.3/ [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ patch -p1 -s </work/AutomatedTesting/mawk-1.3.3.adt.diff [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ debian/rules build >../build-log 2>&1 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ echo $? 0 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ fakeroot debian/rules binary >../binary-log 2>&1 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ echo $? 0 [EMAIL PROTECTED]:/work/AutomatedTesting/demo/patched/mawk-1.3.3$ logout liberator:~> really dchroot Executing shell in 'dapper' chroot. [EMAIL PROTECTED]:~# cd /work/AutomatedTesting/demo/unpatched/ [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched# dpkg -i mawk_1.3.3-11_i386.deb (Reading database ... 70650 files and directories currently installed.) Preparing to replace mawk 1.3.3-11 (using mawk_1.3.3-11_i386.deb) ... Unpacking replacement mawk ... Setting up mawk (1.3.3-11) ... [EMAIL PROTECTED]:/work/AutomatedTesting/demo/unpatched# logout liberator:~> adt-run --output-dir ~/d --build-tree-tb /work/AutomatedTesting/demo/patched/mawk-1.3.3 --- adt-virt-chroot =dapper mawktest PASS liberator:~> cat d/stderr-mawktest liberator:~> cat d/stdout-mawktest mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan testing input and field splitting input and field splitting OK testing regular expression matching regular expression matching OK testing checking for write errors checking for write errors OK testing arrays and flow of control array test OK testing function calls and general stress test general stress test passed tested mawk seems OK liberator:~> really dchroot su - (dapper) su - [EMAIL PROTECTED]:~# dpkg --purge --force-depends mawk dpkg: mawk: dependency problems, but removing anyway as you request: ubuntu-minimal depends on mawk. base-files depends on awk; however: Package awk is not installed. Package mawk which provides awk is to be removed. (Reading database ... 70649 files and directories currently installed.) Removing mawk ... [EMAIL PROTECTED]:~# logout liberator:~> adt-run --output-dir ~/d --build-tree-tb /work/AutomatedTesting/demo/patched/mawk-1.3.3 --- adt-virt-chroot =dapper mawktest FAIL stderr: /tmp/tmp.9omUyv/mawktest: line 24: mawk:... liberator:~> cat d/stderr-mawktest /tmp/tmp.9omUyv/mawktest: line 24: mawk: command not found liberator:~> diff --exclude 'parse.[ch]' -ruN orig/mawk-1.3.3/debian/tests/control mawk-1.3.3/debian/tests/control --- orig/mawk-1.3.3/debian/tests/control 1970-01-01 01:00:00.000000000 +0100 +++ mawk-1.3.3/debian/tests/control 2006-01-31 15:25:36.081499137 +0000 @@ -0,0 +1,2 @@ +Tests: mawktest +Restrictions: rw-build-tree diff --exclude 'parse.[ch]' -ruN orig/mawk-1.3.3/debian/tests/mawktest mawk-1.3.3/debian/tests/mawktest --- orig/mawk-1.3.3/debian/tests/mawktest 1970-01-01 01:00:00.000000000 +0100 +++ mawk-1.3.3/debian/tests/mawktest 2006-01-31 15:25:36.081499137 +0000 @@ -0,0 +1,9 @@ +#!/bin/sh +set -e +sed -e ' + /^PATH=/d; + /^mawk -W version/ s/$/ 2>&1/; +' <test/mawktest >"$TMPDIR/mawktest" +chmod +x "$TMPDIR/mawktest" +cd test +"$TMPDIR/mawktest" -- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
