> -----Original Message----- > From: Daniel Sangorrin on Tuesday, November 21, 2017 12:55 AM > autopkgtest allows running tests included in Debian > source packages. This commits adds support for running > autopkgtest on the target (it needs to be installed) > and parse the results. > > One important thing to notice is that autopkgtest installs > packaged dependencies (testbed) and therefore leaves the > board's Debian in a different state. For that reason, > it is recommended that an OS deploy phase (e.g. using LAVA) > is executed before the test.
Longterm it would be nice to have a setup and cleanup for this test, that detected if autopkgtest needed to be installed, and did so, and kept a record of installed packages before the test. And then after the test removed all packages that were installed before the test. > > The spec currently contains very few tests but there > are many more available: https://ci.debian.net/data/packages/ > > The patch and test specs were tested on BeagleBone Black > using bone-debian-9.2-iot-armhf-2017-10-10-4gb.img.xz I tried to test this here, but my images are too old. See one comment below, but otherwise it's applied as is. > > Signed-off-by: Daniel Sangorrin <daniel.sangor...@toshiba.co.jp> > --- > engine/tests/Functional.autopkgtest/fuego_test.sh | 19 > +++++++++++++++++ > engine/tests/Functional.autopkgtest/parser.py | 25 > +++++++++++++++++++++++ > engine/tests/Functional.autopkgtest/spec.json | 11 ++++++++++ > 3 files changed, 55 insertions(+) > create mode 100755 engine/tests/Functional.autopkgtest/fuego_test.sh > create mode 100755 engine/tests/Functional.autopkgtest/parser.py > create mode 100644 engine/tests/Functional.autopkgtest/spec.json > > diff --git a/engine/tests/Functional.autopkgtest/fuego_test.sh > b/engine/tests/Functional.autopkgtest/fuego_test.sh > new file mode 100755 > index 0000000..5aac549 > --- /dev/null > +++ b/engine/tests/Functional.autopkgtest/fuego_test.sh > @@ -0,0 +1,19 @@ > +function test_pre_check { > + assert_define FUNCTIONAL_AUTOPKGTEST_PACKAGES > + is_on_target autopkgtest AUTOPKGTEST_PROGRAM /usr/bin > + assert_define AUTOPKGTEST_PROGRAM The "standard" (recently adopted), is to put the static portion of the test variable name at the beginning. So I changed this to PROGRAM_AUTOPKGTEST. > +} > + > +function test_run { > + report "echo 'Running tests for these packages: > $FUNCTIONAL_AUTOPKGTEST_PACKAGES'" > + for PACKAGE in $FUNCTIONAL_AUTOPKGTEST_PACKAGES; do > + report_append "cd $BOARD_TESTDIR/fuego.$TESTDIR; mkdir > $PACKAGE; cd $PACKAGE; \ > + apt-get source $PACKAGE; cd $PACKAGE-*; autopkgtest --output-dir > ./output-dir-$PACKAGE -B . -- null; \ > + echo 'Fuego test_set: $PACKAGE' >> > $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt; \ > + cat ./output-dir-$PACKAGE/summary >> > $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt" > + done > +} > + > +function test_fetch_results { > + get $BOARD_TESTDIR/fuego.$TESTDIR/complete-summary.txt > $LOGDIR/summary.txt > +} > diff --git a/engine/tests/Functional.autopkgtest/parser.py > b/engine/tests/Functional.autopkgtest/parser.py > new file mode 100755 > index 0000000..98a2024 > --- /dev/null > +++ b/engine/tests/Functional.autopkgtest/parser.py > @@ -0,0 +1,25 @@ > +#!/bin/python > + > +import os, re, sys > + > +sys.path.insert(0, os.environ['FUEGO_CORE'] + '/engine/scripts/parser') > +import common as plib > + > +AUTOPKGTST_SUMMARY = plib.LOGDIR + "/summary.txt" > + > +print "Parsing: " + AUTOPKGTST_SUMMARY > + > +results = {} > +with open(AUTOPKGTST_SUMMARY) as f: > + for line in f: > + print line.strip() > + if line.startswith("Fuego test_set:"): > + test_set = line.split()[2] > + continue > + if line.strip(): > + print line.strip() > + fields = line.split() > + test_case = fields[0] > + results[test_set+'.'+test_case] = "PASS" if fields[1] == 'PASS' > else > "FAIL" > + > +sys.exit(plib.process(results)) > diff --git a/engine/tests/Functional.autopkgtest/spec.json > b/engine/tests/Functional.autopkgtest/spec.json > new file mode 100644 > index 0000000..afa8653 > --- /dev/null > +++ b/engine/tests/Functional.autopkgtest/spec.json > @@ -0,0 +1,11 @@ > +{ > + "testName": "Functional.autopkgtest", > + "specs": { > + "smoketest": { > + "packages": "fakesleep" > + }, > + "default": { > + "packages": "libzerg libproxy fakesleep" > + } > + } > +} > -- > 2.7.4 > > > _______________________________________________ > Fuego mailing list > fu...@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/fuego
