Hi Ian, thanks for the input! It does make motivation behind source packages-based testing clearer. And Simon's example is a good one ;-)
As a summary: source packages-based testing often provides more convenient and upstream-friendly approach, thus it must not be "excluded", echoing Stefano's comment that ideally we might want having both (source and binary packages testing). Moreover we might need a simple registry/specification on how already existing packages could expose included test batteries (so echoing back to debian/README.test): e.g. many python modules have tests included and could easily be invoked with 'nosetests module' or 'python -c "import module; module.test()"', so we just need a specification installed on how to run the tests (and possibly common output formats so our backend could pick them up, e.g. nosetests, py.test, ctest, etc). As such, those packages do not need separate binary package, nor source package for testing. or alternatively -- they are already 'testing' binary packages without clear specification on how to invoke the tests and collect the results. > > Unfortunately the core aspect of the current autopkgtest - relying > > on tests in source packages -- imho to be not the ideal solution to > > target both sides of the userbase (i.e. maintainers/QA vs mortals). > I'm not sure I know what you mean. in the core -- users usually do not deal with source packages; many of them do not even have deb-src lines for apt. They do not care how things are built, but if we want them to contribute by testing their systems, the simplest approach is exposing test batteries as binary packages. Of cause, user-friendly front-end might overcome those difficulties even if tests are provided in source packages. Once again, many arguments behind source packages-based testing are sound to me, but I must disagree with many statements in the "bigger overhead" argument: > - Binary packages get entries in a large number of databases both > in our central infrastructure and on users' systems and imho it is ok -- we already have -dbg packages which are also of marginal importance to users, unless they need them, so they get installed them explicitly > - Binary packages are highly visible in ways that test infrastructure > doesn't need to be we (at least me and Michael) do want making them visible -- that is the point. Otherwise they would not be exercised, and be left unknown (e.g. like qa-regression-testing ;-) ) > - They are relatively complicated to produce yes -- unless the opposite ;) i.e. for many packages, indeed, source packages testing seems to be more adequate. > - They can only be installed in particular locations I do not see it much as a disadvantage > - Only one version can be present on a system at once (unless even > more complicated things are done) for the goal of testing current system setup, installing the single, most recent battery, sounds sufficient. To complement there are snapshot.debian.org and backports.debian.org, so any previous or backported version could be made available -- =------------------------------------------------------------------= Keep in touch www.onerussian.com Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110202155039.gb13...@onerussian.com
