On 17 July 2017 at 11:20, Daniel P. Berrange <berra...@redhat.com> wrote: > I'm a little concerned about the fact that we've now got three different > sets of tests that are being run on pull requests. There are the tests > that Peter runs on various combinations at time of merge, the tests run > by patchw at time of submissions, and the tests run by travis after > merge. Each of them is covering a different set of scenarios with only > partial overlap between them.
I agree -- I have my ad-hoc test system not because I enjoy collecting random things to run tests on but because we don't have an automated system that does what I need: * wide coverage, focusing on oddball host architectures and OSes * ability to ssh in to a machine where tests have broken if need be * decently fast to complete a test run (travis is terrible for this) Note that "fast to complete a test run for me" is in conflict with "usable by anybody who wants to submit a pull request" :-) thanks -- PMM
