On 23.08.2017 09:16, Cornelia Huck wrote:
> On Wed, 23 Aug 2017 10:29:07 +1000
> David Gibson <da...@gibson.dropbear.id.au> wrote:
>
>> On Tue, Aug 22, 2017 at 01:48:15PM +0200, Cornelia Huck wrote:
>>> On Tue, 22 Aug 2017 21:20:46 +1000
>>> David Gibson <da...@gibson.dropbear.id.au> wrote:
>>>
>>>> Obviously it's not a thing to fix right now, but I've really been
>>>> thinking that none of the tests should use this "TCG or KVM" stuff.
>>>> They should instead be run with *both* options - or at least the ones
>>>> that are available on the host.
>>>
>>> Having one test as a 'smoke test' that is run for everything available
>>> sounds like a good idea, and the boot-serial test may be a good
>>> candidate for that.
>>>
>>> I would not want to run every test with every accelerator, however, as
>>> this makes 'make check' even slower than it is now. (Although it may be
>>> useful to be able to trigger 'run everything' tests on some dedicated
>>> test machines.)
>>
>> I'd be fine with only running the full matrix on a "make check-harder"
>> or whatever, target. But I'd like the option to be there. Sometimes
>> (like when preparing a pull request) a slower check is an acceptable
>> cost for better coverage.
>
> make check with one smoke test + make check-harder (like the name :)
> sounds like a good combination.
While we're at it: I'd like to have a "make check-fast", too. Sometimes
the normal "make check" is already too slow, e.g. while developing new
patches, I sometimes just want to do a very quick sanity test to see
whether I broke some basic things or not, and only do the "make check"
before I submit my patches.
So we would get three stages:
- make check-fast => For very, very quick sanity tests only
- make check => E.g. has to be run before submitting patches
- make check-harder => might run a very long time, so best suited for
nightly regression tests etc.?
Does that sound reasonable? And the crucial question: Who is going to
implement the basic framework for this?
Thomas
