On 2/6/19 9:36 AM, Cornelia Huck wrote:
> On Fri, 1 Feb 2019 19:55:52 -0500
> Cleber Rosa <cr...@redhat.com> wrote:
>
>> The current version of the "check-acceptance" target will only show
>> one line for execution of all tests. That's probably OK if the tests
>> to be run are quick enough and they're always the same.
>>
>> But, there's already one test alone that takes on average ~5 seconds
>> to run, we intend to adapt the list of tests to match the user's build
>> environment (among other choices).
>
> Btw: What are our expectations regarding execution time for tests?
> Especially if we continue adding tests, and architecture-specific tests
> are bound to be slower if run on a foreign architecture via tcg.
>
> Would a make check-acceptance-quick command make sense? ("I only want
> to verify quickly that I didn't break too much, so run the quicker
> tests only, probably only for my host architecture")
>
Yes, it definitely makes sense. Now, let me know if the following also
makes sense to you:
1) Because these tests focus on functional testing, the default
target/shortcut ("make check-acceptance") should run the complete set of
test cases (including the slow ones).
2) Requirements vary greatly from user to user, to while adding a
"check-acceptance-quick" is fine, you just mentioned one extra test
execution variation ("for my host architecture"). For those, the idea
is that:
a) "make check-acceptance[-quick]" will adapt to the build environment
(if you only built s390x targets, that's all it's going to use)
b) "avocado" command line interface *should* be easy enough to fulfill
other requirements, and not necessarily require a "make" target. For
instance, if you're only interested in your host arch and one specific
machine type, a command line such as the following should do the trick:
$ make check-venv
$ ./tests/venv/bin/avocado run -t arch:`uname -m` -t
machine:WHAT_I_CARE_ABOUT tests/acceptance/
How does that sound?
Regards,
- Cleber.
>>
>> Because of that, let's present the default Avocado UI by default.
>> Users can always choose a different output by setting the AVOCADO_SHOW
>> variable.
>>
>> Reviewed-by: Caio Carrara <ccarr...@redhat.com>
>> Reviewed-by: Philippe Mathieu-Daudé <phi...@redhat.com>
>> Reviewed-by: Alex Bennée <alex.ben...@linaro.org>
>> Signed-off-by: Cleber Rosa <cr...@redhat.com>
>> ---
>> .travis.yml | 2 +-
>> tests/Makefile.include | 2 +-
>> 2 files changed, 2 insertions(+), 2 deletions(-)
