On May 22, 2019 11:46 PM, "Cleber Rosa" <cr...@redhat.com> wrote: > > > > ----- Original Message ----- > > From: "Eduardo Habkost" <ehabk...@redhat.com> > > To: "Philippe Mathieu-Daudé" <f4...@amsat.org> > > Cc: qemu-devel@nongnu.org, "Aleksandar Rikalo" <arik...@wavecomp.com>, "Aleksandar Markovic" > > <aleksandar.m.m...@gmail.com>, "Aleksandar Markovic" < amarko...@wavecomp.com>, "Cleber Rosa" <cr...@redhat.com>, > > "Aurelien Jarno" <aurel...@aurel32.net>, "Wainer dos Santos Moschetta" < waine...@redhat.com> > > Sent: Wednesday, May 22, 2019 5:12:30 PM > > Subject: Re: [Qemu-devel] [PATCH 0/4] mips: Add more Avocado tests > > > > On Tue, May 21, 2019 at 01:19:06AM +0200, Philippe Mathieu-Daudé wrote: > > > Hi, > > > > > > It was a rainy week-end here, so I invested it to automatize some > > > of my MIPS tests. > > > > > > The BootLinuxSshTest is not Global warming friendly, it is not > > > meant to run on a CI system but rather on a workstation previous > > > to post a pull request. > > > It can surely be improved, but it is a good starting point. > > > > Until we actually have a mechanism to exclude the test case on > > travis-ci, I will remove patch 4/4 from the queue. Aleksandar, > > please don't merge patch 4/4 yet or it will break travis-ci. > > > > Cleber, Wainer, is it already possible to make "avocado run" skip > > tests tagged with "slow"? > > > > The mechanism exists, but we haven't tagged any test so far as slow. > > Should we define/document a criteria for a test to be slow? Given > that this is highly subjective, we have to think of: > > * Will we consider the average or maximum run time (the timeout > definition)? > > * For a single test, what is "slow"? Some rough numbers from Travis > CI[1] to help us with guidelines: > - boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: PASS (6.04 s) > - boot_linux_console.py:BootLinuxConsole.test_arm_virt: PASS (2.91 s) > - linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16: PASS (18.14 s) > - boot_linux.py:BootLinuxAarch64.test_virt: PASS (396.88 s) > > * Do we want to set a maximum job timeout? This way we can skip > tests after a given amount of time has passed. Currently we interrupt > the test running when the job timeout is reached, but it's possible > to add a option so that no new tests will be started, but currently > running ones will be waited on. >
As far as integrating the patch into my queue, I did it just an hour or so prior the objections of others, but will inform Peter to put the pull request on hold, so it will not go to the main tree. We in Wave Computing (MIPS) are very happy with this test, even in its current state, and understand it as an initial version that will be subject to improvement and expansion. We consider this test seriously and think it is vital for QEMU for MIPS. We would like to put it in the “slow” group for the simple reason that, we gather, this would give us more freedom in future versions. Yours, Aleksandar > Regards, > - Cleber. > > [1] - https://travis-ci.org/clebergnu/qemu/jobs/535967210#L3518 > > > -- > > Eduardo > >
