Re: [Qemu-devel] [PATCH 1/2] Acceptance tests: exclude "flaky" tests

2019-07-05 Thread Wainer dos Santos Moschetta
On 06/30/2019 02:51 PM, Cleber Rosa wrote: On Fri, Jun 28, 2019 at 05:43:09PM -0300, Wainer dos Santos Moschetta wrote: On 06/21/2019 11:38 AM, Cleber Rosa wrote: On Fri, Jun 21, 2019 at 09:03:33AM +0200, Philippe Mathieu-Daudé wrote: On 6/21/19 8:09 AM, Cleber Rosa wrote: It's a fact that

Re: [Qemu-devel] [PATCH 1/2] Acceptance tests: exclude "flaky" tests

2019-06-30 Thread Cleber Rosa
On Fri, Jun 28, 2019 at 05:43:09PM -0300, Wainer dos Santos Moschetta wrote: > > On 06/21/2019 11:38 AM, Cleber Rosa wrote: > > On Fri, Jun 21, 2019 at 09:03:33AM +0200, Philippe Mathieu-Daudé wrote: > > > On 6/21/19 8:09 AM, Cleber Rosa wrote: > > > > It's a fact that some tests may not be 100% r

Re: [Qemu-devel] [PATCH 1/2] Acceptance tests: exclude "flaky" tests

2019-06-28 Thread Wainer dos Santos Moschetta
On 06/21/2019 11:38 AM, Cleber Rosa wrote: On Fri, Jun 21, 2019 at 09:03:33AM +0200, Philippe Mathieu-Daudé wrote: On 6/21/19 8:09 AM, Cleber Rosa wrote: It's a fact that some tests may not be 100% reliable in all environments. While it's a tough call to remove a useful test that from the tr

Re: [Qemu-devel] [PATCH 1/2] Acceptance tests: exclude "flaky" tests

2019-06-21 Thread Cleber Rosa
On Fri, Jun 21, 2019 at 09:03:33AM +0200, Philippe Mathieu-Daudé wrote: > On 6/21/19 8:09 AM, Cleber Rosa wrote: > > It's a fact that some tests may not be 100% reliable in all > > environments. While it's a tough call to remove a useful test that > > from the tree because it may fail every 1/100t

Re: [Qemu-devel] [PATCH 1/2] Acceptance tests: exclude "flaky" tests

2019-06-21 Thread Philippe Mathieu-Daudé
On 6/21/19 8:09 AM, Cleber Rosa wrote: > It's a fact that some tests may not be 100% reliable in all > environments. While it's a tough call to remove a useful test that > from the tree because it may fail every 1/100th time (or so), having > human attention drawn to known issues is very bad for h

[Qemu-devel] [PATCH 1/2] Acceptance tests: exclude "flaky" tests

2019-06-20 Thread Cleber Rosa
It's a fact that some tests may not be 100% reliable in all environments. While it's a tough call to remove a useful test that from the tree because it may fail every 1/100th time (or so), having human attention drawn to known issues is very bad for humans and for the projects they manage. As a c