On Thu, Sep 14, 2017 at 1:59 PM, Sean Busbey <bus...@apache.org> wrote:
> > > On 2017-09-14 15:36, Chris Douglas <cdoug...@apache.org> wrote: > > This has gotten bad enough that people are dismissing legitimate test > > failures among the noise. > > > > On Thu, Sep 14, 2017 at 1:20 PM, Allen Wittenauer > > <a...@effectivemachines.com> wrote: > > > Someone should probably invest some time into integrating the > HBase flaky test code a) into Yetus and then b) into Hadoop. > > > > What does the HBase flaky test code do? Another extension to > > test-patch could run all new/modified tests multiple times, and report > > to JIRA if any run fails. > > > > The current HBase stuff segregates untrusted tests by looking through > nightly test runs to find things that fail intermittently. We then don't > include those tests in either nightly or precommit tests. We have a > different job that just runs the untrusted tests and if they start passing > removes them from the list. > > There's also a project getting used by SOLR called "BeastIT" that goes > through running parallel copies of a given test a large number of times to > reveal flaky tests. > > Getting either/both of those into Yetus and used here would be a huge > improvement. > > I discussed this on yetus-dev a while back and Allen thought it'd be non-trivial: https://lists.apache.org/thread.html/552ad614d1b3d5226a656b60c0108457bcaa1219fb9ad985f8750ba1@%3Cdev.yetus.apache.org%3E I unfortunately don't have the test-patch.sh expertise to dig into this. --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
