One thing which could be done by each author of a patch is to do a search on the HiveQA test failures. If there are flaky tests which fail on the HiveQA run may be should create a JIRA for each of them and link it to the JIRA. The JIRA could have useful information like stacktraces or attaching the logs pertaining to the test run. If there is already an existing JIRA for that flaky test we should still link it. This way we can prioritize the flaky tests which has been linked a lot of times.
On Mon, Mar 13, 2017 at 11:40 AM, Siddharth Seth <ss...@apache.org> wrote: > There's far too many flaky tests to enforce a clean test run prior to > commit. > However, there also needs to be a very conscious effort to fix flaky tests > - otherwise, we're not going to reach the state where this can be enforced. > I'm also not sure whether this is just flaky tests, or whether there are > real problems. Maybe a few volunteers could drive the test fixes, and as an > artifact of this, come up with guidelines on writing tests (avoid stats > comparisons in qfiles, skip lines which may be environment specific, > introduce a new compare mechanism for lines which may be in a different > order based on internal structures, etc). > > On Thu, Mar 9, 2017 at 2:39 PM, Ashutosh Chauhan <hashut...@apache.org> > wrote: > > > I went through all the builds of last 24 hours and though there are > couple > > of green runs, we still have quite a bit of flakiness in our tests. Most > of > > that is captured via jiras on : > > https://issues.apache.org/jira/browse/HIVE-15058 Till we have that > > flakiness I am not sure if we can enforce no commit on test failures > > policy. Once flakiness is straightened out then this discussion will > become > > moot anyways. > > Towards that I think this flaky detector job is a step in right > direction. > > > > > > On Thu, Mar 9, 2017 at 11:38 AM, Sergio Pena <sergio.p...@cloudera.com> > > wrote: > > > > > - Probably avoiding committing a patch if a flaky test is shown on the > > test > > > results? > > > - Should we add a jenkins job that checks for flaky tests like the > hbase > > > project did? > > > https://builds.apache.org/view/H-L/view/HBase/job/HBase- > > Find-Flaky-Tests/ > > > > > > On Thu, Mar 9, 2017 at 10:21 AM, Ashutosh Chauhan < > hashut...@apache.org> > > > wrote: > > > > > > > Great news! Thanks to everyone who contributed in getting our tests > and > > > > test infra sorted out. > > > > We would definitely want to keep the status either green or blue > > > definitely > > > > not red :) All our previous efforts in keeping builds green didn't > bear > > > > fruit. > > > > So, I think we need to make some changes here. > > > > > > > > Any ideas what we can do to ensure green builds going forward? > > > > > > > > Thanks, > > > > Ashutosh > > > > > > > > On Thu, Mar 9, 2017 at 8:07 AM, Sergio Pena < > sergio.p...@cloudera.com> > > > > wrote: > > > > > > > > > It's actually blue Peter :). > > > > > > > > > > But good job, I see that the console output is: > > > > > > > > > > {color:red}ERROR:{color} -1 due to no test(s) being added or > > modified. > > > > > > > > > > {color:green}SUCCESS:{color} +1 due to 10336 tests passed > > > > > > > > > > > > > > > On Thu, Mar 9, 2017 at 8:12 AM, Peter Vary <pv...@cloudera.com> > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Congratulations for everyone who have helped taking care of the > > unit > > > > test > > > > > > failures! > > > > > > I have got my first green run! :) > > > > > > > > > > > > If any of you interested in: > > > > > > https://builds.apache.org/job/PreCommit-HIVE-Build/4049/ > > testReport/ > > > < > > > > > > https://builds.apache.org/job/PreCommit-HIVE-Build/4049/ > > testReport/> > > > > :) > > > > > > > > > > > > Great day, and again thanks everyone! > > > > > > > > > > > > Peter > > > > > > > > > > > > > > >
