We should also do that. But that alone doesn't give us an idea of which tests are flaking the most.
On Wed, Oct 18, 2017 at 2:30 PM, Sijie Guo <guosi...@gmail.com> wrote: > some of the tests were already marked @FlakyTest. > https://github.com/apache/bookkeeper/issues?q=flaky+is%3Aopen > > another idea is to have a issue label `FlakyTest`, create an issue when you > see a flake and label it as `FlakyTest`. > > - Sijie > > On Wed, Oct 18, 2017 at 1:26 PM, Ivan Kelly <iv...@apache.org> wrote: > >> Hi folks, >> >> We need to get a handle on the flakes we're in CI. I've yet to see a >> build come back green on all runs. The flakes seem to be different >> each time, so it's hard to see which to tackle first. >> >> To this end, I've started a spreadsheet with a league table for flakes. >> https://docs.google.com/spreadsheets/d/1CaYA7b4YlUGev8JqXBP8-Q3L_ >> HAmfwY46FgzIb7Qpss/edit#gid=0 >> >> When you see a flake, and you're sure it's a flake (i.e. you've seen >> the test pass on the same commit locally), either add a line, or >> increment the number if it's already there. >> >> This way we can identify which flakes are hurting us the most, and put >> effort into dealing with them first. >> >> Cheers, >> Ivan >>
