Thanks for the reply John! Is there a jira tag or view or something that can be used to find all the failing tests and maybe even try to fix them (even if fix just means extending a timeout)?
On Thu, Nov 24, 2022, 16:03 John Roesler <vvcep...@apache.org> wrote: > Hi Dan, > > Thanks for pointing this out. Flaky tests are a perennial problem. We > knock them out every now and then, but eventually more spring up. > > I’ve had some luck in the past filing Jira tickets for the failing tests > as they pop up in my PRs. Another thing that seems to motivate people is to > open a PR to disable the test in question, as you mention. That can be a > bit aggressive, though, so it wouldn’t be my first suggestion. > > I appreciate you bringing this up. I agree that flaky tests pose a risk to > the project because it makes it harder to know whether a PR breaks things > or not. > > Thanks, > John > > On Thu, Nov 24, 2022, at 02:38, Dan S wrote: > > Hello all, > > > > I've had a pr that has been open for a little over a month (several > > feedback cycles happened), and I've never seen a fully passing build > (tests > > in completely different parts of the codebase seemed to fail, often > > timeouts). A cursory look at open PRs seems to indicate that mine is not > > the only one. I was wondering if there is a place where all the flaky > tests > > are being tracked, and if it makes sense to fix (or at least temporarily > > disable) them so that confidence in new PRs could be increased. > > > > Thanks, > > > > Dan >
