Thanks Colin, I have a draft PR open which I occasionally check on and disable the failing tests, I'll update it and see if it passes.
Thanks, Daniel Scanteianu On Mon, Dec 5, 2022, 18:02 Colin McCabe <cmcc...@apache.org> wrote: > FYI, there was a memory leak that affected some of the tests which was > fixed recently, so hopefully stability will improve a bit. See KAFKA-14433 > for details. > > best, > Colin > > On Thu, Nov 24, 2022, at 12:48, John Roesler wrote: > > Hi Dan, > > > > I’m not sure if there’s a consistently used tag, but I’ve gotten good > > mileage out of just searching for “flaky” or “flaky test” in Jira. > > > > If you’re thinking about filing a ticket for a specific test failure > > you’ve seen, I’ve also usually been able to find out whether there’s > > already a ticket by searching for the test class or method name. > > > > People seem to typically file tickets with “flaky” in the title and > > then the test name. > > > > Thanks again for your interest in improving the situation! > > -John > > > > On Thu, Nov 24, 2022, at 10:08, Dan S wrote: > >> Thanks for the reply John! Is there a jira tag or view or something that > >> can be used to find all the failing tests and maybe even try to fix them > >> (even if fix just means extending a timeout)? > >> > >> > >> > >> On Thu, Nov 24, 2022, 16:03 John Roesler <vvcep...@apache.org> wrote: > >> > >>> Hi Dan, > >>> > >>> Thanks for pointing this out. Flaky tests are a perennial problem. We > >>> knock them out every now and then, but eventually more spring up. > >>> > >>> I’ve had some luck in the past filing Jira tickets for the failing > tests > >>> as they pop up in my PRs. Another thing that seems to motivate people > is to > >>> open a PR to disable the test in question, as you mention. That can be > a > >>> bit aggressive, though, so it wouldn’t be my first suggestion. > >>> > >>> I appreciate you bringing this up. I agree that flaky tests pose a > risk to > >>> the project because it makes it harder to know whether a PR breaks > things > >>> or not. > >>> > >>> Thanks, > >>> John > >>> > >>> On Thu, Nov 24, 2022, at 02:38, Dan S wrote: > >>> > Hello all, > >>> > > >>> > I've had a pr that has been open for a little over a month (several > >>> > feedback cycles happened), and I've never seen a fully passing build > >>> (tests > >>> > in completely different parts of the codebase seemed to fail, often > >>> > timeouts). A cursory look at open PRs seems to indicate that mine is > not > >>> > the only one. I was wondering if there is a place where all the flaky > >>> tests > >>> > are being tracked, and if it makes sense to fix (or at least > temporarily > >>> > disable) them so that confidence in new PRs could be increased. > >>> > > >>> > Thanks, > >>> > > >>> > Dan > >>> >
