Re: [DISCUSS] Releasable trunk and quality
To checkpoint this conversation and keep it going, the ideas I see in-thread (light editorializing by me): 1. Blocking PR merge on CI being green (viable for single branch commits, less so for multiple) 2. A change in our expected culture of "if you see something, fix something" when it comes to test failures on a branch (requires stable green test board to be viable) 3. Clearer merge criteria and potentially updates to circle config for committers in terms of "which test suites need to be run" (notably, including upgrade tests) 4. Integration of model and property based fuzz testing into the release qualification pipeline at least 5. Improvements in project dependency management, most notably in-jvm dtest API's, and the release process around that So a) Am I missing anything, and b) Am I getting anything wrong in the summary above? On Thu, Nov 4, 2021 at 9:01 AM Andrés de la Peña wrote: > Hi all, > > we already have a way to confirm flakiness on circle by running the test > > repeatedly N times. Like 100 or 500. That has proven to work very well > > so far, at least for me. #collaborating #justfyi > > > I think it would be helpful if we always ran the repeated test jobs at > CircleCI when we add a new test or modify an existing one. Running those > jobs, when applicable, could be a requirement before committing. This > wouldn't help us when the changes affect many different tests or we are not > able to identify the tests affected by our changes, but I think it could > have prevented many of the recently fixed flakies. > > > On Thu, 4 Nov 2021 at 12:24, Joshua McKenzie wrote: > > > > > > > we noticed CI going from a > > > steady 3-ish failures to many and it's getting fixed. So we're moving > in > > > the right direction imo. > > > > > An observation about this: there's tooling and technology widely in use > to > > help prevent ever getting into this state (to Benedict's point: blocking > > merge on CI failure, or nightly tests and reverting regression commits, > > etc). I think there's significant time and energy savings for us in using > > automation to be proactive about the quality of our test boards rather > than > > reactive. > > > > I 100% agree that it's heartening to see that the quality of the codebase > > is improving as is the discipline / attentiveness of our collective > > culture. That said, I believe we still have a pretty fragile system when > it > > comes to test failure accumulation. > > > > On Thu, Nov 4, 2021 at 2:46 AM Berenguer Blasi > > > wrote: > > > > > I agree with David. CI has been pretty reliable besides the random > > > jenkins going down or timeout. The same 3 or 4 tests were the only > flaky > > > ones in jenkins and Circle was very green. I bisected a couple failures > > > to legit code errors, David is fixing some more, others have as well, > etc > > > > > > It is good news imo as we're just getting to learn our CI post 4.0 is > > > reliable and we need to start treating it as so and paying attention to > > > it's reports. Not perfect but reliable enough it would have prevented > > > those bugs getting merged. > > > > > > In fact we're having this conversation bc we noticed CI going from a > > > steady 3-ish failures to many and it's getting fixed. So we're moving > in > > > the right direction imo. > > > > > > On 3/11/21 19:25, David Capwell wrote: > > > >> It’s hard to gate commit on a clean CI run when there’s flaky tests > > > > I agree, this is also why so much effort was done in 4.0 release to > > > remove as much as possible. Just over 1 month ago we were not really > > > having a flaky test issue (outside of the sporadic timeout issues; my > > > circle ci runs were green constantly), and now the “flaky tests” I see > > are > > > all actual bugs (been root causing 2 out of the 3 I reported) and some > > (not > > > all) of the flakyness was triggered by recent changes in the past > month. > > > > > > > > Right now people do not believe the failing test is caused by their > > > patch and attribute to flakiness, which then causes the builds to start > > > being flaky, which then leads to a different author coming to fix the > > > issue; this behavior is what I would love to see go away. If we find a > > > flaky test, we should do the following > > > > > > > > 1) has it already been reported and who is working to fix? Can we > > block > > > this patch on the test being fixed? Flaky tests due to timing issues > > > normally are resolved very quickly, real bugs take longer. > > > > 2) if not reported, why? If you are the first to see this issue than > > > good chance the patch caused the issue so should root cause. If you > are > > > not the first to see it, why did others not report it (we tend to be > good > > > about this, even to the point Brandon has to mark the new tickets as > > dups…)? > > > > > > > > I have committed when there were flakiness, and I have caused > > flakiness; > > > not saying I am perfect or that I do the above, just saying that if
Welcome Sumanth Pasupuleti as Apache Cassandra Committer
The PMC members are pleased to announce that Sumanth Pasupuleti has recently accepted the invitation to become committer. Sumanth, thank you for all your contributions to the project over the years. Congratulations and welcome! The Apache Cassandra PMC members
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations, Sumanth! > On Nov 5, 2021, at 11:17 AM, Oleksandr Petrov > wrote: > > The PMC members are pleased to announce that Sumanth Pasupuleti has > recently accepted the invitation to become committer. > > Sumanth, thank you for all your contributions to the project over the years. > > Congratulations and welcome! > > The Apache Cassandra PMC members - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations Sumanth! On Fri, 5 Nov 2021 at 14:22, wrote: > Congratulations, Sumanth! > > > On Nov 5, 2021, at 11:17 AM, Oleksandr Petrov < > oleksandr.pet...@gmail.com> wrote: > > > > The PMC members are pleased to announce that Sumanth Pasupuleti has > > recently accepted the invitation to become committer. > > > > Sumanth, thank you for all your contributions to the project over the > years. > > > > Congratulations and welcome! > > > > The Apache Cassandra PMC members > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Well deserved, congratulations Sumanth! Em sex., 5 de nov. de 2021 às 15:32, Ekaterina Dimitrova < e.dimitr...@gmail.com> escreveu: > Congratulations Sumanth! > > On Fri, 5 Nov 2021 at 14:22, wrote: > > > Congratulations, Sumanth! > > > > > On Nov 5, 2021, at 11:17 AM, Oleksandr Petrov < > > oleksandr.pet...@gmail.com> wrote: > > > > > > The PMC members are pleased to announce that Sumanth Pasupuleti has > > > recently accepted the invitation to become committer. > > > > > > Sumanth, thank you for all your contributions to the project over the > > years. > > > > > > Congratulations and welcome! > > > > > > The Apache Cassandra PMC members > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > >
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations Sumanth! On Fri, Nov 5, 2021 at 1:17 PM Oleksandr Petrov wrote: > > The PMC members are pleased to announce that Sumanth Pasupuleti has > recently accepted the invitation to become committer. > > Sumanth, thank you for all your contributions to the project over the years. > > Congratulations and welcome! > > The Apache Cassandra PMC members - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Great to see this. Congrats Sumanth! On Fri, Nov 5, 2021 at 11:34 AM Brandon Williams wrote: > Congratulations Sumanth! > > On Fri, Nov 5, 2021 at 1:17 PM Oleksandr Petrov > wrote: > > > > The PMC members are pleased to announce that Sumanth Pasupuleti has > > recently accepted the invitation to become committer. > > > > Sumanth, thank you for all your contributions to the project over the > years. > > > > Congratulations and welcome! > > > > The Apache Cassandra PMC members > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations Sumanth! - Yifan > On Nov 5, 2021, at 11:37 AM, Patrick McFadin wrote: > > Great to see this. Congrats Sumanth! > >> On Fri, Nov 5, 2021 at 11:34 AM Brandon Williams wrote: >> >> Congratulations Sumanth! >> >>> On Fri, Nov 5, 2021 at 1:17 PM Oleksandr Petrov >>> wrote: >>> >>> The PMC members are pleased to announce that Sumanth Pasupuleti has >>> recently accepted the invitation to become committer. >>> >>> Sumanth, thank you for all your contributions to the project over the >> years. >>> >>> Congratulations and welcome! >>> >>> The Apache Cassandra PMC members >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: dev-h...@cassandra.apache.org >> >> - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congrats! > On Nov 5, 2021, at 1:52 PM, Yifan Cai wrote: > > Congratulations Sumanth! > > - Yifan > >> On Nov 5, 2021, at 11:37 AM, Patrick McFadin wrote: >> >> Great to see this. Congrats Sumanth! >> On Fri, Nov 5, 2021 at 11:34 AM Brandon Williams wrote: >>> >>> Congratulations Sumanth! >>> On Fri, Nov 5, 2021 at 1:17 PM Oleksandr Petrov wrote: The PMC members are pleased to announce that Sumanth Pasupuleti has recently accepted the invitation to become committer. Sumanth, thank you for all your contributions to the project over the >>> years. Congratulations and welcome! The Apache Cassandra PMC members >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >>> For additional commands, e-mail: dev-h...@cassandra.apache.org >>> >>> > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations Sumanth!! Well deserved. Thanks, Vinay Chella On Fri, Nov 5, 2021 at 11:58 AM J. D. Jordan wrote: > Congrats! > > > On Nov 5, 2021, at 1:52 PM, Yifan Cai wrote: > > > > Congratulations Sumanth! > > > > - Yifan > > > >> On Nov 5, 2021, at 11:37 AM, Patrick McFadin > wrote: > >> > >> Great to see this. Congrats Sumanth! > >> > On Fri, Nov 5, 2021 at 11:34 AM Brandon Williams > wrote: > >>> > >>> Congratulations Sumanth! > >>> > On Fri, Nov 5, 2021 at 1:17 PM Oleksandr Petrov > wrote: > > The PMC members are pleased to announce that Sumanth Pasupuleti has > recently accepted the invitation to become committer. > > Sumanth, thank you for all your contributions to the project over the > >>> years. > > Congratulations and welcome! > > The Apache Cassandra PMC members > >>> > >>> - > >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >>> For additional commands, e-mail: dev-h...@cassandra.apache.org > >>> > >>> > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations Sumanth! Well deserved!! -Joey On Fri, Nov 5, 2021 at 11:17 AM Oleksandr Petrov wrote: > > The PMC members are pleased to announce that Sumanth Pasupuleti has > recently accepted the invitation to become committer. > > Sumanth, thank you for all your contributions to the project over the years. > > Congratulations and welcome! > > The Apache Cassandra PMC members - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Welcome! On Fri, 5 Nov 2021 at 20:11, Joseph Lynch wrote: > > Congratulations Sumanth! > > Well deserved!! > > -Joey > > On Fri, Nov 5, 2021 at 11:17 AM Oleksandr Petrov > wrote: > > > > The PMC members are pleased to announce that Sumanth Pasupuleti has > > recently accepted the invitation to become committer. > > > > Sumanth, thank you for all your contributions to the project over the years. > > > > Congratulations and welcome! > > > > The Apache Cassandra PMC members > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Sumanth Pasupuleti as Apache Cassandra Committer
Congratulations Sumanth! On Fri, Nov 5, 2021 at 12:17 PM Stefan Miklosovic < stefan.mikloso...@instaclustr.com> wrote: > Welcome! > > On Fri, 5 Nov 2021 at 20:11, Joseph Lynch wrote: > > > > Congratulations Sumanth! > > > > Well deserved!! > > > > -Joey > > > > On Fri, Nov 5, 2021 at 11:17 AM Oleksandr Petrov > > wrote: > > > > > > The PMC members are pleased to announce that Sumanth Pasupuleti has > > > recently accepted the invitation to become committer. > > > > > > Sumanth, thank you for all your contributions to the project over the > years. > > > > > > Congratulations and welcome! > > > > > > The Apache Cassandra PMC members > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
