Thanks for bringing up these valuable points, Mick! In fact we focused on the quality epic so far but there is a lot more stuff unaddressed. I commented some of the points you brought up below:
> How will we ensure this QA persists, so it's not a manual checklist every release? This is a great question but I believe it warrants a separate discussion as part of a larger discussion on improving our development/quality process post-4.0. > *** CASSANDRA-15234 – Standardise config and JVM parameters - It looks like we have dropped the ball on this. It's sad that we dropped the ball on this important change but now I think it's too late to make these changes as it will bring entropy towards stabilizing 4.0. In that sense I think we should postpone this to the next major and prioritize it earlier in the next cycle. > Do all remaining flakey and failing units and dtests have jira tickets entered for 4.0-beta? Has the same been done, at least with rough grouping, for the upgrade tests? Are these tied to the testing epics in any way? Not that I know of. Perhaps we should add a new ticket to the quality epic to track flakey and failing tests? (@cc Josh/Jordan) > Has any triage efforts happened here? Not that I know of but maybe Josh/Jordan/Jon (J^3) are planning on looking at it. I can take a stab at triaging some of these tickets. > Do triaged bugs in this list get moved to fix version "4.x" ? I think in the spirit of expediting 4.0RC release we should mark bugs with low severity (ie. those with a simple workaround) to 4.0.1. Any bug with medium-high severity should be marked as 4.0-rc to favor stability. > Are we duplicating efforts in the testing epics when others have already identified and reported the bugs but we just haven't triage them? That's a good point. I think as part of the triaging effort we should link the bugs to existing quality epics so we can keep track of them. Em ter., 29 de set. de 2020 às 06:11, Sam Tunnicliffe <s...@beobal.com> escreveu: > > > > On 29 Sep 2020, at 09:50, Mick Semb Wever <m...@apache.org> wrote: > > > >> Regarding the proposed agenda of going through the unassigned issues to > >> improve visibility on what needs to be done to ship 4.0 GA I think this > is > >> a great start but only covers part of the problem. > >> > >> I think we have 3 outstanding issues that are hampering visibility of > 4.0 > >> progress: > >> a) Quality testing issues with no shepherd; > >> b) Quality testing issues with shepherd, but no recent activity (~2 > months > >> or less); > >> c) Quality testing issues with no objective acceptance > criteria/Definition > >> of Done; > > > > > > These Quality testing epics are a great focal point. How will we ensure > > this QA persists, so it's not a manual checklist every release? > > > > The following is what I can see outstanding for the 4.0 release, that is > > not afaik attached to these epic tickets… > > > > ** Those issues that slipped alpha… > > *** CASSANDRA-15299 – CASSANDRA-13304 follow-up: improve checksumming > > and compression in protocol v5-beta > > *** CASSANDRA-15234 – Standardise config and JVM parameters > > *** CASSANDRA-13701 – Lower default num_tokens (blocked by > > 'CASSANDRA-16079 Improve dtest runtime' ) > > ** 95 jira tickets in 4.0-beta and 4.0-rc > > ** 631 jira bug tickets with no assigned "fix version" > > ** Remaining flakey unit and dtests > > ** Hundreds of failing and flakey upgrade dtests > > ** Reports from driver tests, and other external test systems > > ** Reports and/or integration with Fallout and Harry > > > > > > In a bit more detail… > > > > > > *** CASSANDRA-15299 – CASSANDRA-13304 follow-up: improve checksumming and > > compression in protocol v5-beta > > > > This looks like it is in its final patch and review. Is that correct Sam? > > > > Yes it is. I hope to get review finished and post some further perf > numbers this week. > > > > > *** CASSANDRA-15234 – Standardise config and JVM parameters > > > > It looks like we have dropped the ball on this. > > > > > > *** CASSANDRA-13701 – Lower default num_tokens, and CASSANDRA-16079 > > > > Some effort is undergoing from Ekaterina, David, and myself. > > I've put together a prototype for caching bootstrapped ccm clusters, but > > i'm not yet sure I can get much savings over the current tests and only a > > minimal saving off the 13701 patch. Berenguer brought up that 40% of the > > dtests are single-node, their performance not changed by 13701, and > > probably better off rewritten to in-jvm tests. > > > > > > ** 95 jira tickets in 4.0-beta and 4.0-rc > > ** Remaining flakey unit and dtests > > ** Hundreds of failing and flakey upgrade dtests > > > > Do all remaining flakey and failing units and dtests have jira tickets > > entered for 4.0-beta? > > Has the same been done, at least with rough grouping, for the upgrade > tests? > > > > Are these tied to the testing epics in any way? > > > > > > ** 631 jira bug tickets with no assigned "fix version" (who knows how > many > > of these are applicable to 4.0?) > > > > Has any triage efforts happened here? > > Do triaged bugs in this list get moved to fix version "4.x" ? > > Are we duplicating efforts in the testing epics when others have already > > identified and reported the bugs but we just haven't triage them? > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
