"ML is plaintext bro" - thanks Mick. ಠ_ಠ Since we're stuck in the late 90's, here's some links to a gsheet:
Defects by month: https://docs.google.com/spreadsheets/d/1Qt8lLIiqVvK7mlSML7zsmXdAc-LsvktFW5RXJDRtN8k/edit#gid=1584867240 Defects by component: https://docs.google.com/spreadsheets/d/1Qt8lLIiqVvK7mlSML7zsmXdAc-LsvktFW5RXJDRtN8k/edit#gid=1946109279 Defects by type: https://docs.google.com/spreadsheets/d/1Qt8lLIiqVvK7mlSML7zsmXdAc-LsvktFW5RXJDRtN8k/edit#gid=385136105 On Thu, May 7, 2020 at 12:31 PM Joshua McKenzie <joshua.mcken...@gmail.com> wrote: > Hearing the images got killed by the web server. Trying from gmail (sorry > for spam). Time to see if it's the apache smtp server or the list culling > images: > > ------------------------------------------- > I did a little analysis on this data (any defect marked with fixversion > 4.0 that rose to the level of critical in terms of availability, > correctness, or corruption/loss) and charted some things the rest of the > project community might find interesting: > > 1: Critical (availability, correctness, corruption/loss) defects fixed per > month since about 6 months before 3.11.0: > [image: monthly.png] > > 2: Components in which critical defects arose (note: bright red bar == sum > of 3 dark red): > [image: Total Defects by Component.png] > > 3: Type of defect found and fixed (bright red: cluster down or permaloss, > dark red: temp corrupt/loss, yellow: incorrect response): > > [image: Total Defects by Type.png] > > My personal takeaways from this: a ton of great defect fixing work has > gone into 4.0. I'd love it if we had both code coverage analysis for > testing on the codebase as well as data to surface where hotspots of > defects are in the code that might need further testing (caveat: many have > voiced their skepticism of the value of this type of data in the past in > this project community, so that's probably another conversation to have on > another thread) > > Hope someone else finds the above interesting if not useful. > > -- > Joshua McKenzie > > On Thu, May 7, 2020 at 12:24 PM Joshua McKenzie <jmcken...@apache.org> > wrote: > >> I did a little analysis on this data (any defect marked with fixversion >> 4.0 that rose to the level of critical in terms of availability, >> correctness, or corruption/loss) and charted some things the rest of the >> project community might find interesting: >> >> 1: Critical (availability, correctness, corruption/loss) defects fixed >> per month since about 6 months before 3.11.0: >> [image: monthly.png] >> >> 2: Components in which critical defects arose (note: bright red bar == >> sum of 3 dark red): >> [image: Total Defects by Component.png] >> >> 3: Type of defect found and fixed (bright red: cluster down or permaloss, >> dark red: temp corrupt/loss, yellow: incorrect response): >> >> [image: Total Defects by Type.png] >> >> My personal takeaways from this: a ton of great defect fixing work has >> gone into 4.0. I'd love it if we had both code coverage analysis for >> testing on the codebase as well as data to surface where hotspots of >> defects are in the code that might need further testing (caveat: many have >> voiced their skepticism of the value of this type of data in the past in >> this project community, so that's probably another conversation to have on >> another thread) >> >> Hope someone else finds the above interesting if not useful. >> >> ~Josh >> >> >> On Wed, May 6, 2020 at 3:38 PM Dinesh Joshi <djo...@apache.org> wrote: >> >>> Hi Sankalp, >>> >>> Thanks for bringing this up. At the very minimum, I hope we have >>> regression tests for the specific issues we have fixed. >>> >>> I personally think, the project should focus on building a comprehensive >>> test suite. However, some of these issues can only be detected at scale. We >>> need users to test* C* in their environment for their use-cases. Ideally >>> these folks stand up large clusters and tee their traffic to the new >>> cluster and report issues. >>> >>> If we had an automated test suite that everyone can run at a large scale >>> that would be even better. >>> >>> Thanks, >>> >>> Dinesh >>> >>> >>> * test != starting C* in a few nodes and looking at logs. >>> >>> > On May 6, 2020, at 10:11 AM, sankalp kohli <kohlisank...@gmail.com> >>> wrote: >>> > >>> > Hi, >>> > I want to share some of the serious issues that were found and >>> fixed in >>> > 3.0.x. I have created this list from JIRA to help us identify areas for >>> > validating 4.0. This will also give an insight to the dev community. >>> > >>> > Let us know if anyone has suggestions on how to better use this data in >>> > validating 4.0. Also this list might be missing some issues identified >>> > early on in 3.0.x or some latest ones. >>> > >>> > Link: https://tinyurl.com/30seriousissues >>> > >>> > Thanks, >>> > Sankalp >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org >>> For additional commands, e-mail: dev-h...@cassandra.apache.org >>> >>>
