This may be somewhat long winded. I will make it in the context of Gecko graphics, because that’s where I have the most data and experience. It may or may not apply to other components.
Reviewing all the incoming bugs, in a timely matter, is very important to us. Over the past few years, the graphics team fixed about half the bugs that came in (roughly, we fix a thousand out of the two thousand that come in.) The main goal of any kind of a bug triage process is thus to choose the right half of the bugs to fix, and spend as little time as possible on the rest. With that in mind we started a much less documented and much more minimal process in 2015. I don’t know that we have the data to back the “avoid issues falling between the cracks”, but it certainly seems that way. One data point we do have - the average amount of time it took to fix the bugs triaged in 2015 was almost half of that for 2014 and the previous four years. This to illustrate that I believe in the bug triage, looking at bugs as they come in, and making some quick decisions how to proceed. I’m also a big believer in lightweight processes and minimal documentation, so there are a few comments I have on what the document below describes, and in general. The more states we have, the more not-so-useful conversation we’ll have about assigning those states. I’m glad to see that we have a small number, currently named fix-now, non-urgent and wishlist (the naming is in flux and being argued in the document.) I’m mentally mapping these to “blocking the release”, “can’t ship too many releases with this” and “I hope we eventually fix this”, but perhaps there is a different interpretation. I would expect to see the majority of the graphics bugs end up in the last group. Why? Since you never plan for the full capacity, as that actually reduces your throughput, and since we only fix half the bugs that come in, it stands to reason that we should not have even close to half of the bugs in the first two categories. In other words, we want to be fixing some of the “wishlist” bugs. And we need to be very careful about not making the fix-now turn into “we really should fix this”, and only allow the “team works on nothing else while there are fix-now bugs open”. Otherwise, well, it loses its value. Step aside - my thoughts on the “How Triage Will Work in Bugzilla” section. I would stick with the definition of the states and have dashboards that show the bug counts in different categories for different teams. The current description is a bit too detailed and inflexible. It suggests that we can figure out the best way to do this before we’ve even started doing it (the pilot program non-withstanding), and it mixes the mechanics with the goals. I’m going to start and keep arguing that we do not want to have an explicit name for that largest bucket of “wishlist” bugs, and should instead have it marked by the absence of a tag. This is not about the heavily involved community, those that will stick around regardless of what we do to them, and anybody that reads this e-mail. This is about people that are reporting their first bugs, that are occasionally involved, who are vital to our success. If I was one of those, and I started seeing that most, if not all, of my bugs are marked as wishlist or deferred or in-copious-spare-time, I’m going to get discouraged and stop doing it. After all, I don’t seem to be valuably contributing, because I’m just telling you things you don’t care about. Or, I could start arguing in the bug, that this should be higher priority, and fill up the comments with non-technical information. Getting close to a full page, I’ll stop now. I’m available for live conversations on the topic :) — - Milan > On Mar 29, 2016, at 16:07 , Emma Humphries <e...@mozilla.com> wrote: > > tl;dr > > In Quarter Two I'm implementing the work we’ve been doing to improve > triage, make actionable decisions on new bugs, and prevent us from shipping > regressions in Firefox. > > Today I’m asking for feedback on the plan which is posted at: > > https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko > > Allowing bugs to sit around without a decision on what we will do about > them sends the wrong message to Mozillans about how we treat bugs, how we > value their involvement, and reduces quality. > > The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote, > and Benjamin Smedberg) want to make better assertions about the quality of > our releases by giving you tools to make clear decisions about which bugs > must be fixed for each release (urgent) and actively tracking those bugs. > What We Learned From The Pilot Program > > During the past 6 weeks, we have prototyped and tested a triage process > with the DOM, Hello, and Developer Tools teams. > > Andrew Overholt, who participated in the pilot for the DOM team, said, “A > consistent bug triage process can help us spread the load of watching > incoming bugs and help avoid issues falling through the cracks." > > During the pilot, the DOM team uncovered critical bugs quickly so that > people could be assigned to them. > > The pilot groups also found that the triage process needs to be fast and > have tooling to make going through bugs fast. It’s easy to fall behind on > triage for a component, but if you stay up to date it will take no more > than 15 minutes a day. > > You can find the bugs we triaged during the pilot by looking for whiteboard > tags containing ‘btpp-’. > > It is also important to have consistent, shared definitions for regression > across components so triagers do not waste effort on mis-labeled bugs. > Comments? > > I am posting this plan now for comment over the next week. I intend to > finalize the triage plan for implementation by Tuesday, April 5th. Feedback > and questions are welcome on the document, privately via email or IRC > (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list. > Timeline > > January: finish finding component responsible parties > > February: pilot review of NEW bugs with four groups of components, draft > new process > > Now: comment period for new process, finalize process > > Q2: implement new process across all components involved in shipping Firefox > Q3: all newly triaged bugs following the new process > > -- Emma Humphries, Bugmaster > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
