When I was emailing this yesterday, I neglected to shout out folks for their help: Calixte suggested the improvements, and the plan benefited from Marco's reflecting on what we were trying to achieve. I recommend reading the thread https://bugzilla.mozilla.org/show_bug.cgi?id=1534280 to follow the evolution of the process.
-- Emma On Mon, Apr 29, 2019 at 3:15 PM Emma Humphries <e...@mozilla.com> wrote: > We are making more changes to how we report and track regressions in > Bugzilla. > > > There are multiple fields and keywords we use to track regressions in > Bugzilla. It’s confusing and the fields are used inconsistently across > teams. > > > We are consolidating these fields to reduce confusion and provide a > consistent way to track these bugs. This change will also improve our > understanding of defects by humans, tools and automation. > > > == The Change > > > You have already seen the first part of this work. We’ve added the > regresses and regressed_by fields to identify the bug associated with the > changeset that caused a regression. That step eliminated the confusion over > if a bug caused a regression or was a dependency. > > Currently we use the regression and regressionwindow-wanted keywords to > indicate regressions and look for the changeset responsible. We also have > the has-regression-range field (which is used less frequently.) > > We’re replacing the regression and regressionwindow-wanted keywords, and > the has-regression-range field with is_regression, which has the values: > > > - > > yes-range-known > - > > yes-range-needed > - > > yes-range-not-needed > - > > unsure > - > > no > - > > '---' > > We have yes-range-not-needed as a value because there are some > long-standing regressions we know of, and also to preserve history on > older, resolved regression bugs for which we may not know the cause or for > which finding the range would be too expensive or difficult to find. > == Usage > > > The default value of is_regression is '---'. > > If you believe a bug is a regression, but don’t know what commit > introduced it, set is_regression to yes-range-needed. > > If you are not sure if a bug is a regression, set is_regression to unsure. > > > Once you’ve found the changeset that introduced the regression, set > is_regression to yes-range-known and set the regressed_by field to the > bug containing the changeset. > > If a bug turns out to not be a regression, set is_regression to no and > make sure regressed_by is empty. > > If you set is_regression to yes-range-not-needed, then you’re asserting > that there’s not a changeset you can revert to fix the regression. > > If you are looking for regressions to find ranges for, search on > is_regression = yes-range-needed. > > Bugs with is_regression = unsure need help confirming if they are > regressions. If you can confirm it’s a regression set the value to > yes-range-needed, or yes-range-known. If not, set the value to no. > > We will notify triage owners through setting a needinfo request via the > Autonag Bot when these fields are in an inconsistent state (see below). > > > == Notes > > > The is_regression field is intended for bugs of type defect. > > We will be updating our UI for bug entry so that when setting a bug > > So that we don’t lose history in the move, we will have to note older, > resolved regression bugs for which we don’t have values for the > regresses/regressed_by fields. If the bug is a regression which was RESOLVED > FIXED or RESOLVED VERIFIED, we will mark it as yes-range-known, and look > into backfilling the field using the information in the blocked_by field. > > These changes were discussed, at length, on Bugzilla ( > https://bugzilla.mozilla.org/show_bug.cgi?id=1534280) > > > == Feedback > > > I'll be reading your feedback on this on this thread, or in the bugzilla > bug and will respond later this week. > > > -- Emma > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
