In DOM triage, we've just set up our list to not include bugs with the keyword intermittent failure. Ideally, we'd see intermittent failures that have a high rate, but given that sheriffs keep an eye on oranges, they can look for somebody to fix frequent ones as needed.
On Fri, Sep 2, 2016 at 11:49 AM, Wes Kocher <kwie...@gmail.com> wrote: > I don't believe that's a workable situation. At the moment, the policy is > that every new intermittent failure gets a bug filed for the purpose of > tracking it. There's talk (and has been for at least a year or two, now) > that work will begin on OrangeFactor version 2, where intermittent failures > could get tracked within OrangeFactor's own database, and we only file bugs > for the failures that get too noisy, but no one has had time to work on > that in the last two years, so would need prioritization from higher ups > for someone(s) to dedicate some time to work on it. > > KW > > From: e...@mozilla.com > Date: Fri, 2 Sep 2016 11:35:26 -0700 > Subject: Re: Reorganization of Firefox-UI tests in mozilla-central > To: m...@hskupin.info > CC: dev-platform@lists.mozilla.org; kwie...@gmail.com > > Henrik, > > It's not clear to me if this is just moving the location of existing > tests, or if you're bringing in new tests. > > If it's the later, I'm concerned that we're going to get more > intermittents filed in Bugzilla, which adds noise to the triage process. > > If you are adding more tests, then would it be possible to back off from > filing intermittent bugs unless they become problematic, failing tests more > than a quarter to half the time? > > Thanks! > > Emma > > On Thu, Sep 1, 2016 at 8:37 AM, Henrik Skupin <m...@hskupin.info> wrote: > Hello, > > > > Via bug 1272145 we want to move our existing Firefox-UI tests from > > /testing/firefox-ui/ closer to the code which these are testing, so that > > the former location only contains the test harness and appropriate unit > > tests. > > > > Link to the current set of tests: > > https://dxr.mozilla.org/mozilla-central/source/testing/firefox-ui/tests > > > > For those of you who haven't heard about these tests yet, they are > > basically Marionette tests with an additional layer (firefox-puppeteer) > > on top to ease the test creation for ui specific checks. Benefits we > > have are restarting and quitting the browser, and running the tests in > > any localized build of Firefox. > > > > As listed below you can find the current type of tests and a proposed > > location: > > > > * locationbar tests: /browser/base/content/test/ > urlbar/firefox_ui/ > > * private browsing tests: > > /browser/components/privatebrowsing/test/firefox_ui/ > > * safe browsing tests: > > /browser/components/safebrowsing/content/test/firefox_ui/ > > * session store tests: /browser/components/ > sessionstore/test/firefox_ui/ > > * update tests: /toolkit/mozapps/update/tests/firefox_ui/ > > > > Do those locations sound good? I have heard at least once that > > "firefox_ui" might not be the best choice as folder name, but that's how > > the harness is called, and corresponds to what we have for other > > harnesses too. > > > > Locations for the following tests are not clear yet: > > > > * l10n tests (shortcuts): not clear yet (maybe under > > /browser/base/content/test/) > > * security tests: not clear yet > > > > L10n related tests mostly cover shortcuts and that the correct command > > is invoked to find failures like bug 1173735. > > > > Nearly all of the security tests are running checks against a real > > server with various SSL certificates (DV, OV, EV) and protocol versions. > > We will have a meeting soon to determine which of those tests are needed > > and which ones can be removed. So we might also be able to find the > > correct location for the tests. > > > > For now I would like to know if the proposal is fine or if we should go > > a completely different route. > > > > Thanks > > > > -- > > Henrik > > _______________________________________________ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > > > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
