TL;DR - If you want to run Windows XP tests on try, you need to specify
[Windows XP] after the suite name; they are not longer run by default.
As most people are aware, our Windows test capacity is considerably
overloaded. However, we need to start turning on e10s tests on Windows, so
the e10s tea
I think this is a great idea. Although it won't fix the problem long-term,
what it will do is get engineers and especially engineering managers
thinking about the problem, and hopefully understanding it better so they
can incorporate it into future priorities.
There are two fundamental problems th
The decision not to enforce addon signing on trunk/aurora is not changing.
But, to support running our automation with unsigned addons on
trunk/aurora, but signed addons on beta/release, we would have to implement
some pretty complex logic at the aurora -> beta uplift, and this is
substantial enoug
Engineering Productivity is off to a great start in 2016; here’s what we’ve
been up to in Q1.
Build System
Build system improvements are a major priority for Engineering Productivity
in 2016. The build team made great progress in Q1:
-
Windows builds are now made using VS2015. This shaves
We are planning to decommission git.mozilla.org. It was originally created
to serve the B2G project, but is no longer needed for that purpose, and
other uses have remained slight and don't justify the operational or
maintenance costs.
There is some work underway to migrate existing users to altern
There is no change to https://github.com/mozilla/gecko-dev or related
workflows. Only the git.mozilla.org mirror of this repo is going away.
On Fri, Jul 1, 2016 at 12:40 PM, Nicolas B. Pierron <
nicolas.b.pier...@mozilla.com> wrote:
> On 07/01/2016 06:11 PM, Jonathan Griffin wrote:
&g
Engineering Productivity made great progress on our top-level goals in Q2;
keep reading for a description of the highlights. For a sneak peek of what
we're doing in Q3, see this Google Doc: https://goo.gl/Z2YgQ2
Platform Operations Top-Level Projects
'''Build Faster'''
Overall goal: Reduce build
On this update, we will look at the progress made since our second update.
A reminder that this quarter’s main focus is on:
* Debugging tests on interactive workers (only Linux on TaskCluster)
* Improve end to end times on Try (Thunder Try project)
For all bugs and priorities you can check out t
TL;DR - we'd like some feedback on some UI changes to Treeherder
recommended by a 3rd party UX team.
The longer story - a 3rd party UX team has provided us with some
suggestions on how to improve Treeherder's UI from the perspective of the
average developer. They've created a set of wireframes her
Here’s what Engineering Productivity has accomplished in Q3. In Q4, we’re
switching to OKR’s for tracking goals and progress; these should be
published soon.
== Build System ==
Overall goal: Reduce build times for local developers and automation;
improve maintainability.
Q3 progress:
* TaskCluste
You can't run TPS via tryserver; it isn't run in buildbot at all. It
can't, since it uses live Sync servers.
Raymond, the problem you're experiencing is likely due to changes in
mozprocess/mozrunner API's that TPS hasn't been updated to handle. Can
you file a bug about this, and assign it to
improved. :)
Let me know if this doesn't resolve your issue.
Jonathan
On 4/3/2013 10:31 AM, Jonathan Griffin wrote:
You can't run TPS via tryserver; it isn't run in buildbot at all. It
can't, since it uses live Sync servers.
Raymond, the problem you're experiencing is l
We already have the approximate equivalent of this. It's the
'checkin-needed' keyword.
Add this to your bug, and the sheriffs will land the patch for you,
using the approximate process you describe. The only difference is this
is done out-of-band, so turnaround may take up to 24 hrs.
The a
Splitting the valgrind tests up and running them separately as test jobs
in TBPL is definitely something the A*Team can help with. I've filed
bug 977240 for this.
Jonathan
On 2/25/14 7:25 PM, Nicholas Nethercote wrote:
On Tue, Feb 25, 2014 at 2:32 PM, Mike Hommey wrote:
I never understood
With respect to Autoland, I think we'll need to figure out how to make
it take intermittents into account. I don't think we'll ever be a state
with 0 intermittents.
Jonathan
On 4/4/2014 1:30 PM, Chris Peterson wrote:
On 4/4/14, 1:19 PM, Gavin Sharp wrote:
The majority of the time identifyin
How easy is it to identify CPU-sensitive tests?
I think the most practical solution (at least in the near term) is to
find that set of tests, and run only that set on a faster VM, or on real
hardware (like our ix slaves).
Jonathan
On 4/7/2014 3:16 PM, Randell Jesup wrote:
The B2G emulator
On 4/8/2014 1:05 AM, Thomas Zimmermann wrote:
There are tests that instruct the emulator to trigger certain HW events.
We can't run them on actual phones.
To me, the idea of switching to a x86-based emulator seems to be the
most promising solution. What would be necessary?
Best regards
Thomas
Periodically, we field a request to add support for mochitest-chrome to
Android and B2G. To date, we've avoided this by pointing out ways that
mochitest-plain can be used for the same use case, which usually
involves SpecialPowers.
We have a new request for this, in the context of requestAuto
The A-team would be very interested in being able to track JS code
coverage; if you implemented the ability, we could add jobs in TBPL to
track our test coverage over time, which would probably be useful and
interesting.
We'd be happy to get involved at any stage where it's practical to start
Hey Joshua,
That's awesome!
How long does the try run take that generated this data? We should
consider scheduling a periodic job to collect this data and track it
over time.
Jonathan
On 7/6/2014 10:02 PM, Joshua Cranmer 🐧 wrote:
I don't know how many people follow code-coverage updates in
/2014 10:40 AM, Joshua Cranmer 🐧 wrote:
On 7/7/2014 11:39 AM, Jonathan Griffin wrote:
Hey Joshua,
That's awesome!
How long does the try run take that generated this data? We should
consider scheduling a periodic job to collect this data and track it
over time.
Well, it depends o
an
On 7/7/2014 12:49 PM, Joshua Cranmer 🐧 wrote:
On 7/7/2014 1:11 PM, Jonathan Griffin wrote:
I guess a related question is, if we could run this periodically on
TBPL, what would be the right frequency?
Several years ago, I did a project where I ran code-coverage on
roughly every nightl
Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1035464 for those
that would like to follow along.
Jonathan
On 7/7/2014 3:22 PM, Jonathan Griffin wrote:
So it sounds like it would be valuable to add try syntax to trigger
this, as well as produce periodic reports. Most of the work needed
If this only involves tiny changes to mochitest and it's ready, I'd go
ahead and do that.
I am interested in seeing what your requirements are, though, and
figuring out if we could meet them later with a better architected
solution, whether it's Marionette or something else. Mochitest is kind
I think this is a great idea, although like others have said, I'd like
to have this implemented inside the test manifests, regardless of
directory structure.
A related piece is reporting; for years, we've had tests like this run
on separate systems, reporting to custom dashboards, because they
Hi Neil,
Can you show us the command-line you're using?
Jonathan
On 8/19/2014 1:53 AM, Neil wrote:
Gregory Szorc wrote:
On 8/18/2014 4:45 PM, Neil wrote:
Time was that you could just python runtests.py to run mochitests.
Then we needed modules that you don't get in the default python, so
Our pools of test slaves are often at or over capacity, and this has the
effect of increasing job coalescing and test wait times. This, in turn,
can lead to longer tree closures caused by test bustage, and can cause
try runs to be very slow to complete.
One of the easiest ways to mitigate thi
s less often is on the same scale of bad, but
I would like to express my concerns about heading in that direction.
-Jeff
- Original Message -
From: "Jonathan Griffin"
To: dev-platform@lists.mozilla.org
Sent: Tuesday, August 19, 2014 12:22:21 PM
Subject: Experiment with running
On 8/19/2014 2:41 PM, Ehsan Akhgari wrote:
On 2014-08-19, 3:57 PM, Jeff Gilbert wrote:
I would actually say that debug tests are more important for
continuous integration than opt tests. At least in code I deal with,
we have a ton of asserts to guarantee behavior, and we really want
test cover
/19/14 12:22 PM, Jonathan Griffin wrote:
To assess the impact of doing this, we will be performing an experiment
the week of August 25, in which we will run debug tests on
mozilla-inbound on most desktop platforms every other run, instead of
every run as we do now. Debug tests on linux64 will
Thanks Ed. To paraphrase, no test coverage is being lost here, we're
just being a little more deliberate with job coalescing. All tests will
be run on all platforms (including debug tests) on a commit before a
merge to m-c.
Jonathan
On 8/21/2014 9:35 AM, Ed Morley wrote:
I think much of the
Hey Martin,
This is a good idea, and we've been thinking about approaches like
this. Basically, the idea is to run tests that "(nearly) always pass"
less often. There are currently some tests that fit into this category,
like dom level0,1,2 tests in mochitest-plain, and those are
time-consu
ple who
pushed's job to figure out which push was the culprit and make sure
that that push gets backed out?
I.e. if 4 pushes land between two testruns, and we see a regression,
will the 4 pushes be backed out? Or will sheriffs run the missing
tests and only back out the offending push?
/ Jonas
On 8/20/2014 11:24 AM, Gregory Szorc wrote:
It sounds like the "copy build to remote machine so we can run tests
there" is somewhat common and needs to be better supported by the tools.
I think it makes sense to leverage test archives and build packages
for this. Since mozharness already su
Some more details on how we're approaching this problem from the
infrastructure side:
Releng recently gave us the ability to run select jobs on faster VM's
than the default, see
https://bugzilla.mozilla.org/show_bug.cgi?id=1031083. We have B2G
emulator media mochitests scheduled on cedar usi
I imagine several people on the A-Team would be interested in
attending; can you cc auto-to...@mozilla.com with details when you
create a session?
Jonathan
On 11/26/14, 3:01 AM, David
Rajchenbach-Teller wrote:
The test suites have changed a
Thanks Andrew.
Gijs, if you'd like to see the notes we took in PDX on this topic,
they're here: https://etherpad.mozilla.org/ateam-pdx-intermittent-oranges
Feel free to add more ideas and comments. We're currently working on
our Q1 plan and will see how many of these things we can fit in the
The A-Team is embarking on a project to improve the developer experience
when running unittests locally. This project will address the following
frequently-heard complaints:
* Locally developers often use mach to run tests, but tests in CI use
mozharness, which can result in different behaviors.
OrangeFactor suggests that linux is about equal to our other platforms in
terms of catching intermittents:
http://brasstacks.mozilla.com/orangefactor/?display=BugCount&tree=trunk&includefiltertype=quicksearch&includefilterdetailsexcludeResolved=false&includefilterdetailsexcludeDisabled=false&includ
A mochitest-chrome job is now running on B2G emulators, and appears in
Treeherder as M(c). This job skips most existing chrome tests, since most
of the existing tests are not compatible with B2G. But it provides a
better alternative when writing mochitests that need chrome privileges than
using S
knob, you're doing it wrong. Write a
> mochitest-chrome test instead.
>
> bholley
>
> On Mon, Mar 23, 2015 at 4:38 PM, Jonathan Griffin
> wrote:
>
>> A mochitest-chrome job is now running on B2G emulators, and appears in
>> Treeherder as M(c). This job skip
have
> any insights on what needs to change?
>
> Thanks,
> Panos
>
>
> On Tue, Mar 24, 2015 at 1:38 AM, Jonathan Griffin
> wrote:
>
>> A mochitest-chrome job is now running on B2G emulators, and appears in
>> Treeherder as M(c). This job skips most existing c
Hi all,
At the next buildbot reconfig, the meaning of the string 'mochitest-chrome'
in tryserver syntax is changing. Previously, it was an alias which would
be translated into 'mochitest-o', the job in which mochitest-chrome is
run. Now, it will be interpreted as a job name.
This means if you w
There is also an old, unmaintained GUI for managing profiles:
https://developer.mozilla.org/en-US/docs/Profile_Manager
It still works, although there are a few bugs. It may be an improvement
over command-line arguments for less technical users.
Jonathan
On Wed, Apr 8, 2015 at 12:49 PM, Gavin Sh
We are removing SpecialPowers from Marionette, see
https://bugzilla.mozilla.org/show_bug.cgi?id=1149618. This means
Marionette tests will no longer be able to use SpecialPowers to gain access
to a privileged context.
As part of this effort, I'm adapting all Marionette tests in
mozilla-central and
OrangeFactor [ http://brasstacks.mozilla.com/orangefactor/ ] now displays
oranges/push; it used to display oranges/testrun.
The definition of testrun was always a bit fuzzy, but was intended to
compensate for the fact that some pushes will naturally have more oranges
than others due to manual or p
For quite some time we've wanted unit tests for our test harnesses which
verify issues like correct end-to-end handling of crashes and hangs,
including generation of proper stack traces.
It's not just a per-platform issue, but a per-harness per-platform issue,
so fixing this across the board isn't
For a discussion of current B2G test automation status and future plans,
see this blog post:
http://jagriffin.wordpress.com/2012/07/31/mozilla-a-team-b2g-test-automation-update/
Jonathan
On 8/1/2012 9:30 PM, Boris Zbarsky wrote:
On 8/1/12 5:47 PM, Alex Keybl wrote:
any desktop/mobile change
48 matches
Mail list logo
