Engineering Productivity is off to a great start in 2016; here’s what we’ve been up to in Q1. Build System
Build system improvements are a major priority for Engineering Productivity in 2016. The build team made great progress in Q1: - Windows builds are now made using VS2015. This shaves 100 minutes off of PGO builds! - Install manifest processing is up to 10x faster on Windows (10s now vs 100s before). Tests files are now lazily installed, making builds and test invocation significantly faster. - Many improvements to artifact builds, which have resulted in a 50% speed improvement - Artifact builds now support git-cinnabar users - A lot of work has been done to migrate legacy Makefiles to moz.build files, and to move away from autoconf; more along these lines will be done in Q2 - Build telemetry has been added, which will allow us to track improvements for developer builds; this is currently opt-in, so please consider setting BUILD_SYSTEM_TELEMETRY=1 in your build environment to help us validate this - The ICU build system has been reimplemented so it no longer excessively slows down builds. The build team is a large meta-team comprised of individuals from Engineering Productivity and several other teams; thanks to everyone who has contributed. MozReview and Autoland The primary goal of the MozReview team in Q1 was to increase user adoption by addressing various UX issues that have confused or frustrated users. To that end, a feedback panel consisting of some of Mozilla’s top reviewers has been created to provide a feedback loop for the MozReview developers. We’ve identified a number of issues that impact reviewer productivity and are working on them, starting with the top issue: lack of inline comments in the diff viewer. We also explored confusion around the general layout and flow of MozReview/Review Board, and working with UX designer Tiffanie Shakespeare we’re coming up with some big changes that should improve general usability. We have been working on a framework that will allow us to experiment in the UI without having to completely fork Review Board. In addition, this quarter we implemented various high-priority fixes and improvements, including - Disabling interdiff rebase filtering, since it was unreliable. - Adding options to disable reviewer deduction and to publish without prompting when pushing commits up. - Concatenating MozReview BMO-comment emails to reduce the volume of email sent out when many commits are published. - Adding extra context to the diffs in BMO comments. - Showing a comment button when hovering over the diff viewer, improving discoverability. - Clarifying status of reviewers in the commits table. We’re also very close to landing two other important features: switching from “ship it” to the standard BMO review flags (r?/r+/r-), and letting reviewers delegate reviews to others. Finally, autoland-to-inbound was rolled out, giving MozReview users an easy way to land reviewed patches. TaskCluster Migration Engineering Productivity is helping the TaskCluster team and Release Engineering migrate builds and automated tests from buildbot to TaskCluster. In Q1, this involved a lot of work in crafting a docker image that could be used to run linux64 debug unit tests successfully, and related work in greening up the test suites in that environment. Linux64 builds and tests in TaskCluster are now running as Tier 1 in Treeherder, so the teams are moving on to other linux64 flavors: opt, pgo, and asan. Performance Automation Sheriffing of performance regressions of Talos tests has moved entirely to Perfherder; Talos no longer reports data to graphserver, and graphserver will be retired in the future. Perfherder also now displays performance metrics generated by AreWeFastYet <https://arewefastyet.com/> and AreWeSlimYet <https://areweslimyet.com/>. To support the e10s project, Perfherder now has an e10s dashboard <https://treeherder.mozilla.org/perf.html#/e10s> that can be used to view the differences between e10s and non-e10s Talos tests. Finally, performance benchmarks previously running in Mozbench have been migrated to AreWeFastYet, and Mozbench has been retired. Continuous Integration A lot of work has been completed to support the addon signing project; this includes taking all of the addons used by test automation and either converting them to restartless addons and making them get installed via a new API, or signing them in-tree. All test harnesses now work with addon signing enforced. For e10s, all appropriate test suites have been enabled in e10s mode on Windows 7 on trunk, with the exception of a couple of suites on Windows 7 debug, due to ongoing assertions and leaks. All suites are running in e10s mode on all platforms on the project branch ash <https://treeherder.mozilla.org/#/jobs?repo=ash>. All relevant test suites have been changed to default to e10s mode when run locally, so that developers don’t accidentally introduce new tests which are not e10s-compatible. Try syntax has been made optional for try pushes; if no jobs are specified, users can use Treeherder’s “Add new jobs” feature to schedule Buildbot jobs at will post-push. Work is continuing on automatic classification in Treeherder; this will allow Treeherder to automatically recognize and classify (or “star”) many existing intermittents. You can see an example here <https://treeherder-heroku.herokuapp.com/#/jobs?repo=mozilla-inbound&revision=6c2df11a71b14819993bfe3f29cf8439551b802c&selectedJob=12309443>; this should be rolled out in Q2. Mobile Automation A |mach autophone| command has been added which allows autophone to be downloaded, configured and run locally. All mobile Talos tests have been migrated to autophone, which has allowed us to entirely retire panda boards. Some enhancements to mobile automation have been made to support testing on Android 6.0+. Marionette/WebDriver It’s now possible to write WebDriver specification tests in Web Platform Tests. Element interactability algorithm from W3C WebDriver standard landed in Marionette; gated behind specificationLevel >= 1 capability. Firefox UI Tests and Firefox Media Tests Both of these suites have been migrated to mozilla-central and have corresponding mach commands; firefox-ui-tests on linux64 debug are now run per-checkin in TaskCluster and have try support. Community Engagement Engineering Productivity is actively involved in increasing community engagement across the team. We continue to work on several approaches related to this: We’ve introduced the concept of “Project of the Month” in order to attract contributors to projects which are ready and willing to accept more community involvement. See https://wiki.mozilla.org/Platform_Operations/Project_of_the_Month for a list of past projects. During Q1, we wrapped up our second Quarter of Contribution (loosely modeled after GSoC) successfully; see this blog post <https://elvis314.wordpress.com/2016/02/19/qoc-2-iterations-and-thoughts/> for details on the projects involved. Lastly, we’ve also engaged some college students in UCOSP <http://ucosp.ca/> to help with code coverage; from their efforts we are now collecting some basic JavaScript code coverage data. Metrics We continue to find new uses for ActiveData <https://wiki.mozilla.org/Auto-tools/Projects/ActiveData>; we now have some build metrics for builds in automation <http://people.mozilla.org/~klahnakoski/MoBuildbotTimings/Builds-Overview.html> (caution: this is slow to load), a buildbot simulator (potentially useful for predicting the effects of changing pool sizes of test machines), and a prototype of a dashboard <http://chinhodado.github.io/codecoverage_presenter/> which can be used to examine JavaScript code coverage data. There’s also a new dashboard for Release Management which displays uplift history <http://people.mozilla.org/~klahnakoski/platform-history/release-history.html> . _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
