[cross-posting to firefox-dev] Hello fellow Mozillians,
The Engineering Effectiveness Team works on a disparate number of topics with the goal of improving every aspect of Firefox development. Since most of our activities are not directly visible, we have decided to regularly write a newsletter to get you up to speed! This is the second iteration of the newsletter for the period of April, May and June. = Highlights = - Andrew Halberstadt and Marco Castelluccio improved the `mach try auto` automatic test selector to run fewer tests and catch more failures <https://groups.google.com/forum/#!topic/mozilla.dev.platform/Eeg7vIvc3kM>, by scheduling (for a subset of suites) at the manifest level. Please use it and report any feedback! - Mike Hommey, Connor Sheehan and others stood up the KaiOS project branch <https://bugzilla.mozilla.org/show_bug.cgi?id=1628832>, a big milestone for the first phase of the project. - Christian Holler wrote a tutorial for libfuzzer-based fuzzing <https://firefox-source-docs.mozilla.org/tools/fuzzing/fuzzing_interface.html>. Feel free to contact him should you need help in adding fuzzing to your module. - Emma Humphries transitioned Bugzilla severities to a new system, using S1...S5 <https://docs.google.com/document/d/1HYPBDePvYX26JOrk6ymsEeViwTdWhCVL-elSwInsMKw/edit>. = Cost savings and performance improvements = - Marco Castelluccio and Andrew Halberstadt enabled the first iteration of machine learning-based scheduling on autoland, which means 20-30% fewer jobs to execute with similar coverage of regressions. - David Major started generating the Clang binary for CI using PGO, which leads to a decrease in build time by 5 to 9% <https://bugzilla.mozilla.org/show_bug.cgi?id=1326486#c17>. - Nicholas Nethercote improved Rust compiler's performance even more (up to 18%) <https://blog.mozilla.org/nnethercote/2020/04/24/how-to-speed-up-the-rust-compiler-in-2020/>. - Simon Fraser built a dashboard <https://datastudio.google.com/reporting/1GVefOEovUP_oxJKiRlsAeoJuxcoDjlK3/page/WOR0> to visualize CI costs by person / manager / team. - The Taskcluster team migrated the community and Firefox CI clusters back-end from Azure to Postgres. It is faster and more reliable, and reduces the number of cloud providers involved in Taskcluster setup. - Joel Maher identified orphaned workers as a significant overhead, and is working to reduce them. - Chris AtLee optimized test packages <https://bugzilla.mozilla.org/show_bug.cgi?id=1632601> to reduce test overhead on Windows 7 by 4-5 minutes. - Nicholas Nethercote landed and enabled fix-stacks on all platforms, which leads to a 100x perf improvement in some log processing <https://blog.mozilla.org/nnethercote/2020/04/15/better-stack-fixing-for-firefox/>. - Jesse Schwartzentruber made significant progress in migrating the fuzzing infrastructure to Taskcluster. = Phabricator and code quality = - Bastien Abadie changed the code review bot to use Phabricator lint results <https://lists.mozilla.org/pipermail/dev-platform/2020-March/025348.html> (example on https://phabricator-dev.allizom.org/D1758), making the issues found by the bot less spammy (issues cleared after diff update) and making it clear when issues would break CI. - David Lawrence reworked our Phabricator extensions to unblock a major Phabricator upgrade, which brings two new interesting features: inline comments on character ranges <https://secure.phabricator.com/phame/post/view/776/quick_look_inline_comments_on_character_ranges/>, and edit suggestions <https://secure.phabricator.com/phame/post/view/777/quick_look_suggesting_edits_with_inline_comments/>. As always Chris Kolosiwsky from the Cloud Operations team was instrumental in getting this deployed. - Andi Bogdan Postelnicu made build failure reporting at review phase use clang-tidy instead of coverity, which results in better error messages and detects more issues. - Linting improvements for Python: pylint was added as a new linter <https://bugzilla.mozilla.org/show_bug.cgi?id=1623024> (issues tracked in https://bugzilla.mozilla.org/show_bug.cgi?id=1647263) and flake8 was updated to 3.8.3 <https://phabricator.services.mozilla.com/D80078>. - Bastien Abadie and Sylvestre Ledru wrote a blog post on Hacks <https://hacks.mozilla.org/2020/04/code-quality-tools-at-mozilla/> about Mozilla's tools for code quality. - Sylvestre Ledru wrote a new linter to reject the introduction of new occurrences of some non-inclusive wording <https://bugzilla.mozilla.org/show_bug.cgi?id=1642825>. = Security and stability = - The first security bug <https://bugzilla.mozilla.org/show_bug.cgi?id=1618158> found by PHC (Probabilistic Heap Checker), built by Nicholas Nethercote, was fixed. - Our fuzzing found a security bug in irregexp <https://bugzilla.mozilla.org/show_bug.cgi?id=1638154> which also seems to affect Chrome stable (fixed in Firefox 78). - Jason Kratzer improved bugmon to automatically confirm test cases, bisect issues down to a single commit <https://bugzilla.mozilla.org/show_bug.cgi?id=1647309#c2>, and verify fixes <https://bugzilla.mozilla.org/show_bug.cgi?id=1647309#c9>. Documentation will come soon. - Jason Kratzer published a blog post on Hacks <https://hacks.mozilla.org/2020/04/fuzzing-with-webidl/> detailing the new Domino fuzzer and how it leverages WebIDL definitions as a grammar. - Calixte Denizet rewrote dump_syms support for ELF in Rust <https://github.com/mozilla/dump_syms>, greatly improving its performance along the way (Previously: 3m 29s. Now: 29s). - Aki Sasaki made it so XPI signing and ad-hoc signing requests are using defined, auditable pipelines and helped Mozilla Online (China) addons use the official XPI pipeline. = Bug management = - Sylvestre Ledru moved the docs about Bug Handling in-tree <https://firefox-source-docs.mozilla.org/bug-mgmt/>. - Emma Humphries launched a campaign to identify and prioritize bugs affecting people working from home, introducing the [wfh] whiteboard tag. Please use it if you run into such bugs! - David Lawrence and glob closed 500,000 (half a million!) accounts in Bugzilla that had not been logged into in more than four years. Close to 120,000 are still enabled out of 660,000, so 17% are left. - David Lawrence moved large attachments out of the Bugzilla database (shrinking it by at least 110 GB) and into S3, improving performance and reducing storage costs. = Misc = - The Flatpack packages Mihai Tabara started producing have begun to pick up users <https://sql.telemetry.mozilla.org/dashboard/flatpaks-and-snaps>. - Mike Hoye organized a trial Developer Roadshow. If you are interested in his notes, they are over here: https://docs.google.com/document/d/1PCFyU37jhaOmjEQQwaOApPQhAcZqv4hVlAoZisHYBaY/edit. - Ricky Stewart and Mike Hommey migrated most of build/configure to Python 3. - Mitchell Hentges enabled Sentry for mach errors. We are starting to see interesting results: https://sentry.prod.mozaws.net/share/issue/14699ba2277e4e309978f91836a27015/. - Johan Lorenzo enabled automated screenshot capture for Firefox for iOS to help localizers. - Simon Fraser made it possible to do automated merges, which recently happened from beta -> release and from central -> beta. - Shivam Singhal, a volunteer, added buttons for easier copy/paste to code snippets in our documentation (example on https://firefox-source-docs.mozilla.org/contributing/contribution_quickref.html). - Sonia Singla, a volunteer, made it so that “$” characters are automagically stripped in shell scripts in the docs. - Emil Farisan Singgih, a GSoC student, moved the documentation tasks into a new, separate, Treeherder category. Thanks! - Marco. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
