Hi everyone, As you may have heard by now, Chromium has started to switch their Windows builds to use clang-cl instead of MSVC [1]. This has improved their Speedometer v2 benchmark score on x86 (but not on x86-64) by about 30% according to AWFY [2]. Over the past few days, several people have reached out to me to ask about how close we are to switching to clang-cl on Windows, I think due to seeing this improvement on their side. I thought I'd write up a summary of the current state of affairs to my knowledge in the hopes that many people would find it interesting.
First things first, please note that just because Chromium has seen this improvement shouldn't make us automatically expect a similar improvement in Speedometer v2 scores in Firefox when built with clang-cl. Both Chromium and Firefox are large enough codebases that we shouldn't assume any such results to transfer from one side to the other. Also, you should note that the Chrome have also seen some regressions in some other benchmarks as a result of this change, and those regressions are currently being investigated. Those who are curious can see the dependency list of this bug [3]. About our current status with clang-cl, right now Firefox builds with clang-cl. We use these builds for the purpose of static analysis using our custom clang plugin (similarly to Mac and Linux). These builds are stood up on TreeHerder under "Windows 2012 opt" and "Windows 2012 x64 opt" marked as "S" jobs. That being said, we still have a lot of work ahead of us before we can get to a point where we can consider switching to clang-cl for the builds that we ship to our users. The below is a rough list of things we need to look into before we can consider doing so. * Keeping up with the LLVM trunk. Any serious attempt for us to switch from MSVC to clang-cl will involve fixing bugs on the LLVM side in addition to on the Firefox side (as the long history of the work done so far [4] demonstrates.) Right now, the current LLVM version we use on Windows [5] was last updated in February and is outdated. glandium recently tried building with LLVM trunk and there is a regression on the LLVM side causing the build to fail. :-( But in the periods of time when we have been actively working on the clang-cl port, we have tried to follow the LLVM trunk as closely as possible in order to reduce the amount of work involved in cherry-picking the fixes we need. * Ensuring the correctness of the resulting build. clang-cl implements Microsoft's ABI and attempts to produce object files that are compatible with those produced by MSVC. As such, even though we already build and ship our code with clang, it is possible that we still have bugs lurking either on our side or on the LLVM side that we need to find and fix (not to speak of all of the Windows specific code we have which hasn't been exercised in a shipping environment with clang.) The first step here would be to stand up all of our tests on the clang-cl builds and making them green. Ensuring things like crash rates being similar to MSVC builds, etc. would be the next steps. * Ensuring the performance of the resulting build. The MSVC builds that we ship are compiled using the PGO compiler. In order to perform a fair comparison with clang-cl, we should probably try to get PGO builds with clang-cl to work [6]. There is no theoretical reason why this can't work, but this isn't something that we currently support. Failing that, there is another option which is using LTO [7] but that requires us to also port Firefox to link with lld [8] as well. Another open question is how to compare the performance. The obvious answers would be to run our Talos benchmarks, and AWFY benchmarks against the two builds. Whether that would be enough is an open question. * Ensuring debuggability of the builds. I haven't paid much close attention to the recent LLVM developments for CodeView debug info support, but clang-cl has some support for -Z7 and -Zi flags [9]. We need to ensure that the generated debug info works well for our stackwalking needs (both locally using a debugger/programatically and on the server side for crash-stats) and the generated builds are usefully debuggable on Windows. * (If there are other potential details I'm not thinking of right now, please feel free to mention it here.) Last but not least, you may ask yourself why would we want to spend this much effort to switch to clang-cl on Windows? I believe this is an important long term shift that is beneficial for us. First and foremost, clang is a vibrant open source compiler, and being able to use open source toolchains on our most important platforms is really important for us in terms of being able to contribute to the compiler where needed (anyone remember the issues we had a few years back with regards to MSVC PGO compiler hitting the maximum address space limit on Win32 when linking Firefox?). But more importantly, clang supports many exciting features that MSVC lacks. For example, clang usually implements newer language features faster than MSVC (sometimes by years), and our usage of MSVC holds us back in terms of the adoption of such features. Also, it has features such as various sanitizers, some of which [10] we may want to consider turning on by default in the builds that we ship to our users. I hope this is helpful. Cheers, Ehsan [1] https://groups.google.com/a/chromium.org/forum/#!msg/chromium-dev/Y3OEIKkdlu0/TCcT1SvwAwAJ [2] https://arewefastyet.com/#machine=37&view=single&suite=speedometer-misc&subtest=score&start=1501023532&end=1501883073 [3] https://bugs.chromium.org/p/chromium/issues/detail?id=82385 [4] https://bugzilla.mozilla.org/show_bug.cgi?id=winclang [5] See llvm_revision in https://searchfox.org/mozilla-central/source/build/build-clang/clang-win32.json and https://searchfox.org/mozilla-central/source/build/build-clang/clang-win64.json [6] While it is possible to compare the MSVC PGO build's performance to clang non-PGO build's performance, that is really an apples vs. oranges comparison since the types of optimizations done by the compiler would be quite different. [7] http://llvm.org/docs/LinkTimeOptimization.html [8] https://lld.llvm.org/ [9] https://msdn.microsoft.com/en-us/library/958x11bc.aspx [10] For example, ubsan < https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html>, CPI < https://clang.llvm.org/docs/SafeStack.html> and CFI < https://clang.llvm.org/docs/ControlFlowIntegrity.html>. -- Ehsan _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
