Following up on this, thanks to Chris we have fast artifact builds for PGO, so the time to develop and use try server is in parity with current opt solutions for many cases (front end development, most bisection cases).
I have also looked in depth at what the impact on the integration branches would be. In the data set from July-December (H2 2018) there were 11 instances of tests that we originally only scheduled in the OPT config and we didn't have PGO or Debug test jobs to point out the regression (this is due to scheduling choices). Worse case scenario is finding the regression on PGO up to 1 hour later 11 times or roughly 2x/month. Backfilling to find the offending patch as we do now 24% of the time would be similar time. In fact running the OPT jobs on Debug instead would result in same time for all 11 instances (due to more chunks on debug and similar runtimes). In short, little to no impact. Lastly there was a pending question about talos. There is an edge case where we can see a regression on talos that is PGO, but it is unrelated to the code and just a side effect of how PGO works. I looked into that in https://bugzilla.mozilla.org/show_bug.cgi?id=1514829. I found that if we didn't get opt alerts that we would not have missed any regressions. Furthermore, for the regressions, for the ones that were pgo only regressions (very rare) there were many other regressions at the same time (say a build change, or test change, etc.) and usually these were accepted changes, backed out, or investigated on a different test or platform. In the past when we have determined a regression is a PGO artifact we have resolved it as WONTFIX and moved on. Given this summary, I feel that most concerns around removing testing for OPT are addressed. I would also like to extend the proposal to remove the OPT builds since no unit or perf tests would run on there. As my original timeline is not realistic, I would like to see if there are comments until next Wednesday- January 23rd, then I can follow up on remaining issues or work towards ensuring we start the process of making this happen and what the right timeline is. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
