Hi all, Quick update on our CI system during the "hot phase" of the release: 1. We are (for the first time in this release cycle) *hitting our e2e test processing capacity limit:* All our 20 slots are in use, 13 builds are in the queue. The other tests are running without delays. I will monitor this closely. Please try to use your personal Azure accounts as much as possible for testing, and only push to your PR once you know the change should pass.
2. I probably made a *mistake in our e2e test scripts, causing broken e2e tests to not fail the build*. I'm tracking the issue here: https://issues.apache.org/jira/browse/FLINK-19839 I'm trying to fix this in the next hours. You can see if the e2e test failed if there's a warning with the cache upload, as with this example: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=8385&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529 I have checked the recent master builds, and the e2e tests on master seem to be stable. Until I've fixed this, please take a quick look onto the CI overview page of your PR before merging! Thanks & sorry! On Mon, Oct 19, 2020 at 11:15 AM Till Rohrmann <trohrm...@apache.org> wrote: > Thanks for the reminder Robert. I think you are completely right. Since we > are getting closer to the feature freeze and therefore also the testing > period, having green builds and only merging PRs where we are sure that CI > passes will decrease the likelihood of introducing new bugs and, hence, it > will decrease the time we need for hardening the release. Of course, this > should also be the modus operandi for not only shortly before the feature > freeze. > > Cheers, > Till > > On Mon, Oct 19, 2020 at 11:00 AM Robert Metzger <rmetz...@apache.org> > wrote: > > > Hi folks, > > > > As we are approaching the final stages of the Flink 1.12 release cycle, I > > would like to remind everybody to *only merge pull requests when the CI > > system gives green light*! > > I know that the CI system sometimes shows "FAILED" due to CI system > > instabilities, but please carefully check those cases to make sure it is > > really the CI systems fault, and not an unstable or failing test. > > It is much easier to identify and fix an unstable or failing test in the > > context of a pull request then after it's been merged. > > > > An essential part of keeping our CI system useful is helping to address > > build instabilities early and proactively. > > In this phase, I'll try to proactively revert commits introducing severe > > issues, or disable failing/unstable tests early to keep the CI system > > meaningful. > > > > Please post in this thread or reach out to me personally if you are > > uncertain about a build failure or the overall state of the CI system. > > > > Best, > > Robert > > >
