Hi all, tl;dr: I will have to cancel some E2E test executions of pull requests because we have reached the capacity limit of Flink's Azure Pipelines account.
Long version: We have two types of agent pools in Azure Pipelines: Microsoft-hosted VMs and Alibaba-hosted Docker environment. In the Microsoft VMs, we are running the E2E tests, because we have an environment that will always be destroyed after each execution (and the E2E tests often leave dangling docker containers, processes etc.; and they modify files in system directories) In the Alibaba-hosted Docker environment, we are compiling and testing the regular Maven tests. We only have 10 Microsoft-hosted VMs available, and each E2E execution takes around 3.5 hours. That means we have a daily capacity of ~70 E2E tests a day. On Tuesday, we had 110 builds, on Wednesday 98 builds. Because of this, I will (manually) cancel some E2E test executions for pull requests. If I see that a PR is explicitly changing something on E2E tests, I will keep it. If I see that a PR is a docs change, has other test failures etc., I will cancel the E2E execution. If you want to verify that the E2E tests are passing for your own changes, you can set up Azure Pipelines for your GitHub account, it's free and works quite well. Here's a tutorial: https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository What can we do to avoid this situation in the future? Sadly, Microsoft does not allow to buy additional processing slots for open source projects [1]. However, I'm in touch with a product manager at Microsoft who promised me (yesterday) to increase the limit for us. In the Alibaba environment, we have 80 slots available, and usually no capacity constraints. This means we don't need to make compromises there. Sorry for this inconvenience. Best, Robert PS: I'm considering keeping this thread as a permanent "status update" thread for Azure Pipelines [1] https://developercommunity.visualstudio.com/content/problem/1028884/additionally-purchased-microsoft-hosted-build-agen.html
