Hi all, We've been struggling with our CI usage. With significant recent effort from the community, we have managed to keep usage barely under the limit (though sometimes slightly over). Efforts to improve it are still ongoing. However, we achieved this by sacrificing scheduled CIs; many now run only once every four days. If we save some time on our post-merge CIs, we can run the scheduled CIs a bit more frequently.
I noticed that we could've skipped some post merge CIs. For example, we sometimes have consecutive version upgrades. Each commit should be relatively safe, yet we run a full suite on all of them. We could skip the CI for the former upgrades and run a full CI only on the last one to confirm the upgrade is successful. We also have commits that obviously have no impact (or are extremely safe) on the code base but trigger a full CI. ( https://github.com/apache/spark/commit/b7d304f001690510691110e2acc43872b39f22f2 , https://github.com/apache/spark/commit/9ccb1ddab42aef78e6eb2d3b260fdedbb60e0002 , https://github.com/apache/spark/commit/714cb540ebda17e21ba2c53959628087a6412427 ). Our automatic system is not smart enough (yet) to skip the CI, but our committers can be. This is not a rare occurrence in our commit history; I believe skipping these CIs could save us significant time. My proposal is to have a manual way to skip the post merge CI. I favor the idea to make it a git trailer - aka having a `CI: skip` at the end of the commit mesasge. Good things about it: 1. git already recognizes this format (similar to other trailers like Co-authored-by:) and has a dedicated command to extract it. 2. this leaves some flexibility in the future, if we want to have something like `CI: light` for safe commits. 3. Less possibility of conflict compared to a raw string in the commit message. Why commit message? 1. It's traceable in the commit history. 2. Super easy to implement - we can even add a question in our merge script. 3. Minimal effort for the committer - they can either edit the message in github or do it in the merge script. Could also be specified by the author of the PR. 4. Our current merge system does not provide a strong link between the commit and the PR, so achieving this via a GitHub label would be more difficult. Happy to hear from the community! Tian Gao
