All, I've some updates on the initiative:
* I've added some initial support of @acs-robot which along with Github actions is used to (a) detect UI changes and kick UI QA builds, (b) detect java/xml changes and kick quality check and report coverage report on the PR * Enabled coverage/quality, example the coverage reports can be seen here: http://qa.cloudstack.cloud/coverage/ (I'll set up a daily job, on main branch) * Quality metrics/analysis re-setup with Sonar: https://sonarcloud.io/project/overview?id=apachecloudstack (will set up a daily job on main branch) To explore and investigate: (based on feedback) * Build/re-kick failed Travis jobs * Assign (latest) milestones to new issues and PRs * Detect if PR needs packaging and running smoketests, build/re-kick failed pkg jobs * Assign PR approvers based on file-path matching * Explore automatic triaging of issues (label, assign to maintainers) * Label PRs by size of the PR changes (similar to k8s, s/m/l/xl/xxl/...) * Per PR Sonar-cloud integration (not sure if this is necessary unless contributors want this) Please do share your ideas and thoughts. Thanks. Regards. ________________________________ From: Rohit Yadav <rohit.ya...@shapeblue.com> Sent: Thursday, April 7, 2022 13:37 To: dev@cloudstack.apache.org <dev@cloudstack.apache.org> Subject: Re: [DISCUSS] Community CI/CD/Triage Bot Thanks for the feedback and suggestions Nicolas. I've been exploring the ASF infra wiki and docs to see what's already out there we can adapt and use. So far I've report progress [1][2]: * boring-cyborg: this probot based Github app/bot approved by ASF-infra and used by other projects such as Airflow. This allows automatic tagging of PRs by filepath changes [1], and can help put welcome messages and encouraging msg to a new contributor's first PR merge * merge-conflict checker: based on Nicolas's feedback I put together this Github Action workflow [2] which labels PRs with merge conflicts and comments to the PR author to address the conflict * dependabot: not enabled by me, but it appears ASF-infra is enabling this asf-repo wide (https://github.com/apache/cloudstack/pulls/app%2Fdependabot) I'll be exploring the following: * Jacoco/code coverage stats on a PR (unit test coverage for changes) * Explore automatic changes in the following to assist and reduce release management effort: * assign milestone to new PRs and issues (or maybe a triage label?) * re-kick of failed Travis tests (for ex. try 2-3 times before giving) * detect if a PR needs smoke tests or not; kick UI builds if needed * Assign reviewers/approvers on PRs based on file paths (or last commit changes?) - this may need some thinking, as most projects such as k8s keep an owner file in directories to keep track of who's best to review changes * Explore how issues can be triaged and automatically labeled * Explore other things mentioned in this thread and report findings to dev@ [1] https://github.com/apache/cloudstack/blob/main/.github/boring-cyborg.yml [2] https://github.com/apache/cloudstack/blob/main/.github/workflows/merge-conflict-checker.yml [3] https://github.com/apache/cloudstack/pulls?q=is%3Apr+is%3Aopen+label%3Astatus%3Ahas-conflicts Regards. ________________________________ From: Nicolas Vazquez <nicolas.vazq...@shapeblue.com> Sent: Wednesday, April 6, 2022 19:29 To: dev@cloudstack.apache.org <dev@cloudstack.apache.org> Subject: Re: [DISCUSS] Community CI/CD/Triage Bot Thanks Rohit for this initiative, I think it is very useful. From the list of tasks I’m not really sure about the automatic merging of PRs – but the rest could be very useful, specially the merge conflicts detection which is a very common case near the releases freeze date. I would like to propose a few more tasks: * For new issues/PRs - assign the milestone in case it is not set * Close/reopen a PR in case of failures on the checks * Detect whether to build packages and run tests or build only UI based on the files changed/labels Regards, Nicolas Vazquez From: Rohit Yadav <rohit.ya...@shapeblue.com> Date: Tuesday, 5 April 2022 at 16:37 To: dev@cloudstack.apache.org <dev@cloudstack.apache.org> Subject: [DISCUSS] Community CI/CD/Triage Bot All, I'm trying to investigate and experiment with a few things and new ideas around our QA/automation and CI/CD. I want to build initial support around this and ask if anybody has any feedback, objections on this; Taking inspiration from kubernetes and other apache projects [2][3], I'm trying to explore the following use-cases and requirements that are initiated by a new community github bot [1]: * Triage issues and PRs: * Add labels on issues and PRs * Assign reviewers and approvers for PRs * Perform any other checks and misc things (for ex. welcome new contributors?) * Have ability to remind people on issues and PRs (for ex. /remind in 2 weeks... etc) * Kick tests, lints etc using apache approved/integrated automation hooks, Github Actions etc. * Generally, explore use of Github Actions, marketplace apps/bots or existing opensource systems such as prow * Report code coverage, stats, and other relevant/related advice And, if ASF infra permits, eventually the bot may perform the following mundane tasks which can help our RMs and contributors who end of spending a lot of time on: * Keep track of latest CI/CD reports (for ex. smoketests reports) and remove old reports/comments * Help re-kick failed jobs (builds and tests, for example try to build 3 time before giving up, or rekick failed test) or notify relevant people (maybe an external test-runner dashboard?) * Remind people when their PRs are outdated, have merge-conflicts etc. * Automated merging of PRs that pass community merge guidelines, tests; also ability for RM/contributors to put a /hold on a PR * For docs, website repos: build docs/website and publish on staging; on another command merge staging (Apache Airflow and others have some good examples of how they do it) * Discuss and automate other things to assist RMs and contributors in their day-to-day activities (take feedback for some use-cases, examples) (Note: the above things are largely beyond the scope of and additional features to the current blueorangutan bot which is currently limited to pkging+smoketests with different hypervisors and is gated due to limited lab capacities) For PoC/investigation purposes, I've created a new community github bot [1] which assuming it works and meets the goals can be moved and managed under the ACS PMC wing. I hope to have your support in the initial phase of investigation and see what PoC integrations we can enable to begin with on the apache/cloudstack, apache/cloudstack-documentation, and apache/cloudstack-www repos. [1] https://github.com/acs-robot [2] https://github.com/apache/airflow/blob/main/CI_DIAGRAMS.md [3] https://github.com/apache/airflow/pull/22753 [4] https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features [5] https://infra.apache.org/github-actions-secrets.html Regards.
