Hi Kaxil and team, I’d like to discuss our strategy for "copilot reviews" versus SKILL based assisted triage and maintainer review experiences.
My recent experiments with skill-based auto-triage show a downward trend in stale pull requests by filtering out "drive-by" contributions, allowing maintainers to focus on high-impact work (more stats and trends soon). Unlike 3rd-party tools, this approach is agent-agnostic, easily fine-tuned via English prompts, and potentially reusable across other ASF projects. Regarding transparency, I’ve proposed specific attribution formats [1] to distinguish between purely automated comments and those reviewed by humans. This aligns with ongoing [email protected] conversations [2] about using "Assisted By" instead of "Generated-by" to maintain accountability and contributor motivation. I have concerns that fully automated Copilot reviews may discourage contributors who expect human interaction. My experimental /maintainer-review skill [3] allows maintainers to drive the process by reviewing AI-generated comments before posting. This reduces "token ping-pong," integrates with the CODEOWNERS discussion [4], and ensures we only spend tokens on relevant, triaged PRs. And keep maintainers ultimately responsible for comments they accept to send. With Copilot moving to usage-based billing on June 1 [5], using personal or open-source credits via the skill-based approach introduces some change (but I am still unclear what it means to the billing - who pays for the tokens). Conversely, for skill-based processes, it's entirely in the hands (and pocket) of those who run the skill locally. We are also working with ASF and Alpha-Omega to secure long-term resources for maintainers for that. I would love to hear your thoughts about it - especially Kaxil's experiences regarding the Copilot review experiment - my observations might be incomplete and biased :). J. [1] https://github.com/apache/airflow/pull/65965 - comment attribution [2] https://lists.apache.org/thread/qbdkky8ls6zybyy9o3pvqnpf68r089qp - legal discussion thread on attributions [3] https://github.com/apache/airflow/pull/65981 - /maintainer-review skill [4] https://lists.apache.org/thread/5ssp4ksyohdzclxqvj7ngz0hz5wy9j68 - CODEOWNERS discussion [5] https://github.blog/news-insights/company-news/github-copilot-is-moving-to-usage-based-billing/ - change in billing for Copilot Best, Jarek Potiuk
