alamb opened a new issue, #25148: URL: https://github.com/apache/datafusion/issues/25148
### Is your feature request related to a problem or challenge? Recently we have been seeing the Merge Queue get stuck: - https://github.com/apache/datafusion/issues/25088 We enquired with ASF Infrastructure on [INFRA-28374](https://issues.apache.org/jira/browse/INFRA-28374) and they told us the issue is that the ASF's GitHub Action queue is full which is why it is taking a long time to start jobs: > We are aware of a long term usage pattern whereby our limited number of > concurrent GitHub Actions jobs running on GitHub hosted runners is at 100% > utilisation most of the day most of the time. Thus, from time to time we > communicate to projects who utilise the most minutes to review their > workflows for efficiency improvements. > > At the moment, datafusion is the top consumer of GitHub Action minutes over > the last 7 days, and 3x the 10th in the list of minutes used. Note the "datafusion" project as measured by ASF Infra includes all DataFusion subprojects (datafusion, datafusion-comet, datafusion-ballista, datafusion-python, datafusion-site, etc.), so reducing usage may require a cross-repository effort. ### History This is the second time we have been contacted by ASF Infra about GHA usage. The first was in **May 2026**, when ASF Infra identified DataFusion as a top-5 consumer: https://github.com/apache/datafusion/issues/22455. After some tweaks, by June 2026, our usage dropped below the policy thresholds and DataFusion fell out of the top 5. ### Useful resources - ASF GitHub Actions policy (limits: 250,000 minutes / calendar week, 216,000 minutes / any 5-day window): https://infra.apache.org/github-actions-policy.html - ASF usage report for the datafusion project (login required): https://infra-reports.apache.org/#ghactions&project=datafusion&hours=168&limit=15&group=name - ASF Infra best-practices wiki page: https://cwiki.apache.org/confluence/spaces/INFRA/pages/430408443/GitHub+Actions+Recommended+Practices - ASF Slack channel for sharing workflow optimisations across projects: `#project-workflow-optimisations` ### Related issues - #22455 (previous round of this same problem) - #13813 - #21048 - #25088 (CI takes hours to start — likely a symptom of pool saturation) - #21241 (discussion of when extended tests should run) - #25011 (example of a runaway CI job: extended tests hitting the 360 minute limit) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
