Hi Spark project, Further to my last mail of 2026-06-12 14:55: https://lists.apache.org/thread.html/wsqbg8jplfnwrx6p0gsl23rh76hlxs2t , we are seeing your project in the top 5 of the list for consumption of GitHub hosted Actions runners for the past week, therefore we require you to review your usage and the workflows that are consuming those Runner resources and to make any improvements that can be made to reduce this usage. If the usage remains high your access to GitHub Actions may be removed per the policy quoted above.
In addition to the information I provided last time, I have included some suggestions from prompting Claude to examine your project repo's workflow code thus, "Analyze the GitHub Actions workflows in this repo and suggest changes that would be beneficial for the goal of a) Reducing the number of concurrent workflow jobs started and b) The duration of the jobs that do start https://github.com/apache/spark"; The suggested changes are listed below, of course generated by GenAI, so use your discretion and own reading there and I am no GitHub Actions SME, so have not curated them, merely listed them here. Note, there is also now a skill available that has been developed by the magpie project which can be used to give more detailed feedback on C using your preferred GenAII: https://github.com/apache/magpie/blob/main/docs/repo-health/README.md#workflow-optimization-experimental Make every effort to optimise your usage ASAP as the near constant 100% utilisation of our available GitHub hosted runner jobs is making it practically impossible for all projects to get their CI jobs to start and complete in a reasonable length of time, and it is incumbent on all projects to do their part to help other projects get their fair share of this limited resource. Thanks. Kind regards, -Bob Thomson, on behalf of ASF Infrastructure Apache Spark Fewer concurrent jobs Stagger the scheduled variants. The */2-day crons, including branch-scheduler dispatches, start about 53 full runs on every odd day. Four start at 20:00 UTC and three in each of nine other hours. A full run peaks at roughly 25-30 jobs. Move these same-hour pairs apart or chain them with needs: build_java21 and build_java25 (04:00) build_maven_java21 and build_maven_java25 (14:00) build_python_3.11 and build_python_connect (19:00) build_python_3.13 and build_python_connect40 (20:00) build_python_3.14 and build_python_3.12_pandas_3 (21:00) Slow the release-branch schedulers. Branches 4.1, 4.2, 4.3 and 4.x each run 8 variants every 2 days. Move them to */4, as 3.5 and 4.0 already are. Make max-parallel effective. It’s 20 on the build, pyspark, and Maven matrices, but those have only 10 to 12 legs, so the cap never limits anything. When github.repository == 'apache/spark', set it to about 6. Trim the variant job lists. build_java21 enables almost the whole suite, including buf, ui, oidc-e2e, docker-integration-tests, and k8s-integration-tests. build_non_ansi runs docs, docker-integration-tests, and yarn. Drop the jobs that don’t depend on the variable under test, and confirm each one first. Rotate the Python and Maven variants. There are about 10 Python workflows and 5 Maven ones, each a full matrix for a single interpreter or JDK version. Run one version per scheduled run on a rotating day. This is a detection-latency trade-off. Shorter jobs Add timeout-minutes where it’s missing: Every job in maven_test.yml and python_hosted_runner_test.yml, including the macOS and ARM hosted variants. infra-image, where the file’s own comment says a cold cache costs about 50 minutes. buf. Cut the jobs that add no signal. The trimming in actions 4 and 5 above also shortens each variant run. --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
