alamb commented on issue #25148: URL: https://github.com/apache/datafusion/issues/25148#issuecomment-5617493901
# CI time breakdown by workflow Breaking down the report screenshot above, DataFusion is as Total usage: **336,347 minutes, or 33 full-time runners** — over the [ASF policy ceiling](https://infra.apache.org/github-actions-policy.html) of 250,000 minutes / 25 full-time runners per calendar week. Three workflows exceed 5% of total time, and together account for ~87% of all usage: ## 1. CI — 49.31% (~165,900 min) - **Repo:** [`apache/datafusion-comet`](https://github.com/apache/datafusion-comet) - **Source:** https://github.com/apache/datafusion-comet/blob/main/.github/workflows/ci.yml - Comet's top-level CI orchestrator ## 2. Rust — 27.16% (~91,350 min) - **Repo:** [`apache/datafusion`](https://github.com/apache/datafusion) - **Source:** https://github.com/apache/datafusion/blob/main/.github/workflows/rust.yml - The main DataFusion core test suite that runs on every PR push, again in the merge queue, and again on push to `main`. ## 3. Datafusion extended tests — 10.85% (~36,500 min) - **Repo:** [`apache/datafusion`](https://github.com/apache/datafusion) - **Source:** https://github.com/apache/datafusion/blob/main/.github/workflows/extended.yml - The long-running correctness suite (extended tests, hash-collision tests, sqlite sqllogictests) that runs on every push to `main`/`branch-*` **and** on every push to any PR touching `datafusion/physical*`, `expr*`, `optimizer`, or `sql` — a large fraction of all PRs. ## Everything else (< 2.5% each) | Workflow | Share | | --- | --- | | Dev | 2.38% | | Dependencies | 2.01% | | Ballista Rust | 1.73% | | Delta Contrib Build Gate | 1.47% | | Detect breaking changes | 0.96% | | TPC-H SF10 | 0.88% | | CodeQL | 0.76% | | TPC-DS SF1 | 0.47% | | PyArrow UDF Tests | 0.42% | | Docker | 0.29% | | h2o | 0.24% | | Security audit | 0.22% | | (other builds) | 0.86% | ## Observations - Unlike the May 2026 ([#22455](https://github.com/apache/datafusion/issues/22455)) where Comet was ~78% of usage, the core `datafusion` repo now accounts for ~40%+ of usage itself - The "Datafusion extended tests" slice (10.85%) is exactly the one the merge-queue gating idea in #21241 would help -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
