nevzheng opened a new pull request, #13530: URL: https://github.com/apache/gravitino/pull/13530
### What changes were proposed in this pull request? This patch lets the explicit `core` H2, MySQL, and PostgreSQL lanes use up to two isolated Gradle test workers while keeping JUnit execution serial inside each worker. It: - derives `maxParallelForks` from CPU, JVM-visible memory, Gradle worker capacity, concurrently selected database lanes, and a conservative rollout cap; - accepts strict property and environment overrides, enforces every known bound, and falls back to one worker when automatic resource signals are incomplete; - keeps Docker lanes at one worker on macOS fixed-network connector configurations; - adds exact-type `DatabaseTestContext` injection while preserving #13518's class-scoped cleanup, poisoning, rebuild, and fresh-fixture behavior; - records per-worker and per-class `[CORE-DB-SHARD]` telemetry and requires it in each database lane's CI evidence artifact; and - adds focused policy, injection, and telemetry regression tests. Adaptive forks apply only to the explicit database lane tasks. The unit lane and legacy `:core:test` behavior remain unchanged, and JUnit method-level parallelism remains disabled. ### Why are the changes needed? #13518 gives each test class an explicit fixture lifecycle, but each database lane still runs in one JVM and leaves available CI capacity unused. Bounded JVM-level concurrency reduces wall time while preserving worker-owned database servers and exact test identities. This PR is stacked on #13518. It does not introduce connection pooling, shared database servers, namespace sharding, or method-level parallelism. Fix: #13529 Depends on: #13518 Related to: #13515 and #13517 ### Does this PR introduce _any_ user-facing change? No production or runtime user-facing change. Public APIs, runtime configuration, and storage formats are unchanged. The only new controls are test/CI-only Gradle properties and matching environment variables: - `coreDatabaseForks` / `CORE_DATABASE_FORKS` - `coreDatabaseBuildMemoryReserveMiB` / `CORE_DATABASE_BUILD_MEMORY_RESERVE_MIB` - `coreDatabaseForkMemoryOverheadMiB` / `CORE_DATABASE_FORK_MEMORY_OVERHEAD_MIB` ### How was this patch tested? Matched local runs compared #13518 at `095f7d7430f4a8c441ba7c095210fed72045b7c2` with this patch at `5a51aa804b834b7f198c472ee1f045ebea778b65`. The environment was macOS 27.0 arm64, OrbStack, JDK 17, Gradle 8.2, 14 JVM-visible processors, 48 GiB JVM-visible memory, and `--max-workers=2`; every run used `--rerun-tasks`. | Lane | Baseline task wall | Candidate task wall | Improvement | Result parity | Fixtures | Containers | Candidate workers | | --- | ---: | ---: | ---: | --- | ---: | ---: | ---: | | H2 | 63.696s | 53.455s | 10.241s / 16.1% / 1.19x | 519 passed; exact identities | 48 → 48 | n/a | 2 | | MySQL | 1326.146s | 932.091s | 394.055s / 29.7% / 1.42x | 519 passed; exact identities | 48 → 48 | 1 → 2 | 2 | | PostgreSQL | 485.720s | 307.759s | 177.961s / 36.6% / 1.58x | 517 passed, 2 skipped; exact identities | 50 → 50 | 1 → 2 | 2 | Additional completed checks: - `./gradlew -p buildSrc test --rerun-tasks`: 36 policy tests passed. - `:core:coreUnitTest`: 1,972 passed and 2 skipped. - Four-lane reconciliation: 3,531 tests total, identical database-lane identity multisets, and no unit/database overlap. - Candidate telemetry reconciled with all 40 JUnit classes and all 519 tests in each database lane. - Docker containers were absent after each lane completed. - `actionlint .github/workflows/build.yml`, Spotless, Kotlin Gradle formatting, RAT, and `:core:compileTestJava` passed. - `python3 -B -m unittest discover -s dev/ci/tests -p 'test_core_test_identity.py'`: 9 passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
