nevzheng commented on issue #13515: URL: https://github.com/apache/gravitino/issues/13515#issuecomment-5880193524
**Five independent concurrency layers are stacked here — worth naming them, and sizing their real impact, so future work doesn't conflate them.** **Why it matters:** "make tests faster" collapses into one bucket in conversation, but the actual knobs operate at completely different layers with very different payoffs — some large, some closer to zero (or a cost win rather than a speed win). | Layer | Technique | Status | Speedup vs. baseline (SWAG) | Basis | |:--|:--|:--|:-:|:--| | Backend/lane sharding | Split into `coreH2Test`/`coreMySQLTest`/`corePostgreSQLTest`, separate CI jobs | Shipped, settled | **XL** | Structural enabler — everything below runs in parallel across 3 independent CI machines because of this | | Process-fork parallelism | `maxParallelForks`, multiple JVMs per lane | Shipped (#13530) | **L** | Measured: 33-50% reduction vs. fixture-reuse baseline | | Container/server sharing | One physical DB container per backend instead of one per fork | In review (#13553) | **XS** (speed) / **M** (cost) | Measured on Mac + CI-matched hardware: wall-clock is a wash (±3-11%, noise-level); real win is resource footprint, not speed | | Database-per-fork naming | Unique DB name per fork on the shared server | Shipped, quietly | **XS** | Pure enabler for the layer above; no standalone speedup | | Intra-JVM thread parallelism | JUnit-native threads instead of forks, `-PcoreDatabaseThreaded=true` (default off) | Spike, unresolved, racing | **L** *(speculative, unverified)* | Not measured yet. Fork count today is memory-budget-capped at 2; a correctly-isolated thread pool could approach the real CPU ceiling. Blocked on a current race under a shared cache-cleanup executor | **The honest read:** the container-sharing PR is a **cost** win dressed as a **speed** claim — worth being explicit about that distinction. The thread spike is the only line with real unrealized upside left, and it's still unproven. **Go deeper:** #13530, #13553. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
