Hyukjin Kwon created SPARK-59071:
------------------------------------
Summary: Measure Rust coverage against a live Spark Connect server
so the badge reflects real coverage
Key: SPARK-59071
URL: https://issues.apache.org/jira/browse/SPARK-59071
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Hyukjin Kwon
The Coverage workflow (.github/workflows/coverage.yml) runs 'cargo llvm-cov -p
apache-spark-connect' with no Spark Connect server and no SPARK_REMOTE set. The
behavioral e2e_*.rs tests self-gate on SPARK_REMOTE (should_run() returns early
when it is unset), so they no-op in the coverage job and the runtime execution
paths (collect/count/RPC-backed DataFrame, functions, session, catalog, ...)
are never exercised. As a result the Rust coverage badge reports ~73.3%, well
below the real coverage.
These same e2e tests run in build_python_connect.yml, which downloads Spark
4.2.0, starts a Connect server, and sets SPARK_REMOTE.
Fix: make the coverage job start a Spark Connect server (mirroring
build_python_connect.yml) and measure coverage in two accumulated
cargo-llvm-cov passes, then combine them: (1) the default unit + golden tests,
and (2) the behavioral e2e suite against the live server. This makes the badge
reflect the coverage actually achieved when the runtime paths are exercised.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
