andygrove opened a new pull request, #5182:
URL: https://github.com/apache/datafusion-comet/pull/5182
## Which issue does this PR close?
N/A — routine patch-version bump within the already-supported Spark 4.0 line.
## Rationale for this change
Spark 4.0 support should track the latest Spark 4.0 patch release. Apache
Spark
4.0.4 has been released (tag `v4.0.4`), so this moves Comet's Spark 4.0
build,
CI, and documentation from 4.0.2 to 4.0.4 (skipping 4.0.3).
## What changes are included in this PR?
- Bumps `<spark.version>` in the `spark-4.0` Maven profile to 4.0.4. Spark
4.0.4
makes no transitive dependency changes relative to 4.0.2 (`scala.version`,
`parquet.version`, `slf4j.version`, `jetty.version`, `hadoop.version` and
`arrow.version` are all unchanged), so no other profile properties move.
- Renames `dev/diffs/4.0.2.diff` to `dev/diffs/4.0.4.diff` and regenerates it
against the `v4.0.4` tag. Three hunks needed manual reapplication, all pure
import-line context drift with no behavioural change:
- `DataFrameWindowFunctionsSuite.scala` — upstream added a
`catalyst.trees.UnaryLike` import between the patched neighbours.
- `KeyGroupedPartitioningSuite.scala` — upstream added `RDDScanExec` to a
neighbouring import.
- `WriteDistributionAndOrderingSuite.scala` — upstream added
`TransformExpression` to a neighbouring import.
- Updates the two version references inside the diff's
`HiveUDFDynamicLoadSuite`
comment. `hive-test-udfs.jar` is still stripped from the 4.0.4 release
source
tag (only a `README` remains under `sql/hive/src/test/noclasspath/`), so
the
`IgnoreCometSuite` mixin is still required.
- Points the Spark SQL CI job and the PyArrow UDF job's `pyspark` pin at
4.0.4,
and updates `dev/ci/compute-changes.py`'s `spark_4_0` path filter to the
new
diff filename.
- Updates the user guide's supported-version table and the Spark version
compatibility page.
## How are these changes tested?
- `git apply --check dev/diffs/4.0.4.diff` against a pristine `v4.0.4`
checkout
passes with no rejects or fuzz.
- Verified the regenerated diff touches exactly the same 88 files as the
4.0.2
diff, and that its only content differences from 4.0.2 are the context
lines
and comment version strings above.
- `npx prettier --check` on the modified markdown.
- Full verification is the Spark SQL CI job, which is label-gated for Spark
4.0
(`run-spark-4.0-tests`) — that label should be applied before merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
