andygrove commented on issue #1576:
URL:
https://github.com/apache/datafusion-comet/issues/1576#issuecomment-2770828974
> Update: spun up another cluster on Spark 3.5.3 and used the same prebuilt
Comet JAR 0.7.0
>
> Was able to get past the initial error documented here but now running into
>
> `Caused by: java.lang.NoSuchMethodError: 'void
org.apache.spark.shuffle.IndexShuffleBlockResolver.writeMetadataFileAndCommit(int,
long, long[], long[], java.io.File)' `
I checked out Spark v3.5.3 and the method is defined as:
```
def writeMetadataFileAndCommit(
shuffleId: Int,
mapId: Long,
lengths: Array[Long],
checksums: Array[Long],
dataTmp: File): Unit = {
```
It also looks like this method signature has not changed recently.
Can you confirm that you are using the open-source Apache Spark v3.5.3 and
not a GCP Dataproc version of Spark?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
