nitin-kalyankar25 commented on issue #872:
URL:
https://github.com/apache/datafusion-comet/issues/872#issuecomment-2313023795
@radhikabajaj123 :
As your error indicates, `initialize Spark session.
java.lang.ClassNotFoundException: org.apache.spark.CometPlugin`, the
`CometPlugin` class is not being found. This typically happens when the JAR
containing the class is either missing or not properly referenced in your
project.
1. ```Check the JAR Path:``` Ensure that you are passing the correct path to
the JAR file and jar should be *-SNAPSHOT.jar.
2. ```Clean Existing Builds:``` Try deleting/cleaning existing JAR if it is
partially created.
3. ```Build with Profiles:``` Rebuild your project with the following
command
```make release PROFILES="-Pspark-3.4 -Pscala-2.13
```
Below command worked fine with me
```
export COMET_JAR=<*-SNAPSHOT.jar>
./spark-shell \
--jars $COMET_JAR \
--conf spark.driver.extraClassPath=$COMET_JAR \
--conf spark.executor.extraClassPath=$COMET_JAR \
--conf spark.plugins=org.apache.spark.CometPlugin \
--conf spark.comet.enabled=true \
--conf spark.comet.exec.enabled=true \
--conf spark.comet.explainFallback.enabled=true \
--conf spark.comet.exec.shuffle.mode=jvm \
--conf spark.executor.memory=1g \
--conf
spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager
\
--conf spark.comet.exec.shuffle.enabled=true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
