xhumanoid commented on issue #864: URL: https://github.com/apache/datafusion-comet/issues/864#issuecomment-2317421729
@radhikabajaj123 spark.[driver|executor].extraClassPath - is a part which will be added as classpath parameters so have restriction: 1. have to use paths to libs presented on machines 2. have to use OS specific delimiters for classpath, on linux is a `:` every time when you use spark-submit all libraries from `--jars` will be loaded to local working dir, so you don't need to provide relative path, possible options: 1. do file file distribution on cluster and use this paths later with spark.[driver|executor].extraClassPath + absolute local path 2. remove any spark.[driver|executor].extraClassPath from you spark-submit and include all necessary jars into `--jars` parameter for test purpose i recommend to use option 2 but for production better to use option 1, because you can rely on yarn-site.xml config and include this jars into classpath by default -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
