Re: --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath

2019-04-20 Thread Jason Nerothin
Hi Rajat, A little more color: The executor classpath will be used by the spark workers/slaves. For example, all JVMs that are started with $SPARK_HOME/sbin/start-slave.sh. If you run with --deploy-mode cluster, then the driver itself will be run from on the cluster (with executor classpath). If

Re: --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath

2019-04-20 Thread Subash Prabakar
Hey Rajat, The documentation page is self explanatory.. You can refer this for more configs https://spark.apache.org/docs/2.0.0/configuration.html or any version of Spark documentation Thanks. Subash On Sat, 20 Apr 2019 at 16:04, rajat kumar wrote: > Hi, > > Can anyone pls explain ? > > > O

Re: --jars vs --spark.executor.extraClassPath vs --spark.driver.extraClassPath

2019-04-20 Thread rajat kumar
Hi, Can anyone pls explain ? On Mon, 15 Apr 2019, 09:31 rajat kumar Hi All, > > I came across different parameters in spark submit > > --jars , --spark.executor.extraClassPath , --spark.driver.extraClassPath > > What are the differences between them? When to use which one? Will it > differ > if