Re: Add jar files on classpath when submitting tasks to Spark

2016-11-02 Thread Mich Talebzadeh
h and > the folder on all workers contains the same .jar files. > > > > Thank you for your help, > > > > Regards, > > Jan > > > > *From:* Mich Talebzadeh [mailto:mich.talebza...@gmail.com] > *Sent:* Tuesday, November 1, 2016 3:22 PM > > *To:* Jan B

RE: Add jar files on classpath when submitting tasks to Spark

2016-11-02 Thread Jan Botorek
local C:\_resources\spark-1.0-SNAPSHOT.jar -DtrainDataPath="/path/to/model/data" From: Mich Talebzadeh [mailto:mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>] Sent: Tuesday, November 1, 2016 2:51 PM To: Jan Botorek mailto:jan.boto...@infor.com>> Cc: Vinod Mangip

Re: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Mich Talebzadeh
ests.LinearRegressionTest > --master local C:\_resources\spark-1.0-SNAPSHOT.jar > -DtrainDataPath="/path/to/model/data" > > > > > > > > *From:* Mich Talebzadeh [mailto:mich.talebza...@gmail.com] > *Sent:* Tuesday, November 1, 2016 2:51 PM >

RE: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Jan Botorek
2016 2:51 PM To: Jan Botorek Cc: Vinod Mangipudi ; user Subject: Re: Add jar files on classpath when submitting tasks to Spark Are you submitting your job through spark-submit? Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8

Re: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Mich Talebzadeh
talebza...@gmail.com] > *Sent:* Tuesday, November 1, 2016 2:18 PM > *To:* Vinod Mangipudi > > *Cc:* user > *Subject:* Re: Add jar files on classpath when submitting tasks to Spark > > > > you can do that as long as every node has the directory referenced. > > &g

RE: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Jan Botorek
this settings you are able to submit jobs with according dependencies, please? From: Mich Talebzadeh [mailto:mich.talebza...@gmail.com] Sent: Tuesday, November 1, 2016 2:18 PM To: Vinod Mangipudi Cc: user Subject: Re: Add jar files on classpath when submitting tasks to Spark you can do that as

Re: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Mich Talebzadeh
f?? >> >> >> >> >> >> *From:* ayan guha [mailto:guha.a...@gmail.com] >> *Sent:* Tuesday, November 1, 2016 1:49 PM >> *To:* Jan Botorek >> *Cc:* user >> *Subject:* Re: Add jar files on classpath when submitting tasks to Spark >> >

Re: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Vinod Mangipudi
„in global“ - in the spark-defaults.conf?? > > > > > > *From:* ayan guha [mailto:guha.a...@gmail.com] > *Sent:* Tuesday, November 1, 2016 1:49 PM > *To:* Jan Botorek > *Cc:* user > *Subject:* Re: Add jar files on classpath when submitting tasks to Spark > > >

RE: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Jan Botorek
, November 1, 2016 1:49 PM To: Jan Botorek Cc: user Subject: Re: Add jar files on classpath when submitting tasks to Spark There are options to specify external jars in the form of --jars, --driver-classpath etc depending on spark version and cluster manager.. Please see spark documents for

Re: Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread ayan guha
There are options to specify external jars in the form of --jars, --driver-classpath etc depending on spark version and cluster manager.. Please see spark documents for configuration sections and/or run spark submit help to see available options. On 1 Nov 2016 23:13, "Jan Botorek" wrote: > Hello,

Add jar files on classpath when submitting tasks to Spark

2016-11-01 Thread Jan Botorek
Hello, I have a problem trying to add jar files to be available on classpath when submitting task to Spark. In my spark-defaults.conf file I have configuration: spark.driver.extraClassPath = path/to/folder/with/jars all jars in the folder are available in SPARK-SHELL The problem is that jars are