Here is a script I use to submit a directory of jar files. It assumes jar files
are in target/dependency or lib/
DRIVER_PATH=
DEPEND_PATH=
if [ -d "lib" ]; then
DRIVER_PATH="lib"
DEPEND_PATH="lib"
else
DRIVER_PATH="target"
DEPEND_PATH="target/dependency"
fi
DEPEND_JARS=log4j.properties
for f in `ls $DEPEND_PATH`; do DEPEND_JARS=$DEPEND_JARS,$DEPEND_PATH/$f; done
$SPARK_HOME/bin/spark-submit \
--class $1 \
--master yarn-client \
--num-executors 1 \
--driver-memory 4g \
--executor-memory 16g \
--executor-cores 4 \
--jars $DEPEND_JARS \
$DRIVER_PATH/core-ingest-*.jar ${*:2}
You would run it with a command like:
./run.sh class.to.submit arg1 arg2 …
> On Dec 22, 2014, at 1:11 AM, durga <[email protected]> wrote:
>
> One more question.
>
> How would I submit additional jars to the spark-submit job. I used --jars
> option, it seems it is not working as explained earlier.
>
> Thanks for the help,
>
> -D
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/java-sql-SQLException-No-suitable-driver-found-tp20792p20805.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
