Re: Bulk / Fast Read and Write with MSSQL Server and Spark

2018-05-23 Thread kedarsdixit
Yes. Regards, Kedar Dixit -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Adding jars

2018-05-23 Thread kedarsdixit
This can help us to solve the immediate issue, however the ideally one should submit the jars in the beginning of the job. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubsc

Re: Adding jars

2018-05-23 Thread kedarsdixit
In case of already running jobs, you can make use of broadcasters which will broadcast the jars to workers, if you want to change it on the fly you can rebroadcast it. You can explore broadcasters bit more to make use of. Regards, Kedar Dixit Data Science at Persistent Systems Ltd. -- Sent fro

Re: Spark is not evenly distributing data

2018-05-23 Thread kedarsdixit
Hi, Can you elaborate more here? We don't understand the issue in detail. Regards, Kedar Dixit Data Science at Persistent Systems Ltd. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail

Re: Adding jars

2018-05-23 Thread kedarsdixit
Hi, You can add dependencies in spark-submit as below: ./bin/spark-submit \ --class \ --master \ --deploy-mode \ --conf = \ *--jars * \ ... # other options \ [application-arguments] Hope this helps. Regards, Kedar Dixit Data Science at Persistent Systems Ltd -- Sent from

Re: Bulk / Fast Read and Write with MSSQL Server and Spark

2018-05-23 Thread kedarsdixit
Hi, I had came across this a while ago check if this is helpful. Regards, ~Kedar Dixit Data Science @ Persistent Systems Ltd. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/