Hello. I also don't work the loadAndDist() method. Maybe it's deprecated. I'll check and fix the documents.
using spark-shell, you can run your application as following step. 1. remove specified jar configuration in the spark-defaults.conf 2. in the spark-home, bin/spark-shell* --jars "YOUR JAR COMMA SPERATED" * If you share your application code and environments informations(zeppelin and spark version you're using, and zeppelin-env.sh etc..), i might help you. Thanks. <https://gitlab.com/search?group_id=&project_id=769187&scope=issues&search=spark-shell#2-function-define> 2016-01-25 19:08 GMT+09:00 Balachandar R.A. <balachandar...@gmail.com>: > Hello, > > I tried to use z.loadAndDist() but it says > > console>:17: error: value loadAndDist is not a member of > org.apache.zeppelin.spark.dep.DependencyContext > > Any idea here what this method is for? > > > regards > Bala > > On 25 January 2016 at 15:34, Balachandar R.A. <balachandar...@gmail.com> > wrote: > >> Hello, >> >> I have run the code in spark-shell successfully but the jar files were >> all specified in the config files (spark-defaults.conf). However, I will >> not be able to use z.load() in spark-shell. Isn't? I am sorry but I did not >> pick up the idea of running using spark-shell. Wail suggestion is to create >> a fatJar? I will give it a try but still how do i make sure this fatJar is >> accessible to spark executors? ANyway, I will keep you posted on this >> >> regards >> Bala >> >> On 25 January 2016 at 13:39, Hyung Sung Shim <hss...@nflabs.com> wrote: >> >>> Hello. >>> I think Wail Alkowaileet's comment is possible. >>> Balachandar, Could you try to run your application with spark-shell? >>> >>> >>> 2016-01-25 15:45 GMT+09:00 Wail Alkowaileet <wael....@gmail.com>: >>> >>>> I used z.load in my case and it seems to be working just fine. >>>> Can you try spark-shell with your jar file? and see what is the error? >>>> >>>> I assume the problem that your application requires third-party jars. >>>> Therefore, you need to build your app with 'assembly'. >>>> >>>> >>>> On Mon, Jan 25, 2016 at 9:39 AM, Balachandar R.A. < >>>> balachandar...@gmail.com> wrote: >>>> >>>>> Hello Hyung, >>>>> >>>>> There is nothig I could make out from error log as it is plain >>>>> straightforward that classNotFoundException >>>>> >>>>> On 25 January 2016 at 11:34, Hyung Sung Shim <hss...@nflabs.com> >>>>> wrote: >>>>> >>>>>> It's weird..so Could you send the error log for details? >>>>>> >>>>>> 2016-01-25 15:00 GMT+09:00 Balachandar R.A. <balachandar...@gmail.com >>>>>> >: >>>>>> >>>>>>> Hi Hyung, >>>>>>> >>>>>>> Thanks for the response. This I have tried but did not work. >>>>>>> >>>>>>> regards >>>>>>> Bala >>>>>>> >>>>>>> On 25 January 2016 at 11:27, Hyung Sung Shim <hss...@nflabs.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hello. Balachandar. >>>>>>>> In case of third one that you've tried, It must be first executed >>>>>>>> in the notebook. >>>>>>>> Could you try restart the zeppelin and run first the "%dep >>>>>>>> z.load()" paragraph? >>>>>>>> >>>>>>>> >>>>>>>> 2016-01-25 14:39 GMT+09:00 Balachandar R.A. < >>>>>>>> balachandar...@gmail.com>: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> Any help would be greatly appreciated :-) >>>>>>>>> >>>>>>>>> >>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>> From: Balachandar R.A. <balachandar...@gmail.com> >>>>>>>>> Date: 21 January 2016 at 14:11 >>>>>>>>> Subject: Providing third party jar files to spark >>>>>>>>> To: users@zeppelin.incubator.apache.org >>>>>>>>> >>>>>>>>> >>>>>>>>> Hello >>>>>>>>> >>>>>>>>> My spark based map tasks needs to access third party jar files. I >>>>>>>>> found below options to submit third party jar files to spark >>>>>>>>> interpreter >>>>>>>>> >>>>>>>>> 1. export SPARK_SUBMIT_OPTIONS=<all the jar files with comma >>>>>>>>> seprated> in conf/zeppelin-env.sh >>>>>>>>> >>>>>>>>> 2. include the statement spark.jars <all the jar files with comma >>>>>>>>> separated> in <spark>?conf/spark-defaults.conf >>>>>>>>> >>>>>>>>> 3. use the z.load("the location of jar file in the local >>>>>>>>> filesystem") in zepelin notebook >>>>>>>>> >>>>>>>>> I could test the first two and they both works fine. The third one >>>>>>>>> does not work. Here is the snippet i use >>>>>>>>> >>>>>>>>> %dep >>>>>>>>> z.reset() >>>>>>>>> >>>>>>>>> z.load("file:///home/bala/Projects/pocv8.new/mapreduce/build/libs/mapreduce.jar") >>>>>>>>> >>>>>>>>> >>>>>>>>> Further, the import of class belongs to the above jar file is >>>>>>>>> working when I use the statement import com..... in zeppelin >>>>>>>>> notebook. >>>>>>>>> However, I get the class not found exception in the executor for the >>>>>>>>> same >>>>>>>>> class. >>>>>>>>> >>>>>>>>> Any clue here would help greatly >>>>>>>>> >>>>>>>>> >>>>>>>>> regards >>>>>>>>> Bala >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> *Regards,* >>>> Wail Alkowaileet >>>> >>> >>> >> >
