Thanks for this idea. I extended my pom to include clojure-1.5.1.jar in my program jar. However, the problem is still there... I did some research on the Internet, and it seems I need to mess around with Clojure's class loading strategy...
-Matthias On 12/10/2015 06:47 PM, Nick Dimiduk wrote: > I think Mattias's project is using maven though -- there's a pom in the > project that doesn't look generated. If you want to do it from lein, maybe > my old lein-hadoop [0] plugin can help? > > [0]: https://github.com/ndimiduk/lein-hadoop > > On Thu, Dec 10, 2015 at 8:54 AM, Robert Metzger <rmetz...@apache.org> wrote: > >> I had the same though as Nick. Maybe Leiningen allows to somehow build a >> fat-jar containing the clojure standard library. >> >> On Thu, Dec 10, 2015 at 5:51 PM, Nick Dimiduk <ndimi...@apache.org> wrote: >> >>> What happens when you follow the packaging examples provided in the flink >>> quick start archetypes? These have the maven-foo required to package an >>> uberjar suitable for flink submission. Can you try adding that step to >> your >>> pom.xml? >>> >>> On Thursday, December 10, 2015, Stephan Ewen <se...@apache.org> wrote: >>> >>>> This is a problem in Java. >>>> I think you cannot dynamically modify the initial system class loader. >>>> >>>> What most apps do is check for the thread context class loader when >>>> dynamically loading classes. We can check and make sure that one is >> set, >>>> but if Closure does not respect that, we have a problem. >>>> Then Closure is not built for dynamic class loading. >>>> >>>> >>>> >>>> On Thu, Dec 10, 2015 at 5:15 PM, Matthias J. Sax <mj...@apache.org >>>> <javascript:;>> wrote: >>>> >>>>> Would it make sense (if possible?) for Flink to add the user jar >>>>> dynamically to it's own classpath so Clojure can find it? Or somehow >>>>> modify Clojure's class loader? >>>>> >>>>> The jars in lib are added to the classpath at startup. This makes it >>>>> practically impossible to execute a Flink program that is written in >>>>> Clojure right now... >>>>> >>>>> >>>>> On 12/10/2015 05:09 PM, Aljoscha Krettek wrote: >>>>>> Clojure is not considering the user-jar when trying to load the >>> class. >>>>>> >>>>>>> On 10 Dec 2015, at 17:05, Matthias J. Sax <mj...@apache.org >>>> <javascript:;>> wrote: >>>>>>> >>>>>>> Hi Squirrels, >>>>>>> >>>>>>> I was playing with a Flink Clojure WordCount example today. >>>>>>> https://github.com/mjsax/flink-external/tree/master/flink-clojure >>>>>>> >>>>>>> After building the project with "mvn package" I tried to submit it >>> to >>>> a >>>>>>> local cluster. Before I started the cluster, I manually copied >>>>>>> "clojure-1.5.1.jar" into Flink's lib folder. >>>>>>> >>>>>>>> cp ~/.m2/repository/org/clojure/clojure/1.5.1/clojure-1.5.1.jar >>> lib/ >>>>>>>> bin/start-local.sh >>>>>>> >>>>>>> However, when submitting the jar, I get an exception: >>>>>>> >>>>>>>> bin/flink run -c org.apache.flink.clojure.WordCount >>>>>>> >>>>> >>>> >>> >> ~/workspace_flink/flink-external/flink-clojure/target/flink-clojure-0.10.0.jar >>>>>>> >>>>>>> >>>>>>>> ------------------------------------------------------------ >>>>>>>> The program finished with the following exception: >>>>>>>> >>>>>>>> org.apache.flink.client.program.ProgramInvocationException: The >>>>> program's entry point class 'org.apache.flink.clojure.WordCount' >> threw >>> an >>>>> error during initialization. >>>>>>>> at >>>>> >>>> >>> >> org.apache.flink.client.program.PackagedProgram.loadMainClass(PackagedProgram.java:585) >>>>>>>> at >>>>> >>>> >>> >> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:195) >>>>>>>> at >>>>> >> org.apache.flink.client.CliFrontend.buildProgram(CliFrontend.java:784) >>>>>>>> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:288) >>>>>>>> at >>>>> >>>> >>> >> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1050) >>>>>>>> at >> org.apache.flink.client.CliFrontend.main(CliFrontend.java:1103) >>>>>>>> Caused by: java.lang.ExceptionInInitializerError >>>>>>>> at java.lang.Class.forName0(Native Method) >>>>>>>> at java.lang.Class.forName(Class.java:278) >>>>>>>> at >>>>> >>>> >>> >> org.apache.flink.client.program.PackagedProgram.loadMainClass(PackagedProgram.java:578) >>>>>>>> ... 5 more >>>>>>>> Caused by: java.io.FileNotFoundException: Could not locate >>>>> org/apache/flink/clojure/WordCount__init.class or >>>>> org/apache/flink/clojure/WordCount.clj on classpath: >>>>>>>> at clojure.lang.RT.load(RT.java:443) >>>>>>>> at clojure.lang.RT.load(RT.java:411) >>>>>>>> at clojure.core$load$fn__5018.invoke(core.clj:5530) >>>>>>>> at clojure.core$load.doInvoke(core.clj:5529) >>>>>>>> at clojure.lang.RestFn.invoke(RestFn.java:408) >>>>>>>> at clojure.lang.Var.invoke(Var.java:415) >>>>>>>> at org.apache.flink.clojure.WordCount.<clinit>(Unknown Source) >>>>>>>> ... 8 more >>>>>>> >>>>>>> I am not sure why the class is not found. It is contained in the >> jar >>>>>>> file. I can fix this error by copying the user jar >>>>>>> (flink-clojure-0.10.0.jar) into Flink's lib-folder. >>>>>>> >>>>>>> It seems, that Flink is not looking into the user-jar when loading >>>> this >>>>>>> class. Can anybody explain why? >>>>>>> >>>>>>> Thx. >>>>>>> >>>>>>> -Matthias >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >
signature.asc
Description: OpenPGP digital signature
