On Thursday, January 29, 2015 at 9:45:25 AM UTC-8, Sean Corfield wrote: > > Which version of Clojure are you using? >
1.6.0, both for AOT and at runtime. > > Does clojure-hadoop or Schema include AOT-compiled versions of other > libraries and/or core namespaces? > No, as far as I know the only AOT compiled code present is clojure-hadoop and possibly the Clojure jar itself. We may depend on other libs that are AOT as well, but none that live above schema. > > If the answers are 1.7.0 Alpha 5 and "yes" then you've run into the same > problem I and a few others did: the previously undefined behavior of > loading both AOT and JIT versions of the same code now has a defined > behavior (preferring AOT) - and you get this exception. > I guess this is probably a different issue then, but maybe there's a common workaround? Thanks for the quick reply! > > Sean > > On Jan 28, 2015, at 11:39 PM, Jason Wolfe <ja...@w01fe.com <javascript:>> > wrote: > > First off, I apologize in advance for not having a reduced test case, and > express my sincere gratitude in advance for any assistance. I've been > tearing my hair out for a day or so and not making headway, and figured > someone here might recognize some keywords and have a pointer in the right > direction. (I'm admittedly pretty green when it comes to class loading, and > have largely exhausted my google fu). > > *Problem: * > > I'm submitting a hadoop job using clojure-hadoop. All is well with a > simple job, but once I require something that transitively depends on > Schema, I end up with: > > clojure.lang.Compiler$CompilerException: > java.lang.IllegalArgumentException: No implementation of method: :walker of > protocol: #'schema.core/Schema found for class: clojure.core$long, > compiling:(crane/config.clj:33:4) > > It works fine when run in-process with hadoop-mapreduce-client-jobclient, > but not with bin/hadoop -jar. This stunk of a classloader issue, and after > digging in it seems that there are multiple versions of clojure.core$long > floating around. The version on which the protocol is extended is not the > same class for the fn that the symbol 'long resolves to in client code. > > *Context: * > > clojure-hadoop is AOT-compiled, and after being loaded by hadoop it > dynamically loads the target namespace (not AOT-compiled, nor any other of > the code in question) using > https://github.com/alexott/clojure-hadoop/blob/master/src/clojure_hadoop/load.clj#L3 > > From here, schema is transitively required, and then client namespaces > attempt to use the Schema protocol to generate validators, and when the > schema 'long is used (which resolves to the fn with class > clojure.core$long), it fails to find the appropriate method. > > After repeated head-bashing, I've determined that there are (at least two) > versions of the clojure.core$long class floating around -- the one used to > extend the protocol, which stems from a DynamicClassLoader, and the one > that 'long resolves to in client code, which stems from a URLClassLoader. > The URLClassLoader is the loader of the current thread and Compiler, but > not @(clojure.lang.Compiler/LOADER). > > *Attempts:* > > I've tried wrapping the clojure-hadoop loading code with > .setContextClassLoader on some obvious candidates and binding > *use-context-classloader* around the code doing the loading, with no avail. > I've tried changing the schema code to reference the class in different > ways (class (resolve 'long)), (class 'long), etc and that hasn't made a > difference. I've checked and the clojure-hadoop jar doesn't contain any > .class files for clojure, schema, or other offending code. > > *Plea:* > > I suspect there's something obvious I'm missing. (In retrospect it seems > like the design of Schema may be suboptimal in light of this, but if > possible I'd like to figure out a workaround without changing that > substantially). Thanks in advance for your help -- any and all pointers are > welcome. > > -Jason > > > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.