Hi Steven just fyi - more recent version of clojure-hadoop is available at https://github.com/alexott/clojure-hadoop - it includes many patches/improvements
On Tue, Mar 6, 2012 at 5:18 PM, Steven E. Harris <s...@panix.com> wrote: > Recently I was studying Stuart Sierra's clojure-hadoop project¹, and > there saw a technique that I'd like to discuss. There's a Java class > generated whose method definitions get patched based on a provided > configuration, and I'd like to understand the scope of this patching and > why the technique's effects are acceptable for this project. > > The namespace clojure-hadoop.job² uses the macro > clojure-hadoop.gen/gen-job-classes³ to, well, generate a few Java > classes. Later, in the job/configure-functions functionº, we see use of > the alter-var-root function to change the functions associated with the > Java methods Mapper#map() (here, mapper-map) and Reducer#reduce() (here, > reducer-reduce). > > Now to my questions. This mutation of the > class-method-to-Clojure-function binding appears to be "global." Will > it impact /all/ instances of the generated classes within this Clojure > process? > > If this impact is global, is it the case that this software never needs > to accommodate instances of these generated classes with different > configurations? In other words, the "job"-related classes wind up > getting mutated to conform to one particular configuration. That means > that a given run of the program can't handle more than one configuration > at a time. Is that acceptable here because Hadoop is only going to load > these classes for use with a single configuration? > > The patching looks like an optimization to avoid associating functions > with each Mapper or Reducer /instance/, so that it avoids that kind of > which-function-should-I-call-now lookup on each invocation. Is that a > correct interpretation of the design rationale? > > > Footnotes: > ¹ https://github.com/stuartsierra/clojure-hadoop > ² > https://github.com/stuartsierra/clojure-hadoop/blob/master/src/main/clojure/clojure_hadoop/job.clj > ³ > https://github.com/stuartsierra/clojure-hadoop/blob/master/src/main/clojure/clojure_hadoop/gen.clj#L5 > º > https://github.com/stuartsierra/clojure-hadoop/blob/master/src/main/clojure/clojure_hadoop/job.clj#L31 > > -- > Steven E. Harris > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- With best wishes, Alex Ott http://alexott.net/ Tiwtter: alexott_en (English), alexott (Russian) Skype: alex.ott -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
