RD,I don't know about the gen-class, to be sincere with you I have not used. But clojure follows a standard and natural sequence if you look closely. What you used in your former code were referring to different class path and the paths were not even defined.
The below is from clojure.org A stand-alone gen-class <http://clojure.org/API#gen-class> facility is provided to create named classes for direct use as Java classes, with facilities for: - Naming the generated class http://clojure.org/compilation So were do you want clojure to place the generated class. My guess is within the same folder as your 'ns or need near. Now I hope you have seen why you need that change. I invite comments from experts here if I have strayed. Regards, Emeka On Mon, Jul 27, 2009 at 9:47 AM, RD <rdsr...@gmail.com> wrote: > Hi Stuart, > Thanks for the link > Relating to the hadoop code I posted above, I got it to compile by > putting the code under a namespace and changing > the class names appropriately , but still I have no clue as to why the code > didn't work without the namespace. > > I'll defenitely check the link though, thks. > > regards, > rdsr > > > Here's the code with the modification > > (ns com.rdsr.maxt > (:gen-class) > (:import [org.apache.hadoop.io IntWritable Text] > [org.apache.hadoop.fs Path] > [org.apache.hadoop.mapred > JobConf JobClient > FileInputFormat FileOutputFormat > Mapper Reducer MapReduceBase]) > (:use [clojure.contrib.str-utils :only (re-split)])) > > (gen-class > :name com.rdsr.mapper > :extends org.apache.hadoop.mapred.MapReduceBase > :implements [org.apache.hadoop.mapred.Mapper] > :prefix "m-") > > (defn m-map [this key value output-collector reporter] > (let [line (str value) > year (subs line 15 19) > air-temperature (if (= (nth line 87) \+) > (Integer/parseInt (subs line 88 92)) > (Integer/parseInt (subs line 87 92))) > quality (subs line 92 93)] > (if (and (not (= air-temperature 9999)) > (re-matches #"[01459]" quality)) > (.collect output-collector (Text. year) (IntWritable. > air-temperature))))) > > (gen-class > :name com.rdsr.reducer > :extends org.apache.hadoop.mapred.MapReduceBase > :implements [org.apache.hadoop.mapred.Reducer] > :prefix "r-") > > (defn r-reduce [this key values output-collector reporter] > (let [values-seq (iterator-seq values)] > (.collect output-collector key > (apply max (map (fn [n] (.get n)) values-seq))))) > > (defn -main [& args] > (let [conf (JobConf. com.rdsr.maxt)] > (doto conf > (.setJobName "Max temperature") > (.setMapperClass com.rdsr.mapper) > (.setReducerClass com.rdsr.reducer) > (.setOutputKeyClass Text) > (.setOutputValueClass IntWritable)) > (FileInputFormat/addInputPath conf (Path. (first args))) > (FileOutputFormat/setOutputPath conf (Path. (second args))) > (JobClient/runJob conf))) > > (compile 'com.rdsr.maxt) > > > On Mon, Jul 27, 2009 at 9:46 PM, Stuart Sierra < > the.stuart.sie...@gmail.com> wrote: > >> >> Hi rdsr, >> >> The problem is that you're trying to use the "maxtemperature" as a >> class name before it's finished compiling. To work around that, use >> (Class/forName "maxtemperature") instead. >> >> You can also see my (very task-specific) Hadoop/Clojure integration at >> <http://tinyurl.com/mqv2os> >> and <http://tinyurl.com/lznlg2>. >> >> -SS >> >> >> On Jul 27, 4:55 am, RD <rdsr...@gmail.com> wrote: >> > Hi all, >> > I'm having trouble converting the basic hadoop example to clojure. >> > Here's my code. >> > >> > (ns maxtemperature >> > (:gen-class) >> > (:import [org.apache.hadoop.io IntWritable Text] >> > [org.apache.hadoop.mapred >> > JobConf JobClient >> > FileInputFormat FileOutputFormat >> > Mapper Reducer MapReduceBase]) >> > (:use [clojure.contrib.str-utils :only (re-split)])) >> > >> > (gen-class >> > :name mapper >> > :extends org.apache.hadoop.mapred.MapReduceBase >> > :implements [org.apache.hadoop.mapred.Mapper] >> > :prefix "m-") >> > >> > (defn m-map [this key value output-collector reporter] >> > (let [line (str value) >> > year (subs line 15 19) >> > air-temperature (if (= (nth line 87) \+) >> > (Integer/parseInt (subs line 88 92)) >> > (Integer/parseInt (subs line 87 92))) >> > quality (subs line 92 93)] >> > (if (and (not (= air-temperature 9999)) >> > (re-matches quality "[01459]")) >> > (.collect output-collector (Text. year) (IntWritable. >> > air-temperature))))) >> > >> > (gen-class >> > :name reducer >> > :extends org.apache.hadoop.mapred.MapReduceBase >> > :implements [org.apache.hadoop.mapred.Reducer] >> > :prefix "r-") >> > >> > (defn r-reduce [this key values output-collector] >> > (.collect output-collector key (apply max values))) >> > >> > (defn -main [s] >> > (let [conf (JobConf. maxtemperature) >> > args (re-split " +" s 2)] >> > (doto conf >> > (.setJobName "Max temperature") >> > (.setMapperClass mapper) >> > (.setReducerClass reducer) >> > (.setOutputKeyClass Text) >> > (.setOutputValueClass IntWritable)) >> > (FileInputFormat/addInputPath conf (first args)) >> > (FileOutputFormat/setOutputPath conf (second args)) >> > (.runJob JobClient conf))) >> > >> > (compile 'maxtemperature) >> > >> > The problem is that, the compilation step fails when it encounters the >> > maxtemperature in the main function. >> > Without the main function the code works fine and even the compilation >> > succeeds. I can't figure out what's wrong. >> > Greatly appreciate help on this. >> > >> > thanks, >> > rdsr. >> >> > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
