Hi all,
I'm having trouble converting the basic hadoop example to clojure.
Here's my code.
(ns maxtemperature
(:gen-class)
(:import [org.apache.hadoop.io IntWritable Text]
[org.apache.hadoop.mapred
JobConf JobClient
FileInputFormat FileOutputFormat
Mapper Reducer MapReduceBase])
(:use [clojure.contrib.str-utils :only (re-split)]))
(gen-class
:name mapper
:extends org.apache.hadoop.mapred.MapReduceBase
:implements [org.apache.hadoop.mapred.Mapper]
:prefix "m-")
(defn m-map [this key value output-collector reporter]
(let [line (str value)
year (subs line 15 19)
air-temperature (if (= (nth line 87) \+)
(Integer/parseInt (subs line 88 92))
(Integer/parseInt (subs line 87 92)))
quality (subs line 92 93)]
(if (and (not (= air-temperature 9999))
(re-matches quality "[01459]"))
(.collect output-collector (Text. year) (IntWritable.
air-temperature)))))
(gen-class
:name reducer
:extends org.apache.hadoop.mapred.MapReduceBase
:implements [org.apache.hadoop.mapred.Reducer]
:prefix "r-")
(defn r-reduce [this key values output-collector]
(.collect output-collector key (apply max values)))
(defn -main [s]
(let [conf (JobConf. maxtemperature)
args (re-split " +" s 2)]
(doto conf
(.setJobName "Max temperature")
(.setMapperClass mapper)
(.setReducerClass reducer)
(.setOutputKeyClass Text)
(.setOutputValueClass IntWritable))
(FileInputFormat/addInputPath conf (first args))
(FileOutputFormat/setOutputPath conf (second args))
(.runJob JobClient conf)))
(compile 'maxtemperature)
The problem is that, the compilation step fails when it encounters the
maxtemperature in the main function.
Without the main function the code works fine and even the compilation
succeeds. I can't figure out what's wrong.
Greatly appreciate help on this.
thanks,
rdsr.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
