I've written a simple map/reduce job that demonstrates a problem I'm having. Please see attached example.
Environment: hadoop 0.19.0 cluster resides across linux nodes client resides on cygwin To recreate the problem I'm seeing, do the following: - Setup a hadoop cluster on linux - Perform the remaining steps on cygwin, with a hadoop installation configured to point to the linux cluster. (set fs.default.name and mapred.job.tracker) - Extract the tarball. Change into created directory. tar xvfz Example.tar.gz cd Example - Edit build.properties, set your hadoop.home appropriately, then build the example. ant - Load the file Example.in into your dfs hadoop dfs -copyFromLocal Example.in Example.in - Execute the provided shell script, passing it testID 1. ./Example.sh 1 This test does not use -libjars, and it completes successfully. - Next, execute testID 2. ./Example.sh 2 This test uses -libjars with 1 jarfile (Foo.jar), and it completes successfully. - Next, execute testID 3. ./Example.sh 3 This test uses -libjars with 1 jarfile (Bar.jar), and it completes successfully. - Next, execute testID 4. ./Example.sh 4 This test uses -libjars with 2 jarfiles (Foo.jar and Bar.jar), and it fails with a ClassNotFoundException. This behavior only occurs when calling from cygwin to linux or vice versa. If both the cluster and the client reside on either linux or cygwin, the problem does not occur. I'm continuing to dig to see what I can figure out, but since I'm very new to hadoop (started using it this week), I thought I'd go ahead and throw this out there to see if anyone can help. Thanks!
Example.tar.gz
Description: GNU Zip compressed data
