Hi,
I'm having trouble getting the Eclipse plugin to work with my EC2 cluster.
I'm getting the following error as soon as I run the TestDriver
java.net.UnknownHostException: unknown host:
domU-12-31-39-##-##-##.compute-1.internal
at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:195)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:791)
at org.apache.hadoop.ipc.Client.call(Client.java:686)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:348)
at
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:104)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:176)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:75)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1367)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:56)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1379)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:215)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
at org.apache.hadoop.mapred.JobClient.getFs(JobClient.java:469)
at
org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:603)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:788)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1142)
at TestDriver.main(TestDriver.java:41)
I'm using Eclipse Europa 3.3.2 and followed the instructions at
http://v-lad.org/Tutorials/Hadoop/03%20-%20Prerequistes.html (except using
EC2 cluster), and ran the ec2 proxy included with hadoop on my machine
pointed toward my cluster
The Hadoop location settings in Eclipse are:
location name: ec2-75-101-xxx-xxx.compute-1.amazonaws.com
host: ec2-75-101-xxx-xxx.compute-1.amazonaws.com
port: 50002
DFS Master (checked), port 50001
User name: root
Enable SOCKS proxy (checked)
Host: localhost
Port: 6666
When I go to domU-12-31-39-##-##-##.compute-1.internal in Firefox with proxy
turned on, it works fine (FoxyProxy with "use SOCKS proxy for DNS lookup"
enabled). But in Eclipse, I get the error. I know it's accessing the proxy,
because when I turn the proxy server off it times out, and the port numbers
are right too b/c if I change them it times out. But somehow the DNS lookup
isn't happening right.
Any help much appreciated!
Thanks,
Ben
