I am trying to get HDFS example to work on my local machine. I have hadoop
installed and when I use the HDFS API I can transfer files back and forth so
I know hadoop is set up. I tried to use the HDFS unit tests as example but
only one actually is using HDFS in distributed mode while most of the tests
are written in hadoop local mode. I got the local mode unit tests to work
but I need the distributed mode to work. I cannot get the distributed mode
to work using camel. I copied the unit test code and am trying to follow
the unit test as given. The Part I am getting stuck on is
template.sendBody("direct:start", "CIAO" + i); . The template variable has
not been created in the test. I searched all the forums but could not find
a working example with HDFS. Here is my code:
import java.util.List;
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.component.hdfs.HdfsConfiguration;
import org.apache.camel.component.hdfs.HdfsEndpoint;
import org.apache.camel.component.hdfs.HdfsProducer;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
public class HDFSReadUsingCamel {
public static void main(String args[]) throws Exception {
Configuration conf = new Configuration();
conf.addResource(new
Path("/Users/nareshkosgi/booz/hadoop/conf/core-site.xml"));
conf.addResource(new
Path("/Users/nareshkosgi/booz/hadoop/conf/hdfs-site.xml"));
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("direct:start").to("hdfs://localhost:9000/tmp/test/test-camel-simple-write-file?fileSystemType=HDFS&splitStrategy=BYTES:5,IDLE:1000");
from("hdfs://localhost:9000/test?pattern=seg*&initialDelay=2000&fileSystemType=HDFS&chunkSize=5").to("mock:result");
}
});
context.start();
for(int i=0; i<10; ++i) {
template.sendBody("direct:start", "CIAO" + i) // this
is the part that
has an error and I am not able to fix
}
MockEndpoint resultEndpoint =
context.getEndpoint("mock:result",
MockEndpoint.class);
List<Exchange> exchanges = resultEndpoint.getExchanges();
for(Exchange exchange : exchanges) {
System.out.println(exchange.getIn().getBody(String.class));
}
Thread.sleep(1000);
context.stop();
}
}
Sorry if this is a noob question but I searched the forms and google but
could not find any information and I am using the forum as a last resort.
Thanks
Naresh
-----
Naresh
--
View this message in context:
http://camel.465427.n5.nabble.com/HDFS-example-in-distributed-mode-tp5715103.html
Sent from the Camel - Users mailing list archive at Nabble.com.
