Neha:
thanks for your response. I figured it out. it seems that the new
hadoop-1.x default port is changed from 8020 to 9000, so I had to use port
9000 in my default fs setting
I have to put a line
defaultFS=hdfs://localhost:9000/
something like that into my camus.properties.
camus_...@googlegroups.com seems less active :)
thanks
Yang
On Tue, Sep 3, 2013 at 6:31 PM, Neha Narkhede <neha.narkh...@gmail.com>wrote:
> I'm sorry I'm not too familiar with the camus code base. Have you tried
> pinging camus_...@googlegroups.com ?
>
> Thanks,
> Neha
>
>
> On Tue, Sep 3, 2013 at 1:40 PM, Yang <teddyyyy...@gmail.com> wrote:
>
> > I tried to run camus with the properties file in the examples dir:
> >
> > java -cp ...... com.linkedin.camus.etl.kafka.CamusJob -P
> > myproperties.properties
> >
> >
> > then it says that my output dir does not exist:
> > ~/tools/camus/camus-etl-kafka$ java -cp
> > target/camus-etl-kafka-0.1.0-SNAPSHOT.jar
> > com.linkedin.camus.etl.kafka.CamusJob -P camus.properties
> > Starting Kafka ETL Job
> > The blacklisted topics: []
> > The whitelisted topics: []
> > Dir Destination set to: /camus/out
> > Getting the base paths.
> > The execution base path does not exist. Creating the directory
> > The history base path does not exist. Creating the directory.
> > Exception in thread "main" java.io.FileNotFoundException: File
> /camus/exec
> > does not exist.
> > at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:397)
> > at
> >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > at org.apache.hadoop.fs.FileSystem.getContentSummary(FileSystem.java:801)
> > at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:223)
> > at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:556)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> >
> >
> >
> > I tried changing the dir to hdfs://localhost/camus/out it says:
> > $ java -cp target/camus-etl-kafka-0.1.0-SNAPSHOT.jar
> > com.linkedin.camus.etl.kafka.CamusJob -P camus.properties
> > Starting Kafka ETL Job
> > The blacklisted topics: []
> > The whitelisted topics: []
> > Dir Destination set to: hdfs://localhost/camus/out
> > Getting the base paths.
> > Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS:
> > hdfs://localhost/camus/exec, expected: file:///
> > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> > at
> >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> > at
> > org.apache.hadoop.fs.LocalFileSystem.pathToFile(LocalFileSystem.java:61)
> > at org.apache.hadoop.fs.LocalFileSystem.exists(LocalFileSystem.java:51)
> > at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:211)
> > at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:556)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> >
> >
> > so how can I let Camus know that this is in a hdfs environment, and the
> > code in CamusJob.java:140
> >
> > private Job createJob(Properties props) throws IOException {
> > Job job = new Job(getConf());
> >
> > get a conf that points to a hdfs setup?
> >
> > I already set my env HADOOP_CONF_DIR to my running hadoop
> >
> >
> > thanks
> > Yang
> >
>
