I'm sorry I'm not too familiar with the camus code base. Have you tried
pinging camus_...@googlegroups.com ?
Thanks,
Neha
On Tue, Sep 3, 2013 at 1:40 PM, Yang <teddyyyy...@gmail.com> wrote:
> I tried to run camus with the properties file in the examples dir:
>
> java -cp ...... com.linkedin.camus.etl.kafka.CamusJob -P
> myproperties.properties
>
>
> then it says that my output dir does not exist:
> ~/tools/camus/camus-etl-kafka$ java -cp
> target/camus-etl-kafka-0.1.0-SNAPSHOT.jar
> com.linkedin.camus.etl.kafka.CamusJob -P camus.properties
> Starting Kafka ETL Job
> The blacklisted topics: []
> The whitelisted topics: []
> Dir Destination set to: /camus/out
> Getting the base paths.
> The execution base path does not exist. Creating the directory
> The history base path does not exist. Creating the directory.
> Exception in thread "main" java.io.FileNotFoundException: File /camus/exec
> does not exist.
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:397)
> at
>
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at org.apache.hadoop.fs.FileSystem.getContentSummary(FileSystem.java:801)
> at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:223)
> at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:556)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>
>
>
> I tried changing the dir to hdfs://localhost/camus/out it says:
> $ java -cp target/camus-etl-kafka-0.1.0-SNAPSHOT.jar
> com.linkedin.camus.etl.kafka.CamusJob -P camus.properties
> Starting Kafka ETL Job
> The blacklisted topics: []
> The whitelisted topics: []
> Dir Destination set to: hdfs://localhost/camus/out
> Getting the base paths.
> Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS:
> hdfs://localhost/camus/exec, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:381)
> at
>
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:55)
> at
> org.apache.hadoop.fs.LocalFileSystem.pathToFile(LocalFileSystem.java:61)
> at org.apache.hadoop.fs.LocalFileSystem.exists(LocalFileSystem.java:51)
> at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:211)
> at com.linkedin.camus.etl.kafka.CamusJob.run(CamusJob.java:556)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>
>
> so how can I let Camus know that this is in a hdfs environment, and the
> code in CamusJob.java:140
>
> private Job createJob(Properties props) throws IOException {
> Job job = new Job(getConf());
>
> get a conf that points to a hdfs setup?
>
> I already set my env HADOOP_CONF_DIR to my running hadoop
>
>
> thanks
> Yang
>
