Re: Checking out Hadoop source code in Eclipse
@ gaurav - yep did check the project build path @ amogh - yep. Had referred to this vid too . Followed the same steps as shown in the vid. @ philip - yep did a compile with compile-core-test as well as eclipse-files The errors still persist . Regards, Hrishi From: Gaurav Sharma To: common-dev@hadoop.apache.org; smarthr...@yahoo.co.in Sent: Tue, November 3, 2009 2:00:01 PM Subject: Re: Checking out Hadoop source code in Eclipse Please check your project build-path. On Tue, Nov 3, 2009 at 12:28 PM, wrote: > >Hi . >>I am trying to work on building Hadoop in Eclipse. >>The version I am checking out is >>http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1 . >>After checking out this release and following the instructions on the >>http://wiki.apache.org/hadoop/EclipseEnvironment page I am still not able to >> > >build this release. >>After running the build.xml I get a message saying Build Successful but in >>the Problems tab it shows errors like : >>DescriptionResourcePathLocationType >>Counters cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 496 >> Java Problem >>Counters cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 501 >> Java Problem >>Counters cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 503 >> Java Problem >>FileInputFormat cannot be resolvedGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 160 >> Java Problem >>FileOutputFormat cannot be resolvedGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 161 >> Java Problem >>JobClient cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 486 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 123 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 145 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 145 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 192 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 220 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 263 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 300 >> Java Problem >>JobConf cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 487 >> Java Problem >>JobID cannot be resolvedGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 495 >> Java Problem >>JobID cannot be resolvedGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 509 >> Java Problem >>JobID cannot be resolvedGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 511 >> Java Problem >>TaskReport cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 509 >> Java Problem >>TaskReport cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 510 >> Java Problem >>TaskReport cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 522 >> Java Problem >>TaskReport cannot be resolved to a typeGridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 544 >> Java Problem >>The declared package "org.apache.hadoop.mapred" does not match the expected >>package "gridmix2.src.java.org.apache.hadoop.mapred"GridMixRunner.java >>/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 19 >> Java Problem >> > >The method createJob(String[]) from the type CombinerJobCreator refers to the >missing type JobConfGridMixRunner.java >/hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 220 > Java Problem >> > >FileInp
Checking out Hadoop source code in Eclipse
Hi . I am trying to work on building Hadoop in Eclipse. The version I am checking out is http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1 . After checking out this release and following the instructions on the http://wiki.apache.org/hadoop/EclipseEnvironment page I am still not able to build this release. After running the build.xml I get a message saying Build Successful but in the Problems tab it shows errors like : DescriptionResourcePathLocationType Counters cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 496 Java Problem Counters cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 501 Java Problem Counters cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 503 Java Problem FileInputFormat cannot be resolvedGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 160 Java Problem FileOutputFormat cannot be resolvedGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 161 Java Problem JobClient cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 486 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 123 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 145 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 145 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 192 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 220 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 263 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 300 Java Problem JobConf cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 487 Java Problem JobID cannot be resolvedGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 495 Java Problem JobID cannot be resolvedGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 509 Java Problem JobID cannot be resolvedGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 511 Java Problem TaskReport cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 509 Java Problem TaskReport cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 510 Java Problem TaskReport cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 522 Java Problem TaskReport cannot be resolved to a typeGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 544 Java Problem The declared package "org.apache.hadoop.mapred" does not match the expected package "gridmix2.src.java.org.apache.hadoop.mapred"GridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 19 Java Problem The method createJob(String[]) from the type CombinerJobCreator refers to the missing type JobConfGridMixRunner.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 220 Java Problem FileInputFormat cannot be resolvedGenericMRLoadJobCreator.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 52 Java Problem FileInputFormat cannot be resolvedGenericMRLoadJobCreator.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 69 Java Problem FileOutputFormat cannot be resolvedGenericMRLoadJobCreator.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 47 Java Problem JobClient cannot be resolved to a typeGenericMRLoadJobCreator.java /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline 58 Java Problem JobClient cannot be resolved to a typeGenericMRLoadJobCreator.java
Re: Checking out Hadoop source code in Eclipse
Hey Philip. I checked your vid and followed the same steps. I used Subclipse for checking out code and the version of Eclipse which I am using is Galileo for Mac . Dont know if that is the reason. Can you send me an eclipse project for the hadoop-0.20.1 release if possible ? Regards, Hrishi - Original Message From: Philip Zeyliger To: common-dev@hadoop.apache.org Sent: Tue, November 3, 2009 1:32:18 PM Subject: Re: Checking out Hadoop source code in Eclipse Just to confirm, you've run "ant *compile-core-test"* as well as "ant eclipse-files", yes? -- Philip On Tue, Nov 3, 2009 at 10:28 AM, wrote: > Hi . > I am trying to work on building Hadoop in Eclipse. > The version I am checking out is > http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.1 . > After checking out this release and following the instructions on the > http://wiki.apache.org/hadoop/EclipseEnvironment page I am still not able > to > build this release. > After running the build.xml I get a message saying Build Successful but in > the Problems tab it shows errors like : > DescriptionResourcePathLocationType > Counters cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 496Java Problem > Counters cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 501Java Problem > Counters cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 503Java Problem > FileInputFormat cannot be resolvedGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 160Java Problem > FileOutputFormat cannot be resolvedGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 161Java Problem > JobClient cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 486Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 123Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 145Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 145Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 192Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 220Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 263Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 300Java Problem > JobConf cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 487Java Problem > JobID cannot be resolvedGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 495Java Problem > JobID cannot be resolvedGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 509Java Problem > JobID cannot be resolvedGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 511Java Problem > TaskReport cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 509Java Problem > TaskReport cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 510Java Problem > TaskReport cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 522Java Problem > TaskReport cannot be resolved to a typeGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 544Java Problem > The declared package "org.apache.hadoop.mapred" does not match the expected > package "gridmix2.src.java.org.apache.hadoop.mapred"GridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 19Java Problem > The method createJob(String[]) from the type CombinerJobCreator refers to > the missing type JobConfGridMixRunner.java > /hadoop/src/benchmarks/gridmix2/src/java/org/apache/hadoop/mapredline > 220Java Problem > FileInputFormat cannot be resolvedGeneri
Dynamically adjusting replication factor for a data block
Is it possible to set the replication factor for a data block dynamically ? Regards, Hrishikesh
Re: Dynamically adjusting replication factor for a data block
This needs to be done manually. Is there a way to do this at runtime i.e. in the java code ? - Hrishi - Original Message From: Sagar To: common-dev@hadoop.apache.org Sent: Sun, November 15, 2009 1:20:12 AM Subject: Re: Dynamically adjusting replication factor for a data block I dont think we can set replication on block level. We can change replication on a file. This will change the replication on all blocks belonging to a file hadoop fs -setrep [-R] http://hadoop.apache.org/common/docs/current/hdfs_shell.html#setrep -Sagar smarthr...@yahoo.co.in wrote: > Is it possible to set the replication factor for a data block dynamically ? > > Regards, > Hrishikesh > > > > >
Re: Dynamically adjusting replication factor for a data block
Thanks Philip. Also how about setting the replication factor at a block level ? Is there support for this in the current version of hdfs ? - Hrishi - Original Message From: Philip Zeyliger To: common-dev@hadoop.apache.org Sent: Sun, November 15, 2009 12:48:43 PM Subject: Re: Dynamically adjusting replication factor for a data block On Sun, Nov 15, 2009 at 9:45 AM, wrote: > This needs to be done manually. > Is there a way to do this at runtime i.e. in the java code ? > Sure. It's a method in the FileSystem class: http://hadoop.apache.org/common/docs/r0.18.3/api/org/apache/hadoop/fs/FileSystem.html#setReplication%28org.apache.hadoop.fs.Path,%20short%29 -- Philip
Get File path for a blockid
Is it possible to get the filepath for a file given its blockid ? I can see FSImage have that information , but is there any other way out ? Regards, Hrishi
