Re: Moving TB of data from NFS to HDFS

> If it is divided up into several files and you can mount your NFS directory on each of the datanodes. Just curious, how will this help. Praveen On Wed, Jan 25, 2012 at 12:39 AM, Robert Evans wrote: > If it is divided up into several files and you can mount your NFS > directory on each of the

Mapping between the JIRA and files changed

Hi, How do I know the code changed with a particular JIRA? If I go to MAPREDUCE-1943, there are multiple patch attachments. Should I go with the date and pick the latest patch? Is there any other way to identify the changes done to the code with a particular JIRA? Regards, Praveen

Re: Re: how to select the position of a block

Not sure why, but I was able to find ReplicationTargetChooser.java only in branch-1.0 and not in trunk, 21, 22 and 23. Regards, Praveen 2011/12/7 郭冲 > > > isGoodTarget() in ReplicationTargetChooser.java judge if a node is a > good node, it will consider the disk space , load , and the rack wh

Re: Need to know steps of installation of Hadoop 0.23

Ulhas, As Roman said the documentation for 0.23 release is spread out. I tried to consolidate the information I come across. The below link has references to Architecture, Code, Video, Build, Deploy etc. It would be better to get sometime like this into the Apache documentation. http://hadoop-tip

Re: How to test/run Hadoop after build the code

Check the single node and cluster setup documentation for Hadoop http://hadoop.apache.org/common/docs/r0.23.0/ - for the new MR engine (0.23 release) http://hadoop.apache.org/common/docs/r0.20.205.0/ - for the old MR engine (non 0.23 release) Here are some other tutorials for running 0.23 release

Re: Automatically Documenting Apache Hadoop Configuration

> > > --Bobby Evans > > > > On 12/5/11 11:54 AM, "Harsh J" wrote: > > > > Praveen, > > > > (Inline.) > > > > On 05-Dec-2011, at 10:14 PM, Praveen Sripati wrote: > > > >> Hi, > >> > >> Recently there

Automatically Documenting Apache Hadoop Configuration

Hi, Recently there was a query about the Hadoop framework being tolerant for map/reduce task failure towards the job completion. And the solution was to set the 'mapreduce.map.failures.maxpercent` and 'mapreduce.reduce.failures.maxpercent' properties. Although this feature was introduced couple of

Re: how to check which scheduler is currently running on hadoop

Hi, Check the mapreduce.jobtracker.taskscheduler property in the mapred-site.xml, if it's not set then check what it defaults to. Praveen On Thu, Dec 1, 2011 at 5:07 AM, shivam tiwari wrote: > Hi, > > please tell me how I can check which scheduler is currently running on > hadoop > > -- > Regar

Avoiding javadoc generation during MRv2 build

Hi, Once I get the code from svn for branch-0.23, is there any way to avoid generating the javadoc for the different modules. Just to cut sometime in the build process. I use the below command. mvn clean install package -Pdist -Dtar -DskipTests Thanks, Praveen

Re: Hadoop - Distributed sorting

Check the Yahoo Paper on TeraSort for more details. http://sortbenchmark.org/YahooHadoop.pdf Praveen On Sat, Nov 26, 2011 at 7:14 PM, Prashant Sharma wrote: > Madhu, > > You can check out sorting code in examples. Actually you dont need to do > anything for sorting. Map-reduce framework does

Re: Not able to compile 0.23 - may be HADOOP-6614

8 PM, Harsh J wrote: > This appears to have already been fixed since > a7654ba4324a1cc831fcef3988d9dca767bcdef3. I can compile the trunk right now > -- try a fetch+rebase/pull/up again perhaps? > > But yes, that value has to be final as the compiler noted. > > On 21-Nov-2011, at

Not able to compile 0.23 - may be HADOOP-6614

Hi, I got the latest code from branch-0.23 and got the below error. Think this is related to HADOOP-6614. [ERROR] /home/praveensripati/Hadoop/0.23/source/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/RunJar.java:[171,31] local variable workDir is accessed from within in

Running MRv1 jobs through Eclipse

Hi, I am trying to run MRv1 jobs in Eclipse. I have been able to run in Local (Standalone) Mode, but not in Pseudo-Distributed Mode. In the Pseudo-Distributed Mode, the below exception is thrown in the Eclipse console. I see a similar exception in the tasktracker log file also. I start the nameno

Hadoop Configuration Files

Hi, Where are the Hadoop configuration files (hdfs-*.xml and mapred-*.xml) read in the code? In the org.apache.hadoop.conf.Configuration class, the following code is in the static block. if(cL.getResource("hadoop-site.xml")!=null) { LOG.warn("DEPRECATED: hadoop-site.xml found in the cl

Re: ERROR building latest trunk for Hadoop project

Rerun the build with the below options and see if you can get more information to solve this. >> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. >> [ERROR] Re-run Maven using the -X switch to enable full debug logging. Thanks, Praveen On Tue, Aug 30, 2011 at 1