Compressor tweaks corresponding to HDFS-2834, 3051?

In https://issues.apache.org/jira/browse/HDFS-2834, Todd says, " This is also useful whenever a native decompression codec is being used. In those cases, we generally have the following copies: 1) Socket -> DirectByteBuffer (in SocketChannel implementation) 2) DirectByteBuffer -> byte[] (i

Re: How to change the scheduler

Hello Merto, I wish to implement SBF(Stretch Based Fairness), hence I guess the place to look will be dynamic changing of slaves. Regards, Ranjan On 03/06/12, Merto Mertek wrote: > If you will change the default scheduler take a look at the thread "Dynamic > changing of slaves" on the mail

Re: How to change the scheduler

If you will change the default scheduler take a look at the thread "Dynamic changing of slaves" on the mailing list where I described my understanding of the scheduling process. However if you will modify some fairscheduling code, take a look at classes fairscheduler.java and schedulingalgorithms.j

Re: How to change the scheduler

Hello Harsh,    Thanks for the quick info. I will go through the specifics. I hope you will address any further doubts that I have on the issue. Regards, Ranjan On 03/06/12, Harsh J wrote: > Ranjan, > > Schedulers do not apply per-job. You need to change it at the JobTracker. > > Follow inst

[jira] [Created] (HADOOP-8146) FsShell commands cannot be interrupted

FsShell commands cannot be interrupted -- Key: HADOOP-8146 URL: https://issues.apache.org/jira/browse/HADOOP-8146 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 0.23.

Re: How to change the scheduler

Ranjan, Schedulers do not apply per-job. You need to change it at the JobTracker. Follow instructions at http://hadoop.apache.org/common/docs/r1.0.0/fair_scheduler.html to switch scheduler to FairScheduler. On Wed, Mar 7, 2012 at 4:08 AM, Ranjan Banerjee wrote: > > Hello, >    I am relatively n

[jira] [Created] (HADOOP-8145) Automate testing of LdapGroupsMapping against ApacheDS

Automate testing of LdapGroupsMapping against ApacheDS -- Key: HADOOP-8145 URL: https://issues.apache.org/jira/browse/HADOOP-8145 Project: Hadoop Common Issue Type: Improvement Co

How to change the scheduler

Hello,    I am relatively new to Hadoop. Started playing with it around two weeks ago. I have finished running the canonical word count map reduce example. My class project involves coming with a different scheduler for Hadoop. I know that by default Hadoop uses the FIFO and it also has the fai

Re: Hadoop on non x86 systems

To become a contributor, an individual must fax in the ICLA (Individual Contributor License Agreement) which can be found at http://www.apache.org/licenses/icla.txt . When your name shows up on the web page at http://people.apache.org/committer-index.html#unlistedclas then your ICLA has been regis

Need help with getting some patches for 1.0.2 reviewed

Hey folks, I have a couple of patches to the 1.0 branch that I would like to shepherd through the process. I'm not entirely familiar with the process, so bear with me.. - I have a backport of HADOOP-5836 for the 1.0 branch that I have uploaded to jira. This is a no-code-change, just a re-spin

[jira] [Created] (HADOOP-8144) pseudoSortByDistance in NetworkTopology doesn't work properly if no local node and first node is local rack node

pseudoSortByDistance in NetworkTopology doesn't work properly if no local node and first node is local rack node Key: HADOOP-8144 URL: https://issues.

[jira] [Created] (HADOOP-8143) Change distcp to have -pb on by default

Change distcp to have -pb on by default --- Key: HADOOP-8143 URL: https://issues.apache.org/jira/browse/HADOOP-8143 Project: Hadoop Common Issue Type: Improvement Reporter: Dave Thompson

Re: Hadoop on non x86 systems

There are kind of two ways to submit a proposal. 1 - send an e-mail here with the proposal. 2 - file a JIRA and attach your proposal to it. Usually it is a combination of the two. You start a conversation on the mailing list, and then at some point file a JIRA to track the work and capture the

Hadoop on non x86 systems

Hi all, My team is actively involve in porting Hadoop 1.0.x to IBM POWER architecture, that of course includes building Hadoop 1.0 using IBM JAVA 6. So to start I have following questions: 1) How hadoop-common supports multi architecture builds? If it doesn't , we have a proposal, what is the

RE: Looking for a place to start

I suggest you get a copy of Hadoop, The Definitive Guide by Tom White. I found it very informative when I was first starting out. As for sample "big data" , the book uses weather data from the NCDC. You can download it from https://github.com/tomwhite/hadoop-book/tree/master/input/ncdc/all D

Build failed in Jenkins: Hadoop-Common-0.23-Build #184

See Changes: [szetszwo] svn merge -c 1297328 from trunk for HDFS-3032. [acmurthy] Merge -c 1297310 from trunk to branch-0.23 to fix MAPREDUCE-3034. Ensure NodeManager reboots itself on direction from ResourceManager. Contribu

Build failed in Jenkins: Hadoop-Common-trunk #338

See Changes: [szetszwo] HDFS-3032. Change DFSClient.renewLease() so that it only retries up to the lease soft-limit. Contributed by Kihwal Lee [acmurthy] MAPREDUCE-3034. Ensure NodeManager reboots itself on direction from Resourc

Build failed in Jenkins: Hadoop-Common-0.23-PB-Build #13

See -- [...truncated 12383 lines...] [javadoc] Loading source files for package org.apache.hadoop.io... [javadoc] Loading source files for package org.apache.hadoop.io.compress... [javadoc