Re: hadoop fair scheduler

2011-10-17 Thread Kihwal Lee
Look at the attachments section of the jira. The patch should contain the code. Kihwal On 10/17/11 1:37 PM, "shivam tiwari" wrote: thanks, I want to download Adaptive scheduler I found here but not able to figure out where I can download it. https://issues.apache.org/jira/browse/MAPREDUCE-1380

Re: hadoop fair scheduler

2011-10-17 Thread shivam tiwari
thanks, I want to download Adaptive scheduler I found here but not able to figure out where I can download it. https://issues.apache.org/jira/browse/MAPREDUCE-1380?focusedCommentId=13129066#comment-13129066 On Mon, Oct 17, 2011 at 1:27 PM, abhishek sharma wrote: > Hi Shivam, > > The following pa

Re: hadoop fair scheduler

2011-10-17 Thread abhishek sharma
Hi Shivam, The following paper by Zaharia et al. has design insights as well as lots of evaluation. http://www.cs.berkeley.edu/~matei/papers/2010/eurosys_delay_scheduling.pdf Abhishek On Mon, Oct 17, 2011 at 1:20 PM, Harsh J wrote: > Shivam, > > Here lies its inception with good reading stuff

Re: hadoop fair scheduler

2011-10-17 Thread Harsh J
Shivam, Here lies its inception with good reading stuff related to what you ask: https://issues.apache.org/jira/browse/HADOOP-3746 and the general description and usage documents may be found at: http://hadoop.apache.org/common/docs/current/fair_scheduler.html You can browse through the sources u

Re: cluster under-utilization with Hadoop Fair Scheduler

2010-04-11 Thread Todd Lipcon
rt are not going to be very efficient - even with fast assignment there is some constant overhead per task. Thanks -Todd On Sun, Apr 11, 2010 at 11:42 AM, abhishek sharma wrote: > Hi all, > > I have been using the Hadoop Fair Scheduler for some experiments on a > 100 node cluster wit

cluster under-utilization with Hadoop Fair Scheduler

2010-04-11 Thread abhishek sharma
Hi all, I have been using the Hadoop Fair Scheduler for some experiments on a 100 node cluster with 2 map slots per node (hence, a total of 200 map slots). In one of my experiments, all the map tasks finish within a heartbeat interval of 3 seconds. I noticed that the maximum number of