Thanks Harsh. I have a few more questions.
Q1: I found it in my experiments using CS that for any user , its next job does not start until its current one is finished. Is it true and are there any exceptions and if true then why is it so? I I did not find any such condition in the implementation of CS. Q2: The concept of reserved slots is true only if speculative execution is on. Am i correct ? If yes,then the code dealing with reserved slots wont be executed if speculative execution is off? PS: I am working on MRv1. On Sun, Mar 3, 2013 at 2:41 AM, Harsh J <ha...@cloudera.com> wrote: > On Sun, Mar 3, 2013 at 1:41 PM, Jagmohan Chauhan < > simplefundumn...@gmail.com > > wrote: > > > Hi > > > > I am going through the Capacity Scheduler implementation. There is one > > thing i did not understand clearly. > > > > Are you reading the YARN CapacityScheduler or the older, MRv1 one? I'd > suggest reading the newer one for any implementation or research goals, for > it to be more current and future-applicable. > > > > 1. Does the o ff-switch task refers to a task in which data has to be > > fetched over the network. It means its not node-local ? > > > > Off-switch would imply off-rack, i.e. not node local, nor rack-local. > > > > 2. Does off-switch task includes only the tasks for which map input has > to > > be fetched from a node on a different rack across the switch or it also > > includes task where data has to be fetched from another node on same rack > > on same switch? > > > > A task's input split is generally supposed to define all locations of > available inputs. If the CS is unable to schedule to any of those > locations, nor their racks, then it schedules an off-rack (see above) task > which has to pull the input from a different rack. > > > > > > -- > > Thanks and Regards > > Jagmohan Chauhan > > MSc student,CS > > Univ. of Saskatchewan > > IEEE Graduate Student Member > > > > http://homepage.usask.ca/~jac735/ > > > > Feel free to post any further impl. related questions! :) > > -- > Harsh J > -- Thanks and Regards Jagmohan Chauhan MSc student,CS Univ. of Saskatchewan IEEE Graduate Student Member http://homepage.usask.ca/~jac735/
