Hello Zoltan, I'm a spark beginner but i think that the locality preferences should be prepared before the sending of tasks. One important element of a RDD is the metadata on the scheme and location of its partitions. Tasks created in the driver program should be based on this information.
I'm also interested by a detailed answer of this question. Could someone please provide a few more hints on it ? Regards, Hui On Thu, Mar 19, 2015 at 11:20 AM, Zoltán Zvara <zoltan.zv...@gmail.com> wrote: > I'm trying to understand the task scheduling mechanism of Spark, and I'm > curious about where does locality preferences get evaluated? I'm trying to > determine if locality preferences are fetchable before the task get > serialized. A HintSet would be most appreciated! > > Have nice day! > > Zvara Zoltán > > > > mail, hangout, skype: zoltan.zv...@gmail.com > > mobile, viber: +36203129543 > > bank: 10918001-00000021-50480008 > > address: Hungary, 2475 Kápolnásnyék, Kossuth 6/a > > elte: HSKSJZ (ZVZOAAI.ELTE) > -- Hui WANG Tel : +33 (0) 6 71 33 45 39 Blog : http://www.hui-wang.info
