回复：Re: Problems with spark.locality.wait

ty. Thanks for your guy's help. --Ma Chong - 原始邮件 - 发件人：Mridul Muralidharan 收件人：Kay Ousterhout 抄送人：MaChong , dev 主题：Re: Problems with spark.locality.wait 日期：2014年11月14日 04点53分

Re: Re: Problems with spark.locality.wait

t; >> Kay's comment, IMO, is slightly general in nature - and I suspect >> unless we overhaul how preferred locality is specified, and allow for >> taskset specific hints for schedule, we cant resolve that IMO. >> >> >> Regards, >> Mridul >> >> >> >

Problems with spark.locality.wait

Hi, We are running a time sensitive application with 70 partition and 800MB each parition size. The application first load data from database in different cluster, then apply a filter, cache the filted data, then apply a map and a reduce, finally collect results. The application will be finishe