Re: Re: How does Spark honor data locality when allocating computing resources for an application

2015-03-15 Thread bit1...@163.com
se). Thanks. bit1...@163.com From: eric wong Date: 2015-03-14 22:36 To: bit1...@163.com; user Subject: Re: How does Spark honor data locality when allocating computing resources for an application you seem like not to note the configuration varible "spreadOutApps" And it's comme

Re: How does Spark honor data locality when allocating computing resources for an application

2015-03-14 Thread eric wong
you seem like not to note the configuration varible "spreadOutApps" And it's comment: // As a temporary workaround before better ways of configuring memory, we allow users to set // a flag that will perform round-robin scheduling across the nodes (spreading out each app // among all the node

How does Spark honor data locality when allocating computing resources for an application

2015-03-13 Thread bit1...@163.com
Hi, sparkers, When I read the code about computing resources allocation for the newly submitted application in the Master#schedule method, I got a question about data locality: // Pack each app into as few nodes as possible until we've assigned all its cores for (worker <- workers if worker.c