[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/27 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36433891 I've merged this into master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36277558 I see two issues: 1) The deterministic nature of the current scheduler places tasks on the same small set of machines while leaving others largely unused; 2) There is n

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36275312 @shivaram I understand your cautiousness and I agree with Kay on that we would be careful when adding the complexity to the already-complex code base. So, I don't mind cl

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36271766 IMHO it would be better to try the simpler solution first and see how it works -- With a large enough number of executors I think the probability of seeing the same execut

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36238278 en...it's much simpler...but randomization can just mitigate the issue with some probability? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36224189 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36224190 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12912/ --- If your project i

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36221897 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/27#issuecomment-36221889 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-979] Randomize order of offers.

GitHub user kayousterhout opened a pull request: https://github.com/apache/spark/pull/27 [SPARK-979] Randomize order of offers. This commit randomizes the order of resource offers to avoid scheduling all tasks on the same small set of machines. This is a much simpler sol