[GitHub] spark pull request: Add role and checkpoint support for Mesos back...
GitHub user iven opened a pull request: https://github.com/apache/spark/pull/60 Add role and checkpoint support for Mesos backend This commit also changes Mesos version to 0.14.0, which is the earliest version supporting these features. You can merge this pull request into a Git repository by running: $ git pull https://github.com/iven/spark master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/60.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #60 commit 6aad808464d5cdc8e1c8729f5befcb8636e685b6 Author: Xu Lijian Date: 2014-02-26T04:26:49Z Add role and checkpoint support for Mesos backend This commit also changes Mesos version to 0.14.0, which is the earliest version supporting these features. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Add role and checkpoint support for Mesos back...
Github user iven commented on the pull request: https://github.com/apache/spark/pull/60#issuecomment-36484214 If checkpoint is enabled both in spark and Mesos slaves, the slaves will write spark and executors information periodically to the disk. Yes, it's used for Spark to survive slave restarts. The reason why it's not enabled by default is, if checkpoint is enabled, spark will only receive offers from checkpointing slaves. See here for details: http://mesos.apache.org/documentation/latest/slave-recovery/ Hadoop and Storm framework both get it disabled by default: https://github.com/mesos/hadoop/blob/master/configuration.md https://github.com/mesosphere/storm-mesos/blob/master/storm.yaml --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Add role and checkpoint support for Mesos back...
Github user iven commented on the pull request: https://github.com/apache/spark/pull/60#issuecomment-36586768 Done. Please correct me if there's any mistake in my pool English. :D --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Add role and checkpoint support for Mesos back...
Github user iven commented on the pull request: https://github.com/apache/spark/pull/60#issuecomment-37156504 Sorry, I was a little busy last week. I'll push new version today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: Add role and checkpoint support for Mesos back...
Github user iven commented on the pull request: https://github.com/apache/spark/pull/60#issuecomment-37287900 Sorry for late. There's something wrong with my cluster, and I'm afraid the code here is not tested right now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
