jackyoh created SPARK-18935:
-------------------------------
Summary: Use Mesos "Dynamic Reservation" resource for Spark
Key: SPARK-18935
URL: https://issues.apache.org/jira/browse/SPARK-18935
Project: Spark
Issue Type: Bug
Affects Versions: 2.0.2, 2.0.0
Reporter: jackyoh
I'm running spark on Apache Mesos
Please follow these steps to reproduce the issue:
1. First, run Mesos resource reserve:
curl -i -d slaveId=c24d1cfb-79f3-4b07-9f8b-c7b19543a333-S0 -d
resources='[{"name":"cpus","type":"SCALAR","scalar":{"value":20},"role":"spark","reservation":{"principal":""}},{"name":"mem","type":"SCALAR","scalar":{"value":4096},"role":"spark","reservation":{"principal":""}}]'
-X POST http://192.168.1.118:5050/master/reserve
2. Then run spark-submit command:
./spark-submit --class org.apache.spark.examples.SparkPi --master
mesos://192.168.1.118:5050 --conf spark.mesos.role=spark
../examples/jars/spark-examples_2.11-2.0.2.jar 10000
And the console will keep loging same warning message as shown below:
16/12/19 22:33:28 WARN TaskSchedulerImpl: Initial job has not accepted any
resources; check your cluster UI to ensure that workers are registered and have
sufficient resources
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
