For the record, this is what I came up with (ignoring the configurable port
for now):
spark/sbin/start-master.sh
master_ui_response_code=0
while [ "$master_ui_response_code" -ne 200 ]; do
sleep 1
master_ui_response_code="$(
curl --head --silent --output /dev/null \
--
Oh, good point. So I guess I should be able to query the master via code
like this before any slaves are started.
On Sat, Apr 11, 2015 at 7:52 PM Ted Yu wrote:
> From SparkUI.scala :
>
> def getUIPort(conf: SparkConf): Int = {
> conf.getInt("spark.ui.port", SparkUI.DEFAULT_PORT)
> }
> Be
>From SparkUI.scala :
def getUIPort(conf: SparkConf): Int = {
conf.getInt("spark.ui.port", SparkUI.DEFAULT_PORT)
}
Better retrieve effective UI port before probing.
Cheers
On Sat, Apr 11, 2015 at 2:38 PM, Nicholas Chammas <
nicholas.cham...@gmail.com> wrote:
> So basically, to tell if t
Yeah thats the best I can think ok -- Not sure if there is a better way to
do it.
On Sat, Apr 11, 2015 at 2:38 PM, Nicholas Chammas <
nicholas.cham...@gmail.com> wrote:
> So basically, to tell if the master is ready to accept slaves, just poll
> http://master-node:4040 for an HTTP 200 response?
>
So basically, to tell if the master is ready to accept slaves, just poll
http://master-node:4040 for an HTTP 200 response?
On Sat, Apr 11, 2015 at 2:42 PM Shivaram Venkataraman <
shiva...@eecs.berkeley.edu> wrote:
> Yeah from what I remember it was set defensively. I don't know of a good
> way
Yeah from what I remember it was set defensively. I don't know of a good
way to check if the master is up though. I guess we could poll the Master
Web UI and see if we get a 200/ok response
Shivaram
On Fri, Apr 10, 2015 at 8:24 PM, Nicholas Chammas <
nicholas.cham...@gmail.com> wrote:
> Check th
