: Mike Hynes; dev@spark.apache.org
Subject: Re: No speedup in MultiLayerPerceptronClassifier with increase in
number of cores
Hi Alexander,
Thanks for your reply.Actually I am working with a modified version of the
actual MNIST dataset ( maximum samples = 8.2 M)
https://www.csie.ntu.edu.tw/~cjlin
5 9:29 AM
> *To:* Mike Hynes
> *Cc:* dev@spark.apache.org; Ulanov, Alexander
> *Subject:* Re: No speedup in MultiLayerPerceptronClassifier with increase
> in number of cores
>
>
>
> Actually I have 5 workers running ( 1 per physical machine) as displayed
> by the spark UI on
worthwhile for this rather small
dataset.
Best regards, Alexander
From: Disha Shrivastava [mailto:dishu@gmail.com]
Sent: Sunday, October 11, 2015 9:29 AM
To: Mike Hynes
Cc: dev@spark.apache.org; Ulanov, Alexander
Subject: Re: No speedup in MultiLayerPerceptronClassifier with increase in
number of
Actually I have 5 workers running ( 1 per physical machine) as displayed by
the spark UI on spark://IP_of_the_master:7077. I have entered all the
physical machines IP in a file named slaves in spark/conf directory and
using the script start-all.sh to start the cluster.
My question is that is there
Having only 2 workers for 5 machines would be your problem: you
probably want 1 worker per physical machine, which entails running the
spark-daemon.sh script to start a worker on those machines.
The partitioning is agnositic to how many executors are available for
running the tasks, so you can't do
