Hi Vincent
I am not sure whether you are asking me or Nicolas. If me, then no we
didn't. Never used Akka and wasn't even aware that it has such
capabilities. Using Java API so we don't have Akka as a dependency right
now.
On Tue, Oct 18, 2016 at 12:47 PM, vincent gromakowski <
vincent.gromakow...
Hi
Did you try applying the model with akka instead of spark ?
https://spark-summit.org/eu-2015/events/real-time-anomaly-detection-with-spark-ml-and-akka/
Le 18 oct. 2016 5:58 AM, "Aseem Bansal" a écrit :
> @Nicolas
>
> No, ours is different. We required predictions within 10ms time frame so
> w
@Nicolas
No, ours is different. We required predictions within 10ms time frame so we
needed much less latency than that.
Every algorithm has some parameters. Correct? We took the parameters from
the mllib and used them to create ml package's model. ml package's model's
prediction time was much fa
Hi Sean and Aseem,
thanks both. A simple thing which sped things up greatly was simply to load
our sql (for one record effectively) directly and then convert to a
dataframe, rather than using Spark to load it. Sounds stupid, but this took
us from > 5 seconds to ~1 second on a very small instance.
Hi
Faced a similar issue. Our solution was to load the model, cache it after
converting it to a model from mllib and then use that instead of ml model.
On Tue, Oct 11, 2016 at 10:22 PM, Sean Owen wrote:
> I don't believe it will ever scale to spin up a whole distributed job to
> serve one reque
I don't believe it will ever scale to spin up a whole distributed job to
serve one request. You can look possibly at the bits in mllib-local. You
might do well to export as something like PMML either with Spark's export
or JPMML and then load it into a web container and score it, without Spark
(pos
Hi all,
so I have a model which has been stored in S3. And I have a Scala webapp
which for certain requests loads the model and transforms submitted data
against it.
I'm not sure how to run this quickly on a single instance though. At the
moment Spark is being bundled up with the web app in an ub
