Thanks.
The example I used is here
https://spark.apache.org/docs/latest/mllib-linear-methods.html see
SVMClassifier
So there's no way to get a probability based output? What about from
linear regression, or logistic regression?
On 19 October 2014 19:52, Sean Owen <so...@cloudera.com> wrote:
> The problem is that you called clearThreshold(). The result becomes the
> SVM margin not a 0/1 class prediction. There is no probability output.
>
> There was a very similar question last week. Is there an example out there
> suggesting clearThreshold()? I also wonder if it is good to overload the
> meaning of the output indirectly this way.
> On Oct 19, 2014 6:53 PM, "npomfret" <nick-nab...@snowmonkey.co.uk> wrote:
>
>> Hi, I'm new to spark and just trying to make sense of the SVMWithSGD
>> example. I ran my dataset through it and build a model. When I call
>> predict() on the testing data (after clearThreshold()) I was expecting to
>> get answers in the range of 0 to 1. But they aren't, all predictions seem
>> to be negative numbers between -0 and -2. I guess my question is what do
>> these predictions mean? How are they of use? The outcome I need is a
>> probability rather than a binary. Here's my java code: SparkConf conf = new
>> SparkConf() .setAppName("name") .set("spark.cores.max", "1");
>> JavaSparkContext sc = new JavaSparkContext(conf); JavaRDD points =
>> sc.textFile(path).map(new ParsePoint()).cache(); JavaRDD training =
>> points.sample(false, 0.8, 0L).cache(); JavaRDD testing =
>> points.subtract(training); SVMModel model =
>> SVMWithSGD.train(training.rdd(), 100); model.clearThreshold(); for
>> (LabeledPoint point : testing.toArray()) { Double score =
>> model.predict(point.features()); System.out.println("score = " +
>> score);//<- all these are negative numbers, seemingly between 0 and -2 }
>> ------------------------------
>> View this message in context: Using SVMWithSGD model to predict
>> <http://apache-spark-user-list.1001560.n3.nabble.com/Using-SVMWithSGD-model-to-predict-tp16767.html>
>> Sent from the Apache Spark User List mailing list archive
>> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>>
>
