Thank you Anurag Verma for replying. I tried increasing the iterations.
However I still get underfitted results. I am checking the model's
prediction by seeing how many pairs of labels and predictions it gets right
data_predict_with_model=best_model.transform(data_test_df)
final_pred_df=data_predict_with_model.select(col('label'),col('prediction'))
ans=final_pred_df.map(lambda x:((x[0],x[1]),1)).reduceByKey(lambda
a,b:a+b).toDF()
ans.show()
---------+---+
| _1| _2|
+---------+---+
|[1.0,1.0]| 5|
|[0.0,1.0]| 12|
+---------+---+
Do you know any other methods by which I can check the model? and what is it
that I am doing wrong. I have filtered the data and arranged it in a
features and label column. So now only the model creation part is wrong I
guess. Can anyone help me please. I am still learning machine learning.
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Regularized-Logistic-regression-tp19432p19443.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: [email protected]
