LinearRegressionWithSGD requires to tune the step size and # of iteration very carefully. Please try Linear Regression with elastic net implementation in Spark 1.4 in ML framework, which uses quasi newton method and step size will be automatically determined. That implementation also matches the result from R.
Sincerely, DB Tsai ------------------------------------------------------- Blog: https://www.dbtsai.com On Wed, May 27, 2015 at 9:08 PM, Maheshakya Wijewardena <[email protected]> wrote: > > Hi, > > I'm trying to use Sparks' LinearRegressionWithSGD in PySpark with the > attached dataset. The code is attached. When I check the model weights > vector after training, it contains `nan` values. > > [nan,nan,nan,nan,nan,nan,nan,nan] > > But for some data sets, this problem does not occur. What might be the > reason for this? > Is this an issue with the data I'm using or a bug? > > Best regards. > > -- > Pruthuvi Maheshakya Wijewardena > Software Engineer > WSO2 Lanka (Pvt) Ltd > Email: [email protected] > Mobile: +94711228855 > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
