Just subscribing to this list, so apologies for quoting weirdly and any other etiquette offenses.
DB Tsai wrote > Hi Deb, > > I had tried breeze L-BFGS algorithm, and when I tried it couple weeks > ago, it's not as stable as the fortran implementation. I guessed the > problem is in the line search related thing. Since we may bring breeze > dependency for the sparse format support as you pointed out, we can > just try to fix the L-BFGS in breeze, and we can get OWL-QN and > L-BFGS-B. > > What do you think? I'm happy to help fix any problems. I've verified at points that the implementation gives the exact same sequence of iterates for a few different functions (with a particular line search) as the c port of lbfgs. So I'm a little surprised it fails where Fortran succeeds... but only a little. This was fixed late last year. OWL-QN seems to mostly be stable, but probably deserves more testing. Presumably it has whatever defects my LBFGS does. (It's really pretty straightforward to implement given an L-BFGS) We don't provide an L-BFGS-B implementation. We do have a more general constrained qn method based on http://jmlr.org/proceedings/papers/v5/schmidt09a/schmidt09a.pdf (which uses a L-BFGS type update as part of the algorithm). From the experiments in their paper, it's likely to not work as well for bound constraints, but can do things that lbfgsb can't. Again, let me know what I can help with. -- David Hall On Mon, Mar 3, 2014 at 3:52 PM, DB Tsai <dbtsai@> wrote: > Hi Deb, > >> a. OWL-QN for solving L1 natively in BFGS > Based on what I saw from > https://github.com/tjhunter/scalanlp-core/blob/master/learn/src/main/scala/breeze/optimize/OWLQN.scala > , it seems that it's not difficult to implement OWL-QN once LBFGS is > done. > >> >> b. Bound constraints in BFGS : I saw you have converted the fortran >> code. >> Is there a license issue ? I can help in getting that up to speed as >> well. > I tried to convert the code from Fortran L-BFGS-B implementation to > java using f2j; the translated code is just a messy, and it just > doesn't work at all. There is no license issue here. Any idea about > how to approach this? > >> c. Few variants of line searches : I will discuss on it. >> For the dbtsai-lbfgs branch seems like it already got merged by Jenkins. > I don't think it's merged into master. Still have couple things needed > to be cleaned up. Just open the PR to have public feedback. > >> Is this getting merged to the master or there will be revisions on it ? >> >> https://github.com/apache/spark/pull/53 >> >> Thanks. >> Deb > > Sincerely, > > DB Tsai > Machine Learning Engineer > Alpine Data Labs > -------------------------------------- > Web: http://alpinenow.com/ -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/MLLib-Thoughts-about-refactoring-Updater-for-LBFGS-tp2493p3935.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
