Le 28/12/2012 17:51, Konstantin Berlin a écrit : > Hi, > > I can understand Dimitri's frustration, it seems the optimization > framework gets worse with every iteration. However, we should > probably look forward and think about how to design it properly > instead. > > Several times I brought out some problems and ideas about the package > and it seems the only person who has an opinion is Gilles.
Several people contributed to the thread (see <http://commons.markmail.org/thread/i6klmc2ytflb6qnt>), but as Gilles pointed out in one of the message, we lack an optimization expert. I sincerely would not want my opinion to be taken too seriously on this, so I only expressed what I could and did not decide anything by myself (only proposed to remove the wrong binding with DerivativeStructure, which has since been done). > > I will list what I consider to be major problems > 1) The OO design is bad, too much inheritance which could be handled > better by interfaces, the structure has no relation to the actual way > parts of optimizers can be mixed and matched. Input functions should > also have clear inheritance structure and should return both the > value, gradient, hessian in one function call. I strongly agree with Konstantin here. Abstract classes allow to add methods without breaking compatibility (which is good), but they also have some drawbacks (we have seen one drawback with the parameterized classes a few months ago, and this huge hierarchy is another one). So there is no silver bullet and we keep trying to find the good balance. As far as I am concerned, I would prefer we get fewer abstract classes, we remove some intermediate level (I don't know which ones), and we use more delegation than inheritance. > 2) Incorrect handling of constraints. There are only something like 5 > possible constraints possible in optimization, with each > implementation of the solver handling some but not all. There is no > need to this runtime approach, which creates incredible amount of > confusion. All the possible constraints should be explicitly given in > the parameters to a function call, there are only 5. In addition, > constraints should be pre-processed a priori! So they should be an > input to the constructor not to the optimization function call. Our implementation for constraints is really limited (once again, scarce resources). What are the 5 types you consider? Simple/double bounds on parameters, linear/non-linear bounds and equality? > 3) Linear algebra package should be used as an interface and > internally to increase performance for larger datasets. Data copying > should be avoided as much as possible. Yes, but this would require solving another sub-problem first: having a decent sparse linear algebra implementation, which we also lack. Our implementation for full matrices was also in a sorry state prior to 3.0, but now fortunately this has improved at least for systems up to a few thousands rows and columns (so at least we do make progress on some points). > 4) Testing should be done on larger problems. Yes. I guess there are some general well known problems for that, so we should get a few of them and implement them. We did implement a number of tests from Minpack, but they focused on difficult cases rather than on problem size. I think optimization has a good testing coverage, but clearly large size problems is a needed addition. > > I know the response is that I am free to go implemented, but I think > we should at least agree on design principles instead of pushing > through your own ideas because the other person is too busy. The only > discussion we ever had on this was between me and Gilles, everyone > else disappeared. Well, we tried to keep up as our skills allowed, and we were also concerned with 3.1 being released at the same time. We have more time now than we had a few weeks ago. This is an opportunity to restart the discussion. We can refrain from pushing a new release (despite I would like this bug fix to be released officially) and take some time to think calmly. We could also push 3.2 with only the fix and without any revamp and start thinking about 4.0 with a redesign of these two main area: optimization and sparse linear algebra. If you could contribute to this discussion understanding we are not experts of this field and we cannot do it by ourselves, it would be great. best regards, Luc > > Thanks, Konstantin > > On Dec 28, 2012, at 11:27 AM, Phil Steitz <phil.ste...@gmail.com> > wrote: > >> On 12/28/12 8:12 AM, Dimitri Pourbaix wrote: >>> Luc, >>> >>>> So in order to make sure I understand your point, you would be >>>> OK if I deprecate the non-diagonal weights, in which case users >>>> needing this would have to implement it themselves by >>>> premultiplication (as both you and Konstantin seem to >>>> propose)? >>> >>> Yes, exactly. >>> >>>> Sure, but for the record the feature was also a last minute >>>> change. This was discussed on the list, and the final decision >>>> was to add this feature despite the release was close. No >>>> wonder we failed to test it thoroughsly. >>> >>> Last minute? I have been discussing this with Gilles for >>> several months. >> >> Relevant project discussion happens *on this list* >>> >>>> We don't expect our releases to be perfect. We do our best, >>>> with the resources we have. >>> >>> I perfectly understand this but focusing those resources less on >>> rules and more on real cases might help. >> >> As stated before, you are more than welcome to *become* one of >> these resources. >> >> Phil >>> >>> Regards, Dim. >>> ---------------------------------------------------------------------------- >>> >>> >>> Dimitri Pourbaix * Don't worry, be happy >>> Institut d'Astronomie et d'Astrophysique * and CARPE >>> DIEM. CP 226, office 2.N4.211, building NO * Universite Libre >>> de Bruxelles * Tel : +32-2-650.35.71 Boulevard du >>> Triomphe * Fax : +32-2-650.42.26 B-1050 >>> Bruxelles * NAC: HBZSC RG2Z6 >>> http://sb9.astro.ulb.ac.be/~pourbaix * >>> mailto:pourb...@astro.ulb.ac.be >>> >>> --------------------------------------------------------------------- >>> >>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>> For additional commands, e-mail: dev-h...@commons.apache.org >>> >>> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >> For additional commands, e-mail: dev-h...@commons.apache.org >> > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
