On 12/28/12 9:58 AM, Luc Maisonobe wrote: > Le 28/12/2012 17:51, Konstantin Berlin a écrit : >> Hi, >> >> I can understand Dimitri's frustration, it seems the optimization >> framework gets worse with every iteration. However, we should >> probably look forward and think about how to design it properly >> instead. >> >> Several times I brought out some problems and ideas about the package >> and it seems the only person who has an opinion is Gilles. > Several people contributed to the thread (see > <http://commons.markmail.org/thread/i6klmc2ytflb6qnt>), but as Gilles > pointed out in one of the message, we lack an optimization expert. I > sincerely would not want my opinion to be taken too seriously on this, > so I only expressed what I could and did not decide anything by myself > (only proposed to remove the wrong binding with DerivativeStructure, > which has since been done). > >> I will list what I consider to be major problems >> 1) The OO design is bad, too much inheritance which could be handled >> better by interfaces, the structure has no relation to the actual way >> parts of optimizers can be mixed and matched. Input functions should >> also have clear inheritance structure and should return both the >> value, gradient, hessian in one function call. > I strongly agree with Konstantin here. Abstract classes allow to add > methods without breaking compatibility (which is good), but they also > have some drawbacks (we have seen one drawback with the parameterized > classes a few months ago, and this huge hierarchy is another one). So > there is no silver bullet and we keep trying to find the good balance. > As far as I am concerned, I would prefer we get fewer abstract classes, > we remove some intermediate level (I don't know which ones), and we use > more delegation than inheritance. > >> 2) Incorrect handling of constraints. There are only something like 5 >> possible constraints possible in optimization, with each >> implementation of the solver handling some but not all. There is no >> need to this runtime approach, which creates incredible amount of >> confusion. All the possible constraints should be explicitly given in >> the parameters to a function call, there are only 5. In addition, >> constraints should be pre-processed a priori! So they should be an >> input to the constructor not to the optimization function call. > Our implementation for constraints is really limited (once again, scarce > resources). What are the 5 types you consider? Simple/double bounds on > parameters, linear/non-linear bounds and equality? > >> 3) Linear algebra package should be used as an interface and >> internally to increase performance for larger datasets. Data copying >> should be avoided as much as possible. > Yes, but this would require solving another sub-problem first: having a > decent sparse linear algebra implementation, which we also lack. Our > implementation for full matrices was also in a sorry state prior to 3.0, > but now fortunately this has improved at least for systems up to a few > thousands rows and columns (so at least we do make progress on some points). > >> 4) Testing should be done on larger problems. > Yes. I guess there are some general well known problems for that, so we > should get a few of them and implement them. We did implement a number > of tests from Minpack, but they focused on difficult cases rather than > on problem size. I think optimization has a good testing coverage, but > clearly large size problems is a needed addition. > >> I know the response is that I am free to go implemented, but I think >> we should at least agree on design principles instead of pushing >> through your own ideas because the other person is too busy. The only >> discussion we ever had on this was between me and Gilles, everyone >> else disappeared. > Well, we tried to keep up as our skills allowed, and we were also > concerned with 3.1 being released at the same time. > > We have more time now than we had a few weeks ago. This is an > opportunity to restart the discussion. We can refrain from pushing a new > release (despite I would like this bug fix to be released officially) > and take some time to think calmly. We could also push 3.2 with only the > fix
What about 3.1.1 with just the fix for this, then possibly 3.2 or direct to 4.0. Phil > and without any revamp and start thinking about 4.0 with a redesign > of these two main area: optimization and sparse linear algebra. > > If you could contribute to this discussion understanding we are not > experts of this field and we cannot do it by ourselves, it would be great. > > best regards, > Luc > >> Thanks, Konstantin >> >> On Dec 28, 2012, at 11:27 AM, Phil Steitz <phil.ste...@gmail.com> >> wrote: >> >>> On 12/28/12 8:12 AM, Dimitri Pourbaix wrote: >>>> Luc, >>>> >>>>> So in order to make sure I understand your point, you would be >>>>> OK if I deprecate the non-diagonal weights, in which case users >>>>> needing this would have to implement it themselves by >>>>> premultiplication (as both you and Konstantin seem to >>>>> propose)? >>>> Yes, exactly. >>>> >>>>> Sure, but for the record the feature was also a last minute >>>>> change. This was discussed on the list, and the final decision >>>>> was to add this feature despite the release was close. No >>>>> wonder we failed to test it thoroughsly. >>>> Last minute? I have been discussing this with Gilles for >>>> several months. >>> Relevant project discussion happens *on this list* >>>>> We don't expect our releases to be perfect. We do our best, >>>>> with the resources we have. >>>> I perfectly understand this but focusing those resources less on >>>> rules and more on real cases might help. >>> As stated before, you are more than welcome to *become* one of >>> these resources. >>> >>> Phil >>>> Regards, Dim. >>>> ---------------------------------------------------------------------------- >>>> >>>> >>>> > Dimitri Pourbaix * Don't worry, be happy >>>> Institut d'Astronomie et d'Astrophysique * and CARPE >>>> DIEM. CP 226, office 2.N4.211, building NO * Universite Libre >>>> de Bruxelles * Tel : +32-2-650.35.71 Boulevard du >>>> Triomphe * Fax : +32-2-650.42.26 B-1050 >>>> Bruxelles * NAC: HBZSC RG2Z6 >>>> http://sb9.astro.ulb.ac.be/~pourbaix * >>>> mailto:pourb...@astro.ulb.ac.be >>>> >>>> --------------------------------------------------------------------- >>>> >>>> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>>> For additional commands, e-mail: dev-h...@commons.apache.org >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> >>> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >>> For additional commands, e-mail: dev-h...@commons.apache.org >>> >> >> --------------------------------------------------------------------- >> >> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org >> For additional commands, e-mail: dev-h...@commons.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
