On Mon, Dec 03, 2012 at 11:22:21AM -0500, Konstantin Berlin wrote: > Hi, > > In my view there are two related but separate issues: (1) What should we > do for 3.1 release, and (2) How do we want to design the optimization > package for the future, such that it is easily extended to linear and > non-linear constraints, sparse matrix operations, etc. > > In regards to (1), I feel that designing a proper API for the future is a > very important problem that would take a while, to not only implement and > test, but to also come to an agreement. This future API should not be > limited by the backwards capability with 3.0, but should instead be > careful designed from the ground up. That is why I propose that we do not > let these issues set back the much needed release of 3.1. For example in > 3.0 CMAES bound constraints are actually broken. This should be patched > ASAP.
??? Did you report this issue on the bug tracking system? > There are new features. While my combined function issue is > important, it is a new feature, and by no means critical. It is not a new issue. It is an improvement. It is easy to implement. > In my view it > is better to leave the Function classes as they are (or at least similar > to what they are), so the user is comfortable with the interface. Nothing will be removed as 3.1 must be backward-compatible. It is however the right time to introduce changes as it gives users the opportunity to switch to the new features. Even if what I proposed is not the end of the story, it's fine to make the change now. > In regards to (2), Here are some things that we need to think about 1) How > to properly design the function interfaces for the optimization packages, > such that they address my combined value and gradient computation, but > also have proper OO structure such that a function that is used in a > "value+gradient" optimization can also be used for a no-gradient direct > optimization? We've just been thinking about that in this quite extensive thread. Luc has changed the "...Function" interfaces (introducing "DerivativeStructure"). You have convinced us to that this should not be reused into the "optimization" package. That pretty much settles the issue for now IMHO. Of course, that does not preclude further discussion and further changes. > What wrapper classes should we provide for the user? > Should this functions be also comparable with analysis package (or some > other package), or would optimization performance suffer? How should > these values be stored so they can be efficiently used by the optimizer > without significant amount of memory copies. These will be answered either if users provide concret use-cases for testing. Up to now, most answers to similar questions came from regular contributors having their user hat on. ;-) > 2) Changing the return class > for "optimization" function. Like I mentioned before, I don't think the > optimization class should keep state on the previous optimization. All > RMS, chi^2 values are part of the optimization result and not part of the > optimizer. An example of such use is where a user computed values from > different initial points and then compares results. Chi2 values are only used (and subsequently made accessible to the caller) in the "AbstractLeastSquaresOptimizer" hierarchy. In the current design, it would not make any sense to just change the return value type and force every algorithm to fill in data they don't use. Discussion is open but should be based on actual cases, or on patches that demonstrate how the change is applied to all the implementations which are supposed to be affected. > 3) It might *not* be > a good idea to have constraints be given as part of options into an > "optimization" function. First, different implementations have different > limitations, so one method might take non-linear constraints, another > doesn't. This should be explicitly visible through the constructors of > the class. Second, linear constraints,and probably other constrains > require pre-processing before use. For example, if you have linear > equality constraints in an optimization, you need to extract the null > space using QR or SVD. If you have a lot of constraints this process is > expensive, and should be amortized for calls with same constrains but > different initial points. There are probably other examples of this. We > should think and discuss this, before jumping to a design. Personally, I don't know what you are talking about: I can only assume that you refer to the "linear" sub-package. Only some of the other optimizers support constraints, and as of today, only "simple bounds" (parameter value constrained into an interval). We would be quite pleased to count an optimization expert among the contributors, who would provide the feature you mention, but as I indicated previously, Commons Math has a serious shortage of permanent staff. :-D > It is very unfortunate that there is a lack of man power to improve the > algorithms. I am very sad to say, but I have barely time to even reply to > these emails. In practice, much time is wasted in discussions, even on things that regular contributors want to implement. In practice, regular contributors introduce new code because they need it in their applications, or because they identified some bug or weakness. In practive, major design overhauls occur because new use-cases uncover a previously a weakness (that did not get in the way of previous use-cases or which users did not report). > I look forward to hearing what other people think, Yes that would be intersting. But maybe they don't have the time either, and that should not prevent those who participated in this discussion to go forward. Regards, Gilles --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
