Hi, I am not clear on the use of weights in a non-least squares problem is. In a least-squares problem the weights can simply be taken in by the user in their input function. So the weights option is a convenience feature, any non-standard matrix weights could be programmed in by the user, if they need to. I have a hard time imagining a case when the weights are non-diagnal. I think your idea is good and you should go on with it. The quadratic memory scaling with the number of data points is not acceptable.
-Konstantin On Dec 28, 2012, at 9:01 AM, Luc Maisonobe <l...@spaceroots.org> wrote: > Hi all, > > I have encountered a major problem with version 3.1 just released. > This problem occurs in the revamped optimizers. One of the > OptimizationData implementations we can pass to multivariate vector > optimizers is the weight of the various observations. > > In all cases I can think of, this weight is a vector, but it is in fact > stored as a matrix in which only the diagonal is set. It's main > constructor uses a simple double array (which is used in all test cases > and is also used under the hood by curve fitters), and some optimizers > also assume only the diagonal is set (for example the Gauss-Newton > optimizer extract only the diagonal elements and ignores the other elements. > > However, it is also possible to set a non-diagonal weight matrix, and > one class (AbstractLeastSquaresOptimizer) performs an eigen dcomposition > on the matrix to extract its square root. I don't know any use case for > this, but it has been set up this way, so I guess someone has a use for > non-diagonal weights. > > However, merging all cases (vector and matrix) into one (despite the > fact Gauss-Newton ignores non-diagonal elements) leads to huge problems. > I discovered this when attempting a simple polynomial curve fitting on a > large number of points (41201 points). In this case, a full weight > matrix with about 1.7 billions entries was created, and copied... I > ended up with a memory error for something that should have been > completely trivial (fitting a polynomial, when in fact all weights were > equal to 1.0)! So for now, the new curve fitter is really impossible to > use with a large number of points. > > So I would like to be able to cope with vector-only weight easily. As I > don't understand if matrix weight are useful or not, I have simply added > a new WeightVector class implementing OptimizationData and added it to > the various parseOptimizationData, while preserving the former matrix > weights class. This does work well and changing the optimizers to > directly use a vector was really straightforward as in fact the weights > are not really used as a matrix (except in the strange eigen > decomposition and associated Jacobian multiplication of > AbstractLeastSquaresOptimizer). > > I wonder if I should simply add this as is or if we should rather remove > the non-diagonal weights feature and support only vector weights. > > I also wonder if we should release a bug fix version early. > > best regards, > Luc > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
