Paul Gilbert wrote:
> Peter Dalgaard wrote:
> ....
>
>> Well, there's make check-all...
>>
> Ok, this is what I should be running. It reports
> ....
> running tests of LAPACK-based functions
> make[3]: Entering directory `/home/mfa/gilp/toolchain/R/src/R-beta/tests'
> running code in 'lapack.R' ...make[3]: *** [lapack.Rout] Error 1
> make[3]: Leaving directory `/home/mfa/gilp/toolchain/R/src/R-beta/tests'
> make[2]: *** [test-Lapack] Error 2
> make[2]: Leaving directory `/home/mfa/gilp/toolchain/R/src/R-beta/tests'
> make[1]: *** [test-all-devel] Error 1
> make[1]: Leaving directory `/home/mfa/gilp/toolchain/R/src/R-beta/tests'
> make: *** [check-all] Error 2
> ~/toolchain/R/src/R-beta:
> and lapack.Rout.fail reports
> ....
> > ## failed for some 64bit-Lapack-gcc combinations:
> > sm <- cbind(1, 3:1, 1:3)
> > eigenok(sm, eigen(sm))
> Error: abs(A %*% V - V %*% diag(lam)) < Eps is not all TRUE
> Execution halted
>
>> That doesn't check everything either, though. The point is that there
>> is a limit to what we can check: We don't check whether the CPU gets
>> floating point operations wrong in the 5th decimal place on rare
>> occasions either. ...
>>
> Well, I'm talking about 3 to 4% in the maximum eigenvalue, in a problem
> that I don't think is especially ill-conditioned.
You didn't feel the "whoosh" as the reference to the 1994 Pentium FDIV
bug went right past you? The magnitude of the error was never the point,
rather the point was that we need to assume that most things "just
works" -- that log() computes logarithms, that floating point divisions
are accurate up to round off, etc., but also that linear algebra
libraries do what they are expected to do. We check as much as we can,
including comparisons of standard routines with expected output in many
cases, but some inconsistencies need a targeted test, and those don't
get written without a specific suspicion.
> There are not many
> more fundamental calculations in statistics. And all I am asking is
> that the problem gets reported, I'm not asking for a work around. I
> clearly need to fix something other than R on the system.
>
>
Yup.
> I guess it is difficult to know at what level problems should be
> flagged. I don't think many people run make check-all. Some
> calculation errors seem bad enough that plain "make" should catch them
> and not suggest there was a successful build. It would do R's
> reputation a lot of damage if people used and reported the results of
> such bad calculations.
>
It is a bit like Brian's famous "fortune" entry about writing and
reading documentation. We do provide the tools, but developers would go
nuts if they had to run make check-all every time they fixed a typo, so
it is the testers' obligation to, well, run the tests. One might hope
that your example demonstrates to anyone else reading this thread, that
relying on others to have done the testing for you can be dangerous,
especially if your setup is out of the mainstream. (And as you found out
the hard way, the "ultrastable" enterprise editions tend to be just
that: Only largish shops install them, and on top of that, they tend to
have somewhat dated toolchains.)
> It might be nice to have an R package (called something like
> integrityCheck) that runs several numerical checks. This would allow end
> users to take some responsibility for ensuring their system is built
> properly. I'm worried about the situation where a sys-admin installs R
> and does not do any testing.
>
Hmm, a make dependency of "install" on "check-all" is actually possible.
Not quite sure how that would be received, though.
-pd
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
