As far as questions #1 and #2 go: you can probably use the components
of R.Version() (e.g. $arch, maybe some substring of $os) to compare
your test output only for "sufficiently similar" platforms. Depending
on how obsessive you are you could generate different test output for
a bunch of platforms and include them all. Might be cleaner to store
the results in a separate file. For example, say that
inst/testdata/eigentests.RData contains a list of results - then
something like
eigen_results <- list(
x86_64_darwin= ...,
x86_64_linux = ...,
x86_64_windows= ...,
x86_32_darwin=...,
etc.)
load(system.file("testdata","eigentests.RData",package="mypkg"))
rv <- R.Version()
platformname <- paste(rv$arch,gsub(rv$os,"^([[:alpha:]]+)","\\1"),sep="_")
expected <- eigen_results[[platformname]]
should extract the correct version for the platform currently being tested.
On Thu, May 17, 2018 at 11:53 AM, Martin Maechler
<maech...@stat.math.ethz.ch> wrote:
>>>>>> Kevin Coombes <kevin.r.coom...@gmail.com>
>>>>>> on Thu, 17 May 2018 11:21:23 -0400 writes:
>
> > Hi, I wrote and maintain the Thresher package. It includes
> > code to do simulations. In the "tests" directory of the
> > package, I do some simple simulations and run the main
> > algorithm, then write out summaries of the results
>
> > The initial submission of the package to CRAN was delayed
> > because the "Rout.save" files matched the "Rout" files on
> > 64-bit R but *not* on 32-bit R on Windows. After
> > investigating, I realized that when my simulation code
> > called "MASS::mvrnorm", I got different results from
> > 64-bit and 32-bit versions of R on the same machine.
> > Pushing further, I determined that this was happening
> > because mvrnorm used "eigen" to compute the eigenvalues
> > and eigenvectors, and "eigen" itself gave different
> > answers in the two R versions..
>
> > The underlying issue (mathematically) is that the
> > correlation/covariance matrix I was using had repeated
> > eigenvalues, and so there is no unique choice of basis for
> > the associated eigenspace. This observation suggests that
> > the issue is potentially more general than 32-bit versus
> > 64-bit; the results will depend on the implementation of
> > the eigen-decomposition in whatever linear algebra module
> > is compiled along with R, so it can change from machine to
> > machine.
>
> > I "solved" (well, worked around) the immediate problem
> > with package submission by changing the test code to not
> > write out anything that might differ between versions.
>
> > With all of that as background, here are my main
> > questions:
>
> > [1] Is there any way to put something into the "tests"
> > directory that would allow me to use these simulations for
> > what computer scientists call regression testing? (That
> > is, to make sure my changes to the code haven't changed
> > results in an unexpected way.)
>
> > [2] Should there be a flag or instruction to R CMD check
> > that says to only run or interpret this particular test on
> > a specific version or machine? (Or is there already such a
> > flag that I don't know about?)
>
> > [3] Should the documentation (man page) for "eigen" or
> > "mvrnorm" include a warning that the results can change
> > from machine to machine (or between things like 32-bit and
> > 64-bit R on the same machine) because of difference in
> > linear algebra modules? (Possibly including the statement
> > that "set.seed" won't save you.)
>
> The problem is that most (young?) people do not read help pages
> anymore.
>
> help(eigen) has contained the following text for years, and in
> spite of your good analysis of the problem you seem to not have
> noticed the last semi-paragraph:
>
>> Value:
>>
>> The spectral decomposition of ‘x’ is returned as a list with
>> components
>>
>> values: a vector containing the p eigenvalues of ‘x’, sorted in
>> _decreasing_ order, according to ‘Mod(values)’ in the
>> asymmetric case when they might be complex (even for real
>> matrices). For real asymmetric matrices the vector will be
>> complex only if complex conjugate pairs of eigenvalues are
>> detected.
>>
>> vectors: either a p * p matrix whose columns contain the eigenvectors
>> of ‘x’, or ‘NULL’ if ‘only.values’ is ‘TRUE’. The vectors
>> are normalized to unit length.
>>
>> Recall that the eigenvectors are only defined up to a
>> constant: even when the length is specified they are still
>> only defined up to a scalar of modulus one (the sign for real
>> matrices).
>
> It's not a warning but a "recall that" .. maybe because the
> author already assumed that only thorough users would read that
> and for them it would be a recall of something they'd have
> learned *and* not entirely forgotten since ;-)
>
> Martin Maechler
> ETH Zurich
>
> ______________________________________________
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
______________________________________________
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
