Re: [R] Numerical stability of: 1/(1 - cos(x)) - 2/x^2

[Martin Maechler]

> Leonard Mada 
> on Wed, 16 Aug 2023 20:50:52 +0300 writes:

> Dear Iris,
> Dear Martin,

> Thank you very much for your replies. I add a few comments.

> 1.) Correct formula
> The formula in the Subject Title was correct. A small glitch swept into 
> the last formula:
> - 1/(cos(x) - 1) - 2/x^2
> or
> 1/(1 - cos(x)) - 2/x^2 # as in the subject title;

> 2.) log1p
> Actually, the log-part behaves much better. And when it fails, it fails 
> completely (which is easy to spot!).

> x = 1E-6
> log(x) -log(1 - cos(x))/2
> # 0.3465291

> x = 1E-8
> log(x) -log(1 - cos(x))/2
> # Inf
> log(x) - log1p(- cos(x))/2
> # Inf => fails as well!
> # although using only log1p(cos(x)) seems to do the trick;
> log1p(cos(x)); log(2)/2;

> 3.) 1/(1 - cos(x)) - 2/x^2
> It is possible to convert the formula to one which is numerically more 
> stable. It is also possible to compute it manually, but it involves much 
> more work and is also error prone:

> (x^2 - 2 + 2*cos(x)) / (x^2 * (1 - cos(x)))
> And applying L'Hospital:
> (2*x - 2*sin(x)) / (2*x * (1 - cos(x)) + x^2*sin(x))
> # and a 2nd & 3rd & 4th time
> 1/6

> The big problem was that I did not expect it to fail for x = 1E-4. I 
> thought it is more robust and works maybe until 1E-5.
> x = 1E-5
> 2/x^2 - 2E+10
> # -3.814697e-06

> This is the reason why I believe that there is room for improvement.

> Sincerely,
> Leonard

Thank you, Leonard.
Yes, I agree that it is amazing how much your formula suffers from
(a generalization of) "cancellation" --- leading you to think
there was a problem with cos() or log() or .. in R.
But really R uses the system builtin libmath library, and the
problem is really the inherent instability of your formula.

Indeed your first approximation was not really much more stable:

## 3.) 1/(1 - cos(x)) - 2/x^2
## It is possible to convert the formula to one which is numerically more
## stable. It is also possible to compute it manually, but it involves much
## more work and is also error prone:
## (x^2 - 2 + 2*cos(x)) / (x^2 * (1 - cos(x)))
## MM: but actually, that approximation does not seem better (close to the 
breakdown region):
f1 <- \(x) 1/(1 - cos(x)) - 2/x^2
f2 <- \(x) (x^2 - 2 + 2*cos(x)) / (x^2 * (1 - cos(x)))
curve(f1, 1e-8, 1e-1, log="xy" n=2^10)
curve(f2, add = TRUE, col=2,   n=2^10)
## Zoom in:
curve(f1, 1e-4, 1e-1, log="xy",n=2^9)
curve(f2, add = TRUE, col=2,   n=2^9)
## Zoom in much more in y-direction:
yl <- 1/6 + c(-5, 20)/10
curve(f1, 1e-4, 1e-1, log="x", ylim=yl, n=2^9)
abline(h = 1/6, lty=3, col="gray")
curve(f2, add = TRUE, n=2^9, col=adjustcolor(2, 1/2))

Now, you can use the Rmpfr package (interface to the GNU MPFR
multiple-precision C library) to find out more :

if(!requireNamespace("Rmpfr")) install.packages("Rmpfr")
M <- function(x, precBits=128) Rmpfr::mpfr(x, precBits)

(xM <- M(1e-8))# yes, only ~ 16 dig accurate
## 1.20922560830128472675327e-8
M(10, 128)^-8 # would of course be more accurate,
## but we want the calculation for the double precision number 1e-8

## Now you can draw "the truth" into the above plots:
curve(f1, 1e-4, 1e-1, log="xy",n=2^9)
curve(f2, add = TRUE, col=2,   n=2^9)
## correct:
curve(f1(M(x, 256)), add = TRUE, col=4, lwd=2, n=2^9)
abline(h = 1/6, lty=3, col="gray")

But, indeed we take note  how much it is the formula instability: 
Also MPFR needs a lot of extra bits precision before it gets to
the correct numbers: 

xM <- c(M(1e-8,  80), M(1e-8,  96), M(1e-8, 112), 
M(1e-8, 128), M(1e-8, 180), M(1e-8, 256))
## to and round back to 70 bits for display:
R <- \(x) Rmpfr::roundMpfr(x, 70)
R(f1(xM)) 
R(f2(xM))
## [1] 0  0  
0.15407439555097885670915
## [4] 0.1746653133802175779  0.1749979  
0.1750001

## 1. f1() is even worse than f2() {here at x=1e-8} 
## 2. Indeed, even 96 bits precision is *not* sufficient at all, ...
##which is amazing to me as well !!

Best regards,
Martin

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Ben Bolker]

> However, should the numbers
> generated identical if the same seed is used?

  I don't see how using the same seed can overcome floating-point 
differences across platforms (compilers etc.) stemming from differences 
in an eigen() computation (based on arcane details like use of 
registers, compiler optimizations, etc.).  set.seed() guarantees that 
the normal deviates generated by rnorm() will be identical, but those 
random numbers are then multiplied by the eigenvectors derived from 
eigen(), at which point the differences will crop up.


  This has been discussed on Twitter: 
https://twitter.com/wviechtb/status/1230078883317387264


Wolfgang Viechtbauer, 2020-02-18:

Another interesting reproducibility issue that came up. MASS::mvrnorm() 
can give different values despite setting the same seed. The problem: 
Results of eigen() (which is used by mvrnorm) can be machine dependent 
(help(eigen) does warn about this).


Interestingly, mvtnorm::rmvnorm() with method="eigen" gives the same 
values across all machines I tested this on (and method="svd" give the 
same values). With method="chol" I get different values, but again 
consistent across machines.


Ah, mvtnorm::rmvnorm() applies the results from eigen() in a different 
way that appears to be less (not?) affected by the indeterminacy in the 
eigenvectors. Quite clever.



On 2023-08-16 11:10 p.m., Shu Fai Cheung wrote:

Hi All,

When addressing an error in one of my packages in the CRAN check at CRAN,
I found something strange which, I believe, is unrelated to my package.
I am not sure whether it is a bug or a known issue. Therefore, I would like
to have advice from experts here.

The error at CRAN check occurred in a test, and only on R-devel with MKL. No
issues on all other platforms. The test compares two sets of random numbers,
supposed to be identical because generated from the same seed. However, when
I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS, linked to
MKL (2023.2.0), I found that this is not the case in one situation.

I don't know why but somehow only one particular set of means and
covariance matrix, among many others in the code, led to that problem.
Please find
below the code to reproduce the means and the covariance matrix (pardon me
for the long code):

mu0 <- structure(c(0.52252416853188655, 0.39883382931927774, 1.6296642535174521,
2.9045763671121816, -0.19816874840500939, 0.42610841566522556,
0.30155498531316366, 1.0339601619394503, 3.4125587827873192,
13.125481598475405, 19.275480386183503, 658.94225353462195, 1.0997193726987393,
9.9980286642877214, 6.4947188998602092, -12.952617780813679,
8.4991882784024799, 106.82209520278377, 0.19623712449219838), class =
c("numeric"))

sigma0 <- structure(c(0.0047010182215945009, 1.4688424622163808e-17,
-2.5269697696885822e-17,
-1.2451516929358655e-18, 3.6553475725253408e-19, 3.2092363914356227e-19,
-2.6341938382508503e-18, 8.6439582878287556e-19, -1.295240602054123e-17,
1.1154057497258702e-18, -8.2407621704807367e-33, -1.0891686096304908e-15,
-2.6800671450262819e-18, -0.0009225142979911, -1.4833018148344697e-16,
2.292242132203e-16, -3.5128615476876035e-18, 2.8004770623734002e-33,
2.0818099301348832e-34, 1.294907062174661e-17, 0.012788608283099183,
1.5603789410838518e-15, 1.0425182851251724e-16, 2.758318745465066e-17,
-9.7047963858148572e-18, -7.4685438142667792e-17, -1.9426723638193857e-17,
-7.8775307262071738e-17, -4.0773523140359949e-17, 4.1212975037936416e-18,
-1.199934828824835e-14, 2.301740948367742e-17, -2.5410883836579325e-18,
-0.129172198695584, -1.9922957470809647e-14, 1.7077690739402761e-16,
1.663702957392817e-30, 5.5362608501514495e-32, -2.5575000656645995e-17,
7.3879960338026333e-16, 0.052246980157830372, -0.007670030643844231,
-1.5468402204507243e-17, 1.0518757786432683e-17, 2.9992131812421253e-16,
1.6588830885081527e-17, -8.3276045185697992e-16, -3.3713008849128695e-15,
1.5823980389961169e-17, 2.2243227896948194e-15, -3.2071920299461956e-17,
5.0187645877462975e-18, -7.4622951185527265e-15, -0.44701112744184102,
-1.854066605407503e-16, -5.2511356793649759e-30, -6.7993385117653912e-32,
-1.5343740968067595e-18, -3.6785921616583949e-17, -0.0076700306438433133,
0.019231143599164325, 5.7818784939170702e-18, 1.7819897158350214e-17,
-2.3927039320969442e-17, -3.9630951242056644e-18, 1.9779107704573469e-15,
7.8103367010164485e-15, -1.2351275675875924e-17, -8.6959160183013082e-15,
-1.601002787863e-18, 3.0110116065267407e-19, 3.7155867714997651e-16,
-0.12490087179942209, -2.2029631919898945e-16, -8.0869824211018481e-31,
-5.058673986695e-33, 2.0667225433033359e-19, 1.0141735013231455e-18,
3.2845977347050714e-17, -2.9662734037779067e-19, 9.9595082331331027e-05,
-1.9982466114987023e-18, 2.6420113469249318e-18, 5.2415327077690342e-19,
3.007533948014542e-18, -5.1128158961673558e-17, 6.7791246559883483e-19,
-3.042327665547861e-15, 1.9523738271941284e-19, -4.0556768902105205e-20,
-1.024372770865438e-17, -0.010638955366526863, 2.5786757042756338e-17,
1.3546

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Viechtbauer, Wolfgang (NP)]

But I have never seen a case where, on the same machine and using the same math 
routines, using the same seed repeatedly generated different values. That is 
quite bizarre (the issue I was discussing in the thread is quite simple in 
comparison). I tried to reproduce the issue below, but was unable to do so.

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.6 LTS

Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/libmkl_rt.so;  LAPACK version 3.7.0

locale:
 [1] LC_CTYPE=en_US.UTF-8   LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8LC_COLLATE=C
 [5] LC_MONETARY=en_US.UTF-8LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8   LC_NAME=C
 [9] LC_ADDRESS=C   LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

time zone: Europe/Berlin
tzcode source: system (glibc)

attached base packages:
[1] stats graphics  grDevices utils datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.3.1
>
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908

Best,
Wolfgang

>-Original Message-
>From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Ben Bolker
>Sent: Thursday, 17 August, 2023 14:57
>To: r-help@r-project.org
>Subject: Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when
>the seed is the same?
>
> > However, should the numbers
> > generated identical if the same seed is used?
>
>   I don't see how using the same seed can overcome floating-point
>differences across platforms (compilers etc.) stemming from differences
>in an eigen() computation (based on arcane details like use of
>registers, compiler optimizations, etc.).  set.seed() guarantees that
>the normal deviates generated by rnorm() will be identical, but those
>random numbers are then multiplied by the eigenvectors derived from
>eigen(), at which point the differences will crop up.
>
>   This has been discussed on Twitter:
>https://twitter.com/wviechtb/status/1230078883317387264
>
>Wolfgang Viechtbauer, 2020-02-18:
>
>Another interesting reproducibility issue that came up. MASS::mvrnorm()
>can give different values despite setting the same seed. The problem:
>Results of eigen() (which is used by mvrnorm) can be machine dependent
>(help(eigen) does warn about this).
>
>Interestingly, mvtnorm::rmvnorm() with method="eigen" gives the same
>values across all machines I tested this on (and method="svd" give the
>same values). With method="chol" I get different values, but again
>consistent across machines.
>
>Ah, mvtnorm::rmvnorm() applies the results from eigen() in a different
>way that appears to be less (not?) affected by the indeterminacy in the
>eigenvectors. Quite clever.
>
>On 2023-08-16 11:10 p.m., Shu Fai Cheung wrote:
>> Hi All,
>>
>> When addressing an error in one of my packages in the CRAN check at CRAN,
>> I found something strange which, I believe, is unrelated to my package.
>> I am not sure whether it is a bug or a known issue. Therefore, I would like
>> to have advice from experts here.
>>
>> The error at CRAN check occurred in a test, and only on R-devel with MKL. No
>> issues on all other platforms. The test compares two sets of random numbers,
>> supposed to be identical because generated from the same seed. However, when
>> I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS, linked to
>> MKL (2023.2.0), I found that this is not the case in one situation.
>>
>> I don't know why but somehow only one particular set of means and
>> covariance matrix, among many others in the code, led to that problem.
>> Please find
>> below the code to reproduce the means and the covariance matrix (pardon me
>> for the long code):
>>
>> mu0 <- structure(c(0.52252416853188655, 0.39883382931927774,
>1.6296642535174521,
>> 2.9045763671121816, -0.19816874840500939, 0.42610841566522556,
>> 0.30155498531316366, 1.0339601619394503, 3.4125587827873192,
>> 13.125481598475405, 19.275480386183503, 658.94225353462195, 
>> 1.0997193726987393,
>> 9.9980286642877214, 6.4947188998602092, -12.952617780813679,
>> 8.49918827

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Bill Dunlap]

MKL's results can depend on the number of threads running and perhaps other
things.  They blame it on the non-associativity of floating point
arithmetic.  This article gives a way to make results repeatable:

https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-the-conditional-numerical-reproducibility-cnr.html

-Bill

On Wed, Aug 16, 2023 at 8:11 PM Shu Fai Cheung 
wrote:

> Hi All,
>
> When addressing an error in one of my packages in the CRAN check at CRAN,
> I found something strange which, I believe, is unrelated to my package.
> I am not sure whether it is a bug or a known issue. Therefore, I would like
> to have advice from experts here.
>
> The error at CRAN check occurred in a test, and only on R-devel with MKL.
> No
> issues on all other platforms. The test compares two sets of random
> numbers,
> supposed to be identical because generated from the same seed. However,
> when
> I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS, linked
> to
> MKL (2023.2.0), I found that this is not the case in one situation.
>
> I don't know why but somehow only one particular set of means and
> covariance matrix, among many others in the code, led to that problem.
> Please find
> below the code to reproduce the means and the covariance matrix (pardon me
> for the long code):
>
> mu0 <- structure(c(0.52252416853188655, 0.39883382931927774,
> 1.6296642535174521,
> 2.9045763671121816, -0.19816874840500939, 0.42610841566522556,
> 0.30155498531316366, 1.0339601619394503, 3.4125587827873192,
> 13.125481598475405, 19.275480386183503, 658.94225353462195,
> 1.0997193726987393,
> 9.9980286642877214, 6.4947188998602092, -12.952617780813679,
> 8.4991882784024799, 106.82209520278377, 0.19623712449219838), class =
> c("numeric"))
>
> sigma0 <- structure(c(0.0047010182215945009, 1.4688424622163808e-17,
> -2.5269697696885822e-17,
> -1.2451516929358655e-18, 3.6553475725253408e-19, 3.2092363914356227e-19,
> -2.6341938382508503e-18, 8.6439582878287556e-19, -1.295240602054123e-17,
> 1.1154057497258702e-18, -8.2407621704807367e-33, -1.0891686096304908e-15,
> -2.6800671450262819e-18, -0.0009225142979911, -1.4833018148344697e-16,
> 2.292242132203e-16, -3.5128615476876035e-18, 2.8004770623734002e-33,
> 2.0818099301348832e-34, 1.294907062174661e-17, 0.012788608283099183,
> 1.5603789410838518e-15, 1.0425182851251724e-16, 2.758318745465066e-17,
> -9.7047963858148572e-18, -7.4685438142667792e-17, -1.9426723638193857e-17,
> -7.8775307262071738e-17, -4.0773523140359949e-17, 4.1212975037936416e-18,
> -1.199934828824835e-14, 2.301740948367742e-17, -2.5410883836579325e-18,
> -0.129172198695584, -1.9922957470809647e-14, 1.7077690739402761e-16,
> 1.663702957392817e-30, 5.5362608501514495e-32, -2.5575000656645995e-17,
> 7.3879960338026333e-16, 0.052246980157830372, -0.007670030643844231,
> -1.5468402204507243e-17, 1.0518757786432683e-17, 2.9992131812421253e-16,
> 1.6588830885081527e-17, -8.3276045185697992e-16, -3.3713008849128695e-15,
> 1.5823980389961169e-17, 2.2243227896948194e-15, -3.2071920299461956e-17,
> 5.0187645877462975e-18, -7.4622951185527265e-15, -0.44701112744184102,
> -1.854066605407503e-16, -5.2511356793649759e-30, -6.7993385117653912e-32,
> -1.5343740968067595e-18, -3.6785921616583949e-17, -0.0076700306438433133,
> 0.019231143599164325, 5.7818784939170702e-18, 1.7819897158350214e-17,
> -2.3927039320969442e-17, -3.9630951242056644e-18, 1.9779107704573469e-15,
> 7.8103367010164485e-15, -1.2351275675875924e-17, -8.6959160183013082e-15,
> -1.601002787863e-18, 3.0110116065267407e-19, 3.7155867714997651e-16,
> -0.12490087179942209, -2.2029631919898945e-16, -8.0869824211018481e-31,
> -5.058673986695e-33, 2.0667225433033359e-19, 1.0141735013231455e-18,
> 3.2845977347050714e-17, -2.9662734037779067e-19, 9.9595082331331027e-05,
> -1.9982466114987023e-18, 2.6420113469249318e-18, 5.2415327077690342e-19,
> 3.007533948014542e-18, -5.1128158961673558e-17, 6.7791246559883483e-19,
> -3.042327665547861e-15, 1.9523738271941284e-19, -4.0556768902105205e-20,
> -1.024372770865438e-17, -0.010638955366526863, 2.5786757042756338e-17,
> 1.3546130005558746e-32, 1.7947249868721723e-33, 1.9228405363682382e-19,
> -4.1209014001548407e-18, 1.5228291529275058e-17, 2.0443162438349059e-17,
> -1.9572693345249383e-18, 0.0012709581028842473, -0.0018515998737074948,
> 9.3128024867175073e-20, -5.1895788671618993e-17, 2.7373981615289174e-16,
> 1.2812977711597223e-17, -2.2792319486263267e-15, 4.1599503721278813e-19,
> -3.7733269771394201e-20, 4.16234419478765e-17, -1.5986158133468129e-16,
> -0.016037670900735834, -5.0763064708173244e-33, -1.0176066166236902e-50,
> -5.9296704797665404e-18, -8.0698801402861772e-17, 2.9646619457173492e-16,
> -2.8879431119718227e-17, 2.9393253663483117e-18, -0.0018515998737074959,
> 0.090335687736246548, 5.6849412731758135e-19, 8.8934458836007799e-17,
> -4.1390858756690365e-16, 4.120323677410211e-16, 2.8000915545933503e-15,
> 2.8094462743052983e-17, 1.1636214841356605e-18, 8.15103

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Shu Fai Cheung]

Sorry for the confusion caused by that sentence. I did
not expect using the same seed generates the same
sequence of numbers unconditionally. What puzzled me
is not having the same sequence when running the
same code repeatedly in the same computer in the same
session of R. The following four sets of results were generated
from a script with four copies of the two lines, ran in the
same session:

> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6872818  2.7644787 -0.2023346
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6872818  2.7644787 -0.2023346
> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6873036  2.7645014 -0.2020908

I could not figure out the pattern. I could not even
reproduce the order above. Sometimes, the order can be
different. Sometimes, the four sets of results can be identical.

I did a few more tests, on another computer (Windows 10),
with several copies of R installed. For R 4.3.1 and
R-devel (2023-08-12 r84939 ucrt), for which I did not touch
anything about BLAS and LAPACK, they consistently gave
the following results:

> set.seed(1234)
> MASS::mvrnorm(n = 5, mu = mu0, Sigma = sigma0)[1, 1:5]
[1]  0.4851605  0.5704446  1.6872818  2.7644787 -0.2023346

Therefore, it is possible that in my test with MKL, somehow
BLAS and LAPACK came with R were sometimes used, even
though the sessionInfo() output shows that MKL is used.

I am not sure whether it is because I did anything wrong in
setting up R to use MKL.

More on the machine on which I found that issue. It was a virtual
machine built using VirtualBox. The host system is Windows 10
(though I guess it should not matter). The OS is Ubuntu 22.04.3 LTS.
I installed MKL (2023.2.0) following these steps:

https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-download.html?operatingsystem=linux&distributions=aptpackagemanager

I compiled R-devel (2023-08-15 r84957) and then link MKL to it
following these steps (with some modifications, as the description
seems to be outdated):

https://www.intel.com/content/www/us/en/developer/articles/technical/quick-linking-intel-mkl-blas-lapack-to-r.html

Ideally, I should compile R with MKL, instead of using the quick
linking method. However, I am occupied with something else and
could not find an updated guide on how to do this quickly. The
guides I located were several years old.

Regards,
Shu Fai


On Thu, Aug 17, 2023 at 8:57 PM Ben Bolker  wrote:

>
>  > However, should the numbers
>  > generated identical if the same seed is used?
>
>I don't see how using the same seed can overcome floating-point
> differences across platforms (compilers etc.) stemming from differences
> in an eigen() computation (based on arcane details like use of
> registers, compiler optimizations, etc.).  set.seed() guarantees that
> the normal deviates generated by rnorm() will be identical, but those
> random numbers are then multiplied by the eigenvectors derived from
> eigen(), at which point the differences will crop up.
>
>This has been discussed on Twitter:
> https://twitter.com/wviechtb/status/1230078883317387264
>
> Wolfgang Viechtbauer, 2020-02-18:
>
> Another interesting reproducibility issue that came up. MASS::mvrnorm()
> can give different values despite setting the same seed. The problem:
> Results of eigen() (which is used by mvrnorm) can be machine dependent
> (help(eigen) does warn about this).
>
> Interestingly, mvtnorm::rmvnorm() with method="eigen" gives the same
> values across all machines I tested this on (and method="svd" give the
> same values). With method="chol" I get different values, but again
> consistent across machines.
>
> Ah, mvtnorm::rmvnorm() applies the results from eigen() in a different
> way that appears to be less (not?) affected by the indeterminacy in the
> eigenvectors. Quite clever.
>
>
> On 2023-08-16 11:10 p.m., Shu Fai Cheung wrote:
> > Hi All,
> >
> > When addressing an error in one of my packages in the CRAN check at CRAN,
> > I found something strange which, I believe, is unrelated to my package.
> > I am not sure whether it is a bug or a known issue. Therefore, I would
> like
> > to have advice from experts here.
> >
> > The error at CRAN check occurred in a test, and only on R-devel with
> MKL. No
> > issues on all other platforms. The test compares two sets of random
> numbers,
> > supposed to be identical because generated from the same seed. However,
> when
> > I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS,
> linked to
> > MKL (2023.2.0), I found that this is not the case in one situation.
> >
> > I don't know why but somehow only one particular set of means and
> > covariance matrix, among many others in the code, led to that problem.
> 

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Ben Bolker]

  Thanks, I was missing the point that this was *non-repeatability on 
the same platform*.


On 2023-08-17 10:31 a.m., Bill Dunlap wrote:

MKL's results can depend on the number of threads running and perhaps other
things.  They blame it on the non-associativity of floating point
arithmetic.  This article gives a way to make results repeatable:

https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-the-conditional-numerical-reproducibility-cnr.html

-Bill

On Wed, Aug 16, 2023 at 8:11 PM Shu Fai Cheung 
wrote:


Hi All,

When addressing an error in one of my packages in the CRAN check at CRAN,
I found something strange which, I believe, is unrelated to my package.
I am not sure whether it is a bug or a known issue. Therefore, I would like
to have advice from experts here.

The error at CRAN check occurred in a test, and only on R-devel with MKL.
No
issues on all other platforms. The test compares two sets of random
numbers,
supposed to be identical because generated from the same seed. However,
when
I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS, linked
to
MKL (2023.2.0), I found that this is not the case in one situation.

I don't know why but somehow only one particular set of means and
covariance matrix, among many others in the code, led to that problem.
Please find
below the code to reproduce the means and the covariance matrix (pardon me
for the long code):

mu0 <- structure(c(0.52252416853188655, 0.39883382931927774,
1.6296642535174521,
2.9045763671121816, -0.19816874840500939, 0.42610841566522556,
0.30155498531316366, 1.0339601619394503, 3.4125587827873192,
13.125481598475405, 19.275480386183503, 658.94225353462195,
1.0997193726987393,
9.9980286642877214, 6.4947188998602092, -12.952617780813679,
8.4991882784024799, 106.82209520278377, 0.19623712449219838), class =
c("numeric"))

sigma0 <- structure(c(0.0047010182215945009, 1.4688424622163808e-17,
-2.5269697696885822e-17,
-1.2451516929358655e-18, 3.6553475725253408e-19, 3.2092363914356227e-19,
-2.6341938382508503e-18, 8.6439582878287556e-19, -1.295240602054123e-17,
1.1154057497258702e-18, -8.2407621704807367e-33, -1.0891686096304908e-15,
-2.6800671450262819e-18, -0.0009225142979911, -1.4833018148344697e-16,
2.292242132203e-16, -3.5128615476876035e-18, 2.8004770623734002e-33,
2.0818099301348832e-34, 1.294907062174661e-17, 0.012788608283099183,
1.5603789410838518e-15, 1.0425182851251724e-16, 2.758318745465066e-17,
-9.7047963858148572e-18, -7.4685438142667792e-17, -1.9426723638193857e-17,
-7.8775307262071738e-17, -4.0773523140359949e-17, 4.1212975037936416e-18,
-1.199934828824835e-14, 2.301740948367742e-17, -2.5410883836579325e-18,
-0.129172198695584, -1.9922957470809647e-14, 1.7077690739402761e-16,
1.663702957392817e-30, 5.5362608501514495e-32, -2.5575000656645995e-17,
7.3879960338026333e-16, 0.052246980157830372, -0.007670030643844231,
-1.5468402204507243e-17, 1.0518757786432683e-17, 2.9992131812421253e-16,
1.6588830885081527e-17, -8.3276045185697992e-16, -3.3713008849128695e-15,
1.5823980389961169e-17, 2.2243227896948194e-15, -3.2071920299461956e-17,
5.0187645877462975e-18, -7.4622951185527265e-15, -0.44701112744184102,
-1.854066605407503e-16, -5.2511356793649759e-30, -6.7993385117653912e-32,
-1.5343740968067595e-18, -3.6785921616583949e-17, -0.0076700306438433133,
0.019231143599164325, 5.7818784939170702e-18, 1.7819897158350214e-17,
-2.3927039320969442e-17, -3.9630951242056644e-18, 1.9779107704573469e-15,
7.8103367010164485e-15, -1.2351275675875924e-17, -8.6959160183013082e-15,
-1.601002787863e-18, 3.0110116065267407e-19, 3.7155867714997651e-16,
-0.12490087179942209, -2.2029631919898945e-16, -8.0869824211018481e-31,
-5.058673986695e-33, 2.0667225433033359e-19, 1.0141735013231455e-18,
3.2845977347050714e-17, -2.9662734037779067e-19, 9.9595082331331027e-05,
-1.9982466114987023e-18, 2.6420113469249318e-18, 5.2415327077690342e-19,
3.007533948014542e-18, -5.1128158961673558e-17, 6.7791246559883483e-19,
-3.042327665547861e-15, 1.9523738271941284e-19, -4.0556768902105205e-20,
-1.024372770865438e-17, -0.010638955366526863, 2.5786757042756338e-17,
1.3546130005558746e-32, 1.7947249868721723e-33, 1.9228405363682382e-19,
-4.1209014001548407e-18, 1.5228291529275058e-17, 2.0443162438349059e-17,
-1.9572693345249383e-18, 0.0012709581028842473, -0.0018515998737074948,
9.3128024867175073e-20, -5.1895788671618993e-17, 2.7373981615289174e-16,
1.2812977711597223e-17, -2.2792319486263267e-15, 4.1599503721278813e-19,
-3.7733269771394201e-20, 4.16234419478765e-17, -1.5986158133468129e-16,
-0.016037670900735834, -5.0763064708173244e-33, -1.0176066166236902e-50,
-5.9296704797665404e-18, -8.0698801402861772e-17, 2.9646619457173492e-16,
-2.8879431119718227e-17, 2.9393253663483117e-18, -0.0018515998737074959,
0.090335687736246548, 5.6849412731758135e-19, 8.8934458836007799e-17,
-4.1390858756690365e-16, 4.120323677410211e-16, 2.8000915545933503e-15,
2.8094462743052983e-17, 1.1636214841356605e-18, 8.1510367497

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Viechtbauer, Wolfgang (NP)]

Ah, super interesting! Thanks for the pointer, Bill.

I might not have seen the issue because I have

export MKL_NUM_THREADS=1

in my .profile, so MKL never goes beyond using one thread on my machine.

That's a potential source of non-reproducibility I have not come across. So the 
'wisdom' that one can only guarantee full reproducibility if one locks the 
machine in a safe isn't even true unless one takes extra steps when using MKL.

Best,
Wolfgang

>-Original Message-
>From: R-help [mailto:r-help-boun...@r-project.org] On Behalf Of Bill Dunlap
>Sent: Thursday, 17 August, 2023 16:31
>To: Shu Fai Cheung
>Cc: r-help@r-project.org
>Subject: Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when
>the seed is the same?
>
>MKL's results can depend on the number of threads running and perhaps other
>things.  They blame it on the non-associativity of floating point
>arithmetic.  This article gives a way to make results repeatable:
>
>https://www.intel.com/content/www/us/en/developer/articles/technical/introduction
>-to-the-conditional-numerical-reproducibility-cnr.html
>
>-Bill
>
>On Wed, Aug 16, 2023 at 8:11 PM Shu Fai Cheung 
>wrote:
>
>> Hi All,
>>
>> When addressing an error in one of my packages in the CRAN check at CRAN,
>> I found something strange which, I believe, is unrelated to my package.
>> I am not sure whether it is a bug or a known issue. Therefore, I would like
>> to have advice from experts here.
>>
>> The error at CRAN check occurred in a test, and only on R-devel with MKL.
>> No
>> issues on all other platforms. The test compares two sets of random
>> numbers,
>> supposed to be identical because generated from the same seed. However,
>> when
>> I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS, linked
>> to
>> MKL (2023.2.0), I found that this is not the case in one situation.
>>
>> I don't know why but somehow only one particular set of means and
>> covariance matrix, among many others in the code, led to that problem.
>> Please find
>> below the code to reproduce the means and the covariance matrix (pardon me
>> for the long code):
>>
>> mu0 <- structure(c(0.52252416853188655, 0.39883382931927774,
>> 1.6296642535174521,
>> 2.9045763671121816, -0.19816874840500939, 0.42610841566522556,
>> 0.30155498531316366, 1.0339601619394503, 3.4125587827873192,
>> 13.125481598475405, 19.275480386183503, 658.94225353462195,
>> 1.0997193726987393,
>> 9.9980286642877214, 6.4947188998602092, -12.952617780813679,
>> 8.4991882784024799, 106.82209520278377, 0.19623712449219838), class =
>> c("numeric"))
>>
>> sigma0 <- structure(c(0.0047010182215945009, 1.4688424622163808e-17,
>> -2.5269697696885822e-17,
>> -1.2451516929358655e-18, 3.6553475725253408e-19, 3.2092363914356227e-19,
>> -2.6341938382508503e-18, 8.6439582878287556e-19, -1.295240602054123e-17,
>> 1.1154057497258702e-18, -8.2407621704807367e-33, -1.0891686096304908e-15,
>> -2.6800671450262819e-18, -0.0009225142979911, -1.4833018148344697e-16,
>> 2.292242132203e-16, -3.5128615476876035e-18, 2.8004770623734002e-33,
>> 2.0818099301348832e-34, 1.294907062174661e-17, 0.012788608283099183,
>> 1.5603789410838518e-15, 1.0425182851251724e-16, 2.758318745465066e-17,
>> -9.7047963858148572e-18, -7.4685438142667792e-17, -1.9426723638193857e-17,
>> -7.8775307262071738e-17, -4.0773523140359949e-17, 4.1212975037936416e-18,
>> -1.199934828824835e-14, 2.301740948367742e-17, -2.5410883836579325e-18,
>> -0.129172198695584, -1.9922957470809647e-14, 1.7077690739402761e-16,
>> 1.663702957392817e-30, 5.5362608501514495e-32, -2.5575000656645995e-17,
>> 7.3879960338026333e-16, 0.052246980157830372, -0.007670030643844231,
>> -1.5468402204507243e-17, 1.0518757786432683e-17, 2.9992131812421253e-16,
>> 1.6588830885081527e-17, -8.3276045185697992e-16, -3.3713008849128695e-15,
>> 1.5823980389961169e-17, 2.2243227896948194e-15, -3.2071920299461956e-17,
>> 5.0187645877462975e-18, -7.4622951185527265e-15, -0.44701112744184102,
>> -1.854066605407503e-16, -5.2511356793649759e-30, -6.7993385117653912e-32,
>> -1.5343740968067595e-18, -3.6785921616583949e-17, -0.0076700306438433133,
>> 0.019231143599164325, 5.7818784939170702e-18, 1.7819897158350214e-17,
>> -2.3927039320969442e-17, -3.9630951242056644e-18, 1.9779107704573469e-15,
>> 7.8103367010164485e-15, -1.2351275675875924e-17, -8.6959160183013082e-15,
>> -1.601002787863e-18, 3.0110116065267407e-19, 3.7155867714997651e-16,
>> -0.12490087179942209, -2.2029631919898945e-16, -8.0869824211018481e-31,
>> -5.058673986695e-33, 2.0667225433033359e-19, 1.0141735013231455e-18,
>> 3.2845977347050714e-17, -2.9662734037779067e-19, 9.9595082331331027e-05,
>> -1.9982466114987023e-18, 2.6420113469249318e-18, 5.2415327077690342e-19,
>> 3.007533948014542e-18, -5.1128158961673558e-17, 6.7791246559883483e-19,
>> -3.042327665547861e-15, 1.9523738271941284e-19, -4.0556768902105205e-20,
>> -1.024372770865438e-17, -0.010638955366526863, 2.5786757042756338e-17,
>> 1.3546130005558746e-32, 1.7947249868721723e-

[R] Questions about R

[Shaun Parr]

Sent from Outlook for Android


Hi there,

My name is Shaun and I work in an organisation where one of our users wishes to 
install the R software and our process is to assess the safety of anyone 
software prior to authorisation. I can’t seem to locate all the information 
that we require on the webpage, so could someone kindly advise me of the 
following information please?

1. Please can you confirm what user information the software collects (E.g. 
Name, password, e-mail address, any Personally Identifiable Information etc)?
2. If any is collected, please can you confirm if the information collected by 
the software stays locally on the device or if it is transferred anywhere. If 
it is transferred, could you please advise where it is transferred to (E.g. 
your own servers, or a third party data centre such as Amazon Web Services or 
Azure)?
3. Are there any third-party components installed within the software and, if 
so, are these also kept up-to-date?

If you could kindly advise this information, it would be really appreciated, 
thank you 😊


Shaun

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Questions about R

[Jeff Newmiller]

1. R is open source software. You are welcome to review the source code of the 
R interpreter for handling of PII. No warranty is offered by the (volunteer) 
developers of R.

2. R software is a programming interpreter, which by definition allows for a 
wide range of behaviours.  Users of R frequently share useful scripts and 
compilable source code that they have written in the form of packages. There 
are multiple repositories of such packages online... it is up to the consumer 
of any such contributed package to evaluate whether it is fit for the desired 
purpose.

3. See 2. Whether specific packages are out of date or not depends on the rigor 
of the person sharing their package... it is up to the consumer to evaluate 
fitness for purpose.

FWIW I have never encountered an R package that collected PII. But I have only 
used a few hundred out of the tens of thousands available, so YMMV.

On August 17, 2023 4:10:58 AM PDT, Shaun Parr  wrote:
>
>
>Sent from Outlook for Android
>
>
>Hi there,
>
>My name is Shaun and I work in an organisation where one of our users wishes 
>to install the R software and our process is to assess the safety of anyone 
>software prior to authorisation. I can’t seem to locate all the information 
>that we require on the webpage, so could someone kindly advise me of the 
>following information please?
>
>1. Please can you confirm what user information the software collects (E.g. 
>Name, password, e-mail address, any Personally Identifiable Information etc)?
>2. If any is collected, please can you confirm if the information collected by 
>the software stays locally on the device or if it is transferred anywhere. If 
>it is transferred, could you please advise where it is transferred to (E.g. 
>your own servers, or a third party data centre such as Amazon Web Services or 
>Azure)?
>3. Are there any third-party components installed within the software and, if 
>so, are these also kept up-to-date?
>
>If you could kindly advise this information, it would be really appreciated, 
>thank you 😊
>
>
>Shaun
>
>   [[alternative HTML version deleted]]
>
>__
>R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

-- 
Sent from my phone. Please excuse my brevity.

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Questions about R

[Bert Gunter]

This is a volunteer Help list for users of R, which is open source, so you
can see all its code. I can answer no to your questions, unless you are
using one of R's innumerable packages that interacts with the internet and
to which the user may give personal information to enable the desired
functionality (logins, etc.).  But of course how do you know that I am not
some malevolent agent or organization wishing to mislead you for my own
nefarious purposes?

Cheers,
Bert

On Thu, Aug 17, 2023 at 8:37 AM Shaun Parr  wrote:

>
>
> Sent from Outlook for Android
>
>
> Hi there,
>
> My name is Shaun and I work in an organisation where one of our users
> wishes to install the R software and our process is to assess the safety of
> anyone software prior to authorisation. I can’t seem to locate all the
> information that we require on the webpage, so could someone kindly advise
> me of the following information please?
>
> 1. Please can you confirm what user information the software collects
> (E.g. Name, password, e-mail address, any Personally Identifiable
> Information etc)?
> 2. If any is collected, please can you confirm if the information
> collected by the software stays locally on the device or if it is
> transferred anywhere. If it is transferred, could you please advise where
> it is transferred to (E.g. your own servers, or a third party data centre
> such as Amazon Web Services or Azure)?
> 3. Are there any third-party components installed within the software and,
> if so, are these also kept up-to-date?
>
> If you could kindly advise this information, it would be really
> appreciated, thank you 😊
>
>
> Shaun
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Questions about R

[Bert Gunter]

Incidentally, you might be interested in the banner shown when R starts up:

"R is free software and comes with ABSOLUTELY NO WARRANTY."

I believe this is standard for open source software (upon which a lot of
organizations depend). In any case, that might be the most definitive and
"official" answer you can get.

Bert

On Thu, Aug 17, 2023 at 9:17 AM Bert Gunter  wrote:

> This is a volunteer Help list for users of R, which is open source, so you
> can see all its code. I can answer no to your questions, unless you are
> using one of R's innumerable packages that interacts with the internet and
> to which the user may give personal information to enable the desired
> functionality (logins, etc.).  But of course how do you know that I am not
> some malevolent agent or organization wishing to mislead you for my own
> nefarious purposes?
>
> Cheers,
> Bert
>
> On Thu, Aug 17, 2023 at 8:37 AM Shaun Parr 
> wrote:
>
>>
>>
>> Sent from Outlook for Android
>>
>>
>> Hi there,
>>
>> My name is Shaun and I work in an organisation where one of our users
>> wishes to install the R software and our process is to assess the safety of
>> anyone software prior to authorisation. I can’t seem to locate all the
>> information that we require on the webpage, so could someone kindly advise
>> me of the following information please?
>>
>> 1. Please can you confirm what user information the software collects
>> (E.g. Name, password, e-mail address, any Personally Identifiable
>> Information etc)?
>> 2. If any is collected, please can you confirm if the information
>> collected by the software stays locally on the device or if it is
>> transferred anywhere. If it is transferred, could you please advise where
>> it is transferred to (E.g. your own servers, or a third party data centre
>> such as Amazon Web Services or Azure)?
>> 3. Are there any third-party components installed within the software
>> and, if so, are these also kept up-to-date?
>>
>> If you could kindly advise this information, it would be really
>> appreciated, thank you 😊
>>
>>
>> Shaun
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Questions about R

[Rui Barradas]

Às 12:10 de 17/08/2023, Shaun Parr escreveu:



Sent from Outlook for Android


Hi there,

My name is Shaun and I work in an organisation where one of our users wishes to 
install the R software and our process is to assess the safety of anyone 
software prior to authorisation. I can’t seem to locate all the information 
that we require on the webpage, so could someone kindly advise me of the 
following information please?

1. Please can you confirm what user information the software collects (E.g. 
Name, password, e-mail address, any Personally Identifiable Information etc)?
2. If any is collected, please can you confirm if the information collected by 
the software stays locally on the device or if it is transferred anywhere. If 
it is transferred, could you please advise where it is transferred to (E.g. 
your own servers, or a third party data centre such as Amazon Web Services or 
Azure)?
3. Are there any third-party components installed within the software and, if 
so, are these also kept up-to-date?

If you could kindly advise this information, it would be really appreciated, 
thank you 😊


Shaun

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Hello,

1. R itself? None. Download from CRAN and install. There are OS related 
installation issues, namely authorization but that information is not 
asked for nor recorded by R.

2. The answer to "If any is collected" is already given above.
3. I am not sure I understand this point. R comes with third-party 
components and their developers try to keep them up-to-date. This has 
nothing to do with PII.

CRAN is the main official repository for contributed packages.  From [1]:

Available Packages

Currently, the CRAN package repository features 19955 available packages.


and the R  instruction

available.packages() |> nrow()
# [1] 19931

says a number close to that one.


Those packages are developed and contributed by volunteers and it's 
impossible for the CRAN maintainers  to check what exactly those 
packages do but those packages' source code must be submited and anyone 
willing to check them can.


[1] https://cran.r-project.org/web/packages/

Hope this helps,

Rui Barradas

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Questions about R

[Spencer Graves]

Hi, Shaun:


	  I suggest you also check the Wikipedia article on "R (programming 
language)":



https://en.wikipedia.org/wiki/R_(programming_language)


	  That article has 115 numbered "References" that provide more 
information.  You don't have to believe anything you read anywhere. 
However you many know that almost anyone can change almost anything on 
Wikipedia.  What stays tends to be written from a neutral point of view 
citing credible sources.  Many Wikipedia articles contain sections on 
controveries, etc.  I don't find anything like that in this article, 
with good reason.



  Hope this helps.
  Spencer Graves


On 8/17/23 11:48 AM, Bert Gunter wrote:

Incidentally, you might be interested in the banner shown when R starts up:

"R is free software and comes with ABSOLUTELY NO WARRANTY."

I believe this is standard for open source software (upon which a lot of
organizations depend). In any case, that might be the most definitive and
"official" answer you can get.

Bert

On Thu, Aug 17, 2023 at 9:17 AM Bert Gunter  wrote:


This is a volunteer Help list for users of R, which is open source, so you
can see all its code. I can answer no to your questions, unless you are
using one of R's innumerable packages that interacts with the internet and
to which the user may give personal information to enable the desired
functionality (logins, etc.).  But of course how do you know that I am not
some malevolent agent or organization wishing to mislead you for my own
nefarious purposes?

Cheers,
Bert

On Thu, Aug 17, 2023 at 8:37 AM Shaun Parr 
wrote:




Sent from Outlook for Android


Hi there,

My name is Shaun and I work in an organisation where one of our users
wishes to install the R software and our process is to assess the safety of
anyone software prior to authorisation. I can’t seem to locate all the
information that we require on the webpage, so could someone kindly advise
me of the following information please?

1. Please can you confirm what user information the software collects
(E.g. Name, password, e-mail address, any Personally Identifiable
Information etc)?
2. If any is collected, please can you confirm if the information
collected by the software stays locally on the device or if it is
transferred anywhere. If it is transferred, could you please advise where
it is transferred to (E.g. your own servers, or a third party data centre
such as Amazon Web Services or Azure)?
3. Are there any third-party components installed within the software
and, if so, are these also kept up-to-date?

If you could kindly advise this information, it would be really
appreciated, thank you 😊


Shaun

 [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.





[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Questions about R

[Ebert,Timothy Aaron]

When I installed R no personally identifiable information was given, none was 
requested. There is no login, so no password or user name. R resides on the 
host computer, so security is the responsibility of the owner(s) of that 
computer. There are packages and other third party programs, but these are 
installed separately and are not part of the initial installation. Generally 
packages do not collect PII, but I know a very small fraction of all packages. 
Individual packages are updated with new R updates if the people who wrote the 
package maintain the package. Users are responsible for updating to the new 
version on their system. There are third party programs like RStudio that can 
be helpful. I do not remember entering PII when loading RStudio, but it has 
been some time since I went through that process. A great deal of time can be 
saved by appropriate use of third party software, but I am not an expert in 
base R coding.

I work at a large university. R and RStudio are approved software. However, the 
university wants us to download the software from their website. This is to 
avoid non-university fourth party software. This is more a risk with novice 
users who may not know if they are downloading software directly from the 
appropriate website or indirectly through another website.

Malevolent Agent,
Tim

-Original Message-
From: R-help  On Behalf Of Bert Gunter
Sent: Thursday, August 17, 2023 12:18 PM
To: Shaun Parr 
Cc: r-help@r-project.org
Subject: Re: [R] Questions about R

[External Email]

This is a volunteer Help list for users of R, which is open source, so you can 
see all its code. I can answer no to your questions, unless you are using one 
of R's innumerable packages that interacts with the internet and to which the 
user may give personal information to enable the desired functionality (logins, 
etc.).  But of course how do you know that I am not some malevolent agent or 
organization wishing to mislead you for my own nefarious purposes?

Cheers,
Bert

On Thu, Aug 17, 2023 at 8:37 AM Shaun Parr  wrote:

>
>
> Sent from Outlook for
> Android F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Ctebert%40ufl.edu%7C2d345ea4a3ac40f
> 9dd7f08db9f3e701a%7C0d4da0f84a314d76ace60a62331e1b84%7C0%7C0%7C6382788
> 62758357724%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMz
> IiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=F19Y5h34rhBDqWp
> yFLdBNnETu%2Bv47vzq7tK3NfsJs5g%3D&reserved=0>
>
>
> Hi there,
>
> My name is Shaun and I work in an organisation where one of our users
> wishes to install the R software and our process is to assess the
> safety of anyone software prior to authorisation. I can’t seem to
> locate all the information that we require on the webpage, so could
> someone kindly advise me of the following information please?
>
> 1. Please can you confirm what user information the software collects
> (E.g. Name, password, e-mail address, any Personally Identifiable
> Information etc)?
> 2. If any is collected, please can you confirm if the information
> collected by the software stays locally on the device or if it is
> transferred anywhere. If it is transferred, could you please advise
> where it is transferred to (E.g. your own servers, or a third party
> data centre such as Amazon Web Services or Azure)?
> 3. Are there any third-party components installed within the software
> and, if so, are these also kept up-to-date?
>
> If you could kindly advise this information, it would be really
> appreciated, thank you 😊
>
>
> Shaun
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat/
> .ethz.ch%2Fmailman%2Flistinfo%2Fr-help&data=05%7C01%7Ctebert%40ufl.edu
> %7C2d345ea4a3ac40f9dd7f08db9f3e701a%7C0d4da0f84a314d76ace60a62331e1b84
> %7C0%7C0%7C638278862758357724%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAw
> MDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sda
> ta=dB7JDCZ6VylYGAYXKtenZG0I9V5AJfbMe7QKx33ydwQ%3D&reserved=0
> PLEASE do read the posting guide
> http://www.r/
> -project.org%2Fposting-guide.html&data=05%7C01%7Ctebert%40ufl.edu%7C2d
> 345ea4a3ac40f9dd7f08db9f3e701a%7C0d4da0f84a314d76ace60a62331e1b84%7C0%
> 7C0%7C638278862758357724%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
> CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=x7
> YxjXrpwdjfHbxqgQnMIgaualvHumGVCN3A6MM49Vg%3D&reserved=0
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.r-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
_

[R] Timezone question

[Dennis Fisher]

R 4.3.1
OS X

Colleagues

Is there a simple way to determine the timezone offset for my present location. 
 For example, during standard time in the US, the offset from GMT is 8 hours in 
California.

Dennis

Dennis Fisher MD
P < (The "P Less Than" Company)
Phone / Fax: 1-866-PLessThan (1-866-753-7784)
www.PLessThan.com 





[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Timezone question

[Marc Schwartz via R-help]

Hi Dennis,

I believe that it may be as simple as:

> format(Sys.time(), "%z")
[1] "-0400"

Where "%z" returns the offset from UTC as a character vector, and where I am in:

> Sys.timezone()
[1] "America/New_York"

There may be some subtleties, and I would suggest reading the relevant help 
files for ?strptime and the above two functions.

Regards,

Marc Schwartz


-Original Message-
From: R-help mailto:r-help-boun...@r-project.org>> on behalf of Dennis Fisher 
mailto:fis...@plessthan.com>>
Date: Thursday, August 17, 2023 at 4:31 PM
To: mailto:r-help@r-project.org>>
Subject: [R] Timezone question


R 4.3.1
OS X


Colleagues


Is there a simple way to determine the timezone offset for my present location. 
For example, during standard time in the US, the offset from GMT is 8 hours in 
California.


Dennis


Dennis Fisher MD
P < (The "P Less Than" Company)
Phone / Fax: 1-866-PLessThan (1-866-753-7784)
www.PLessThan.com  ;>

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Timezone question

[Bert Gunter]

You may also find the package "lutz" to be of interest, although that may
be overkill for your needs.
(found by an internet search).

Cheers,
Bert


On Thu, Aug 17, 2023 at 1:31 PM Dennis Fisher  wrote:

> R 4.3.1
> OS X
>
> Colleagues
>
> Is there a simple way to determine the timezone offset for my present
> location.  For example, during standard time in the US, the offset from GMT
> is 8 hours in California.
>
> Dennis
>
> Dennis Fisher MD
> P < (The "P Less Than" Company)
> Phone / Fax: 1-866-PLessThan (1-866-753-7784)
> www.PLessThan.com 
>
>
>
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] MASS::mvrnorm() on MKL may produce different numbers even when the seed is the same?

[Shu Fai Cheung]

Thanks a lot! I am not aware of this behavior! I will take this into
account in my work.

Regards,
Shu Fai

On Thu, Aug 17, 2023 at 10:31 PM Bill Dunlap  wrote:
>
> MKL's results can depend on the number of threads running and perhaps other 
> things.  They blame it on the non-associativity of floating point arithmetic. 
>  This article gives a way to make results repeatable:
>
> https://www.intel.com/content/www/us/en/developer/articles/technical/introduction-to-the-conditional-numerical-reproducibility-cnr.html
>
> -Bill
>
> On Wed, Aug 16, 2023 at 8:11 PM Shu Fai Cheung  
> wrote:
>>
>> Hi All,
>>
>> When addressing an error in one of my packages in the CRAN check at CRAN,
>> I found something strange which, I believe, is unrelated to my package.
>> I am not sure whether it is a bug or a known issue. Therefore, I would like
>> to have advice from experts here.
>>
>> The error at CRAN check occurred in a test, and only on R-devel with MKL. No
>> issues on all other platforms. The test compares two sets of random numbers,
>> supposed to be identical because generated from the same seed. However, when
>> I tried that in R-devel (2023-08-15 r84957) on Ubuntu 22.04.3 LTS, linked to
>> MKL (2023.2.0), I found that this is not the case in one situation.
>>
>> I don't know why but somehow only one particular set of means and
>> covariance matrix, among many others in the code, led to that problem.
>> Please find
>> below the code to reproduce the means and the covariance matrix (pardon me
>> for the long code):
>>
>> mu0 <- structure(c(0.52252416853188655, 0.39883382931927774, 
>> 1.6296642535174521,
>> 2.9045763671121816, -0.19816874840500939, 0.42610841566522556,
>> 0.30155498531316366, 1.0339601619394503, 3.4125587827873192,
>> 13.125481598475405, 19.275480386183503, 658.94225353462195, 
>> 1.0997193726987393,
>> 9.9980286642877214, 6.4947188998602092, -12.952617780813679,
>> 8.4991882784024799, 106.82209520278377, 0.19623712449219838), class =
>> c("numeric"))
>>
>> sigma0 <- structure(c(0.0047010182215945009, 1.4688424622163808e-17,
>> -2.5269697696885822e-17,
>> -1.2451516929358655e-18, 3.6553475725253408e-19, 3.2092363914356227e-19,
>> -2.6341938382508503e-18, 8.6439582878287556e-19, -1.295240602054123e-17,
>> 1.1154057497258702e-18, -8.2407621704807367e-33, -1.0891686096304908e-15,
>> -2.6800671450262819e-18, -0.0009225142979911, -1.4833018148344697e-16,
>> 2.292242132203e-16, -3.5128615476876035e-18, 2.8004770623734002e-33,
>> 2.0818099301348832e-34, 1.294907062174661e-17, 0.012788608283099183,
>> 1.5603789410838518e-15, 1.0425182851251724e-16, 2.758318745465066e-17,
>> -9.7047963858148572e-18, -7.4685438142667792e-17, -1.9426723638193857e-17,
>> -7.8775307262071738e-17, -4.0773523140359949e-17, 4.1212975037936416e-18,
>> -1.199934828824835e-14, 2.301740948367742e-17, -2.5410883836579325e-18,
>> -0.129172198695584, -1.9922957470809647e-14, 1.7077690739402761e-16,
>> 1.663702957392817e-30, 5.5362608501514495e-32, -2.5575000656645995e-17,
>> 7.3879960338026333e-16, 0.052246980157830372, -0.007670030643844231,
>> -1.5468402204507243e-17, 1.0518757786432683e-17, 2.9992131812421253e-16,
>> 1.6588830885081527e-17, -8.3276045185697992e-16, -3.3713008849128695e-15,
>> 1.5823980389961169e-17, 2.2243227896948194e-15, -3.2071920299461956e-17,
>> 5.0187645877462975e-18, -7.4622951185527265e-15, -0.44701112744184102,
>> -1.854066605407503e-16, -5.2511356793649759e-30, -6.7993385117653912e-32,
>> -1.5343740968067595e-18, -3.6785921616583949e-17, -0.0076700306438433133,
>> 0.019231143599164325, 5.7818784939170702e-18, 1.7819897158350214e-17,
>> -2.3927039320969442e-17, -3.9630951242056644e-18, 1.9779107704573469e-15,
>> 7.8103367010164485e-15, -1.2351275675875924e-17, -8.6959160183013082e-15,
>> -1.601002787863e-18, 3.0110116065267407e-19, 3.7155867714997651e-16,
>> -0.12490087179942209, -2.2029631919898945e-16, -8.0869824211018481e-31,
>> -5.058673986695e-33, 2.0667225433033359e-19, 1.0141735013231455e-18,
>> 3.2845977347050714e-17, -2.9662734037779067e-19, 9.9595082331331027e-05,
>> -1.9982466114987023e-18, 2.6420113469249318e-18, 5.2415327077690342e-19,
>> 3.007533948014542e-18, -5.1128158961673558e-17, 6.7791246559883483e-19,
>> -3.042327665547861e-15, 1.9523738271941284e-19, -4.0556768902105205e-20,
>> -1.024372770865438e-17, -0.010638955366526863, 2.5786757042756338e-17,
>> 1.3546130005558746e-32, 1.7947249868721723e-33, 1.9228405363682382e-19,
>> -4.1209014001548407e-18, 1.5228291529275058e-17, 2.0443162438349059e-17,
>> -1.9572693345249383e-18, 0.0012709581028842473, -0.0018515998737074948,
>> 9.3128024867175073e-20, -5.1895788671618993e-17, 2.7373981615289174e-16,
>> 1.2812977711597223e-17, -2.2792319486263267e-15, 4.1599503721278813e-19,
>> -3.7733269771394201e-20, 4.16234419478765e-17, -1.5986158133468129e-16,
>> -0.016037670900735834, -5.0763064708173244e-33, -1.0176066166236902e-50,
>> -5.9296704797665404e-18, -8.0698801402861772e-17, 2.9646619457173492e-16,
>> -2.8879431119718227e-17, 2.