Šimone, Thanks for this. I understand your concern about being responsible for the SSL side and can just about imagine the technical complexity of getting it done. Many thanks also for the trial build, I very much appreciate you doing this - once I can get it tested on an M1 machine, I will report back but it’s encouraging to hear that it is feasible and solves the issue.
As for Apple’s approach to SSL, my understanding from the bits of information I have been able to find is that they provide very little guarantee/stability/documentation around this, as also attested by the flip flopping you mentioned. Thanks again. Kind regards Petr > On 11. 1. 2022, at 22:12, Simon Urbanek <simon.urba...@r-project.org> wrote: > > Petře, > > thanks, for the detailed analysis. It is rather curious that the issue > appears only on _newer_ systems - we are more used to issues due to older CA > chains and similar. It looks like an Apple bug on specific systems, so > hopefully it will be fixed eventually. In general I was trying to avoid > having to supply our own SSL library since that opens a whole can of worms - > on one hand due the dependency issues (which libraries get compiled against > what) and on the other hand we become responsible for security updates. > > Thanks to Jeroen for the work-around (CURL_SSL_BACKEND=SecureTransport), > using the native API is certainly preferred, there have been several issues > with both OpenSSL and LibreSSL before. It seems that Apple has been > flip-flopping with libcurl a lot - on El Capitan it was shipped with > SecureTransport, on High-Sierra with LibreSSL, on Catalina and higher with > both, but Libre the default. > > I am somewhat less apprehensive to use static libcurl for R than SSL > libraries as the fallout is a bit smaller. As a trial I have added static > curl[2] which is close to the Apple build minus MultiSSL to big-sur nightly > builds of R[3] and as expected that solves the problem. It may not be > entirely unproblematic for package space, because packages often forget to > prepend --static when using static builds of libraries, and so do other > dependencies that may use curl, but I'll see what comes out of it. > > Cheers, > Šimon > > [1] - https://github.com/R-macos/recipes > [2] - https://github.com/R-macos/recipes/blob/add-ons/recipes/curl > [3] - https://mac.r-project.org/ > > > >> On 10/01/2022, at 11:22 PM, Petr Bouchal <pbouc...@gmail.com> wrote: >> >> Dear all, >> >> In brief: on Monterey, R cannot reach certain web domains due to a bug in >> Libre SSL - and perhaps not relying on system curl/openssl in R would be a >> systematic solution to this and símilar issues. >> >> Specifically: on MacOS Monterey 12.1 using R 4.1.2, download.file() and >> other functions that rely on system-provided curl/openssl/Libre SSL >> (including in the curl package) have been failing on specific domains. >> >> So running >> >> download.file(“https://www.czso.cz/”, tempfile()) >> >> returns: >> >> status was ‘SSL connect error’ >> >> the underlying error being >> >> error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length. >> >> This is caused by the Libre SSL bundled in MacOS Monterey and also affects >> several other domains, most notably https://libzip.org. >> >> It is clearly an OS bug but infortunately also a situation where it affects >> R users because of how R relates to system libraries and is very difficult >> to work around. >> >> It has manifested on CRAN (causing a package archival) and Github outside of >> R, so is not caused by a specific machine. It can be replicated on both M1 >> and Intel and also occurs when using curl in the system command line. >> >> The czso.cz domain is the Czech Statistical Office, which makes it quite >> important for a number of users, also of a package I maintain (czso) which >> relies on accessing this domain. I have reported this to the server admin >> but since the problem is in the OS, I do not expect them to be able to help. >> I am not an expert in web security so cannot tell if there is anything in >> the certificates which could be causing this. In browsers, no such issue >> occurs and the server is configured correctly as per ssllabs.com testing. I >> have also reported to Apple but it is unclear whether they will fix this >> given the rare nature of the issue. >> >> It is difficult to work around even on individual machines as replacing the >> system curl/openssl requires steps beyond what a most users are comfortable >> with (or should be doing to begin with). Using HTTP instead of HTTPS does >> not work, nor does using curl —insecure and equivalents. >> >> This brings back the question of whether R on MacOS should include its own >> openssl instead of relying on the system-provided library. This has been >> discussed on the r-devel list: >> https://stat.ethz.ch/pipermail/r-devel/2020-June/079657.html. >> >> Apple also recommends against relying on shared openssl, if I understand >> this correctly: https://developer.apple.com/forums/thread/89051. Given >> Apple’s approach to openssl/Libre SSL in MacOS (the bundled Libre SSL >> version is 3 years old), such hard-to-handle issues are likely to reappear >> over time. (I don’t have in-depth knowledge of how R is compiled, so >> apologies for any inaccuracies; hopefully it is clear what I mean.) >> >> I’d be grateful for any thoughts on how this might be handled in the >> specific case and perhaps generally. >> >> Kind regards >> Petr Bouchal >> >> _______________________________________________ >> R-SIG-Mac mailing list >> R-SIG-Mac@r-project.org >> https://stat.ethz.ch/mailman/listinfo/r-sig-mac >> > _______________________________________________ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
