Dear list, We have three separate, but interacting, difficulties with SSL/TLS support in Sage. I'll summarize the results of the efforts of several people who tracked them, and propose a couple of solutions.
*I) Python now (discreetly) depends on Open SSL.* Their license page <https://docs.python.org/3/license.html> states : > The modules hashlib > <https://docs.python.org/3/library/hashlib-blake2.html#module-hashlib>, > posix <https://docs.python.org/3/library/posix.html#module-posix>, ssl > <https://docs.python.org/3/library/ssl.html#module-ssl>, crypt > <https://docs.python.org/3/library/crypt.html#module-crypt> use the > OpenSSL library for added performance if made available by the operating > system. Additionally, the Windows and Mac OS X installers for Python may > include a copy of the OpenSSL libraries, so we include a copy of the > OpenSSL license here: > followed by the bizarre OpenSSL license. For our purpose, the important statement is *"use the OpenSSL library for added performance if made available by the operating system."*. "Added performance, my a^htired foot : Thierry has checked the possibilities of an OpenSSL-less Sage, and I have further checked other possibilities. Our trials conclusively demonstrate that Gnu TLS can't be substituted to OpenSSL for at least the following reasons : - Sage's pip is non-functionnal when compiled against Gnu TLS - Ditto for Sage's git - I understand (but have not checked) that Python's hashlib module, which depends on openssl, is used in Sage. However, contrary to my expectations, R 3.3.2 *can* be compiled in Sage against a curl library using Gnu TLS and keep a functional HTTPS access to R repositories. Consequences : - Sage *can*be built and run without OpenSSL support, (as long as R is < 3.3 or some SSL support is available for R >= 3.3), but this system will have severe limitations (among others, no access to pip resources, questionable support for Sage's git). - OpenSSL can be retrofitted in such a system by installing the openssl package, but this retrofit becomes effective after recompilation of python2 (at least). This latter "solution" is, at best, a contraption (even if something in this direction has been proposed <https://groups.google.com/d/msg/sage-devel/iwrF8_kGLzM/aze9lJi8nm8J> back in 2012 to solve the very same problem). Therefore : - we *must at minimum* advertise this problem in the REAME.md file and recommend checking the presence of OpenSSL, and recommend the installation of openssl development files for Sage compilation. In this case, we would have to : - provide a standard package providing some HTTPS-capable SSL support. Ideally, this package should be able to check for the presence of suitable systemwide libraries, and in this case, do nothing ; - use this SSL support to provide an HTP-enabled curl for R>=3.3 (with again, the possibility of usinf a systemwide curl library). - We *should* acknowledge our *de facto* dependence on a systemwide OpenSSL (in terms close to those used by the Python license). In this case, we would have to provide a standard curl package, with the same provisions as before. The first solution, used on a system without OpenSSL, will create a crippled Sage. Furthermore, it needs writing two standard packages, installing widely-diffused utilities (it seems awfully difficult to install a Debian system *sans* OpenSSL : even a freshly installed "base system + common utilities" has openssl, on which Debian's reportbug and various utilities depend). I would rather acknowledge our dependence on OpenSSL, recommend its installation and advertise the limitations of an OpenSSL-less Sage, leaving this possibility open to prudes... *II) OpenSSL has broken a lot of software.* OpenSSL 1.1.0 has broken a lot of OpenSSL-using software *at the source level* (older binaries still can use the libraries, but the macro mechanisms used in source are not compatible with those used in OpenSSL 1.0.x, and compilations fail). This has happened in "our" Python ; our now-current 2.7.12 version does not compile against OpenSSL 1.1. A patch against this version, allowing compilation against OpenSSL 1.1 has been released after the version we used in Trac#19735 <https://trac.sagemath.org/ticket/19735>. I tried <https://trac.sagemath.org/ticket/22089> to port it in our current version, and failed miserably (someone with more experience than me should have wielded this chainsaw...). BTW, this has also happened to "our" git, which was easier to upgrade (see Trac#22058 <https://trac.sagemath.org/ticket/22058>, which needs review, BTW). This *is* a problem for us because OpenSSL 1.1 has now reached the stage of diffusion in commonly-used distributions (Debian testing, which means the next Ubuntu, etc...). It has been said that this move was (unduly) hastened by a nearing "freeze" in Debian testing ; true or not, the move has happened, and I don't fight the weather... (Interestingly, cygwin still is at openSSL-devel-1.0.2j). I think that our best bet is the upgrade proposed in Trac#22037 <https://trac.sagemath.org/ticket/22037>, whose development seems to have stopped dead in its tracks after sagemath has hit Debian unstable... This is especially important if we adopt the idea of openly depending on OpenSSL as a solution to I). *III) OpenSSL is problematic on Macintoshes.* (This is by hearsay : I do not have access to a Mac, and don't really understand the problem ; I'm tryin to summarize what I've read). Apple seems to have its own SSL implementation, and specific procedures for updating its collection of root certificates. This makes installing a Sage-specific SSL library problematic, and makes necessary a specific procedure fot root certificates maintenance. 1) I do not know if Apple's ssl implementation is sufficient for a) Sage and related utilities (Sage's pip, Saage's git, etc...) b) Curl (needed bty R>=3.3, see above). 2) It seems also difficult to develop an utility making Apple's root certificates usable by Sage. *Qiscussion and questions* In view of these difficulties, what should be done ? I think that our first priority should be to get a Python that will compile against OpenSSL>=1.1, which will become ubiquitous sooner or later (ant I think it will be sooner...). That means completing Trac#22037 <https://trac.sagemath.org/ticket/22037> as soon as possible. In parallel, we should document the SSL problem right at the startof teh README.md and in the developer's documentation (README.md and the Developer's Guide). I will propose a patch to these effect of these docs. The SSL-using parts of Sage should be reviewed, for answers to three questions : - do they compile against OpenSSL>1.1 on Linux (and other Unices) ? - do they compile efficiently (i. e. with full functionality) against Apple's SSL library ? - will they compile against a future OpenSSL>=1.1 on cygwin ? Platform-specific adaptations should be considered for both Macs and Windows. Questions : - Should we openly depend on OpenSSL ? If so, how to express it ? I'd vote for that, and for warning of the penalties involved by the non-use of OpenSSL, probably in terms close to those of the Python license. - Do we need a standard SSL package ? This is necessary to allow for R>3.3 if we do NOT openly depend on OpenSSL. That's the only way to allow to upgrade to R>3.3, which has become urgent... - How can we help completing Trac#22037 ? <https://trac.sagemath.org/ticket/22037> and, last but not least : - how can we help with the platform-specific aspects of this thorny problem ? Your advice, please ? HTH, -- Emmanuel Charpentier -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
