A first step <https://trac.sagemath.org/ticket/22058> towards a solution awaits your comments and review.
Plan : 1. Document OpenSSL dependency, mention the possibility of compiling againts GnuTLS (with drawbacks) 2. Get OpenSSL development libs on the machines producing Unix binary tarballs/packages. 3. (To be discussed) : create a standard "SSL" package serving as a backup, allowing compilation on OpenSSL-less machines. As done for git, this package should do nothing if OpenSSL is installed systemwide. 4. Complete curl as a standard package, which would allow : 5. Upgrade R. Pffeeeewww... Unsolved problem : What about Macs (I don't have a Mac and can't contribute). To be discussed : Cygwin (advoce from Erik Bray keenly awaited...). HTH, -- Emmanuel Charpentier Le dimanche 1 janvier 2017 02:55:42 UTC+1, Emmanuel Charpentier a écrit : > > Dear list, > > We have three separate, but interacting, difficulties with SSL/TLS support > in Sage. I'll summarize the results of the efforts of several people who > tracked them, and propose a couple of solutions. > > *I) Python now (discreetly) depends on Open SSL.* > > Their license page <https://docs.python.org/3/license.html> states : > >> The modules hashlib >> <https://docs.python.org/3/library/hashlib-blake2.html#module-hashlib>, >> posix <https://docs.python.org/3/library/posix.html#module-posix>, ssl >> <https://docs.python.org/3/library/ssl.html#module-ssl>, crypt >> <https://docs.python.org/3/library/crypt.html#module-crypt> use the >> OpenSSL library for added performance if made available by the operating >> system. Additionally, the Windows and Mac OS X installers for Python may >> include a copy of the OpenSSL libraries, so we include a copy of the >> OpenSSL license here: >> > followed by the bizarre OpenSSL license. For our purpose, the important > statement is *"use the OpenSSL library for added performance if made > available by the operating system."*. > > "Added performance, my a^htired foot : Thierry has checked the > possibilities of an OpenSSL-less Sage, and I have further checked other > possibilities. Our trials conclusively demonstrate that Gnu TLS can't be > substituted to OpenSSL for at least the following reasons : > > - Sage's pip is non-functionnal when compiled against Gnu TLS > - Ditto for Sage's git > - I understand (but have not checked) that Python's hashlib module, > which depends on openssl, is used in Sage. > > > However, contrary to my expectations, R 3.3.2 *can* be compiled in Sage > against a curl library using Gnu TLS and keep a functional HTTPS access to > R repositories. > > Consequences : > > - Sage *can*be built and run without OpenSSL support, (as long as R is > < 3.3 or some SSL support is available for R >= 3.3), but this system > will > have severe limitations (among others, no access to pip resources, > questionable support for Sage's git). > - OpenSSL can be retrofitted in such a system by installing the > openssl package, but this retrofit becomes effective after recompilation > of > python2 (at least). > > This latter "solution" is, at best, a contraption (even if something in > this direction has been proposed > <https://groups.google.com/d/msg/sage-devel/iwrF8_kGLzM/aze9lJi8nm8J> > back in 2012 to solve the very same problem). Therefore : > > - we *must at minimum* advertise this problem in the REAME.md file and > recommend checking the presence of OpenSSL, and recommend the installation > of openssl development files for Sage compilation. In this case, we would > have to : > - provide a standard package providing some HTTPS-capable SSL > support. Ideally, this package should be able to check for the presence > of > suitable systemwide libraries, and in this case, do nothing ; > - use this SSL support to provide an HTP-enabled curl for R>=3.3 > (with again, the possibility of usinf a systemwide curl library). > - We *should* acknowledge our *de facto* dependence on a systemwide > OpenSSL (in terms close to those used by the Python license). In this > case, > we would have to provide a standard curl package, with the same provisions > as before. > > The first solution, used on a system without OpenSSL, will create a > crippled Sage. Furthermore, it needs writing two standard packages, > installing widely-diffused utilities (it seems awfully difficult to install > a Debian system *sans* OpenSSL : even a freshly installed "base system + > common utilities" has openssl, on which Debian's reportbug and various > utilities depend). > > I would rather acknowledge our dependence on OpenSSL, recommend its > installation and advertise the limitations of an OpenSSL-less Sage, leaving > this possibility open to prudes... > > *II) OpenSSL has broken a lot of software.* > > OpenSSL 1.1.0 has broken a lot of OpenSSL-using software *at the source > level* (older binaries still can use the libraries, but the macro > mechanisms used in source are not compatible with those used in OpenSSL > 1.0.x, and compilations fail). > > This has happened in "our" Python ; our now-current 2.7.12 version does > not compile against OpenSSL 1.1. A patch against this version, allowing > compilation against OpenSSL 1.1 has been released after the version we used > in Trac#19735 <https://trac.sagemath.org/ticket/19735>. I tried > <https://trac.sagemath.org/ticket/22089> to port it in our current > version, and failed miserably (someone with more experience than me should > have wielded this chainsaw...). > > BTW, this has also happened to "our" git, which was easier to upgrade (see > Trac#22058 <https://trac.sagemath.org/ticket/22058>, which needs review, > BTW). > > This *is* a problem for us because OpenSSL 1.1 has now reached the stage > of diffusion in commonly-used distributions (Debian testing, which means > the next Ubuntu, etc...). It has been said that this move was (unduly) > hastened by a nearing "freeze" in Debian testing ; true or not, the move > has happened, and I don't fight the weather... (Interestingly, cygwin still > is at openSSL-devel-1.0.2j). > > I think that our best bet is the upgrade proposed in Trac#22037 > <https://trac.sagemath.org/ticket/22037>, whose development seems to have > stopped dead in its tracks after sagemath has hit Debian unstable... This > is especially important if we adopt the idea of openly depending on OpenSSL > as a solution to I). > > *III) OpenSSL is problematic on Macintoshes.* > > (This is by hearsay : I do not have access to a Mac, and don't really > understand the problem ; I'm tryin to summarize what I've read). > > Apple seems to have its own SSL implementation, and specific procedures > for updating its collection of root certificates. This makes installing a > Sage-specific SSL library problematic, and makes necessary a specific > procedure fot root certificates maintenance. > > 1) I do not know if Apple's ssl implementation is sufficient for > a) Sage and related utilities (Sage's pip, Saage's git, etc...) > b) Curl (needed bty R>=3.3, see above). > > 2) It seems also difficult to develop an utility making Apple's root > certificates usable by Sage. > > *Qiscussion and questions* > > In view of these difficulties, what should be done ? > > I think that our first priority should be to get a Python that will > compile against OpenSSL>=1.1, which will become ubiquitous sooner or later > (ant I think it will be sooner...). That means completing Trac#22037 > <https://trac.sagemath.org/ticket/22037> as soon as possible. > > In parallel, we should document the SSL problem right at the startof teh > README.md and in the developer's documentation (README.md and the > Developer's Guide). I will propose a patch to these effect of these docs. > > The SSL-using parts of Sage should be reviewed, for answers to three > questions : > > - do they compile against OpenSSL>1.1 on Linux (and other Unices) ? > - do they compile efficiently (i. e. with full functionality) against > Apple's SSL library ? > - will they compile against a future OpenSSL>=1.1 on cygwin ? > > > Platform-specific adaptations should be considered for both Macs and > Windows. > > Questions : > > - Should we openly depend on OpenSSL ? If so, how to express it ? > > I'd vote for that, and for warning of the penalties involved by the > non-use of OpenSSL, probably in terms close to those of the Python license. > > - Do we need a standard SSL package ? > > This is necessary to allow for R>3.3 if we do NOT openly depend on > OpenSSL. That's the only way to allow to upgrade to R>3.3, which has become > urgent... > > - How can we help completing Trac#22037 ? > <https://trac.sagemath.org/ticket/22037> > > and, last but not least : > > - how can we help with the platform-specific aspects of this thorny > problem ? > > Your advice, please ? > > HTH, > > -- > Emmanuel Charpentier > > -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
