On Wednesday, 7 October 2015 19:54:30 UTC+2, Dima Pasechnik wrote:
>
>
>
> On Wednesday, 7 October 2015 07:35:22 UTC-7, Bill Hart wrote:
>>
>> HI all,
>>
>> William Stein recently bemoaned the fact that SageMath currently only
>> runs natively on some brands of Linux, and not natively on the latest
>> Windows or OSX (that is to say nothing of BSD). [1]
>>
>> Until recently, a port of SageMath to Windows has seemed like a pipe
>> dream. However, things have changed a lot, and I think it would now be
>> possible to achieve this (for some definition of the word "native").
>>
>> I've tried VM's before and it has always ended in disaster. They either
>> screw up my networking, they have performance issues, or don't support my
>> mouse properly, or change my keyboard layout, or it's impossible to get
>> files from my hard drive into the system easily, or they take up too much
>> disk space or need to be constantly downloaded to update them, or some
>> features don't work, or people stop supporting them, etc. The disadvantages
>> of VMs are so numerous I hardly need to enumerate them. And even if it is a
>> solution for users, it's hardly a solution for serious Windows developers.
>>
>> As some people know, I've been recently working on a Julia based
>> "computer algebra system" for a much more limited subset of "computer
>> algebra" than Sage targets. What people may not know is that that entire
>> technology stack works natively on Windows.
>>
>> I can't express how fantastic it was to be sitting on a train recently,
>> with no web access whatsoever and to be able to do work on my Windows 10
>> (64 bit) laptop on the train. And everything ran as fast, or in some cases
>> faster, than my Linux server. That's the first time that has happened since
>> I started doing computational stuff about 10 years ago!
>>
>> MSYS2 as a solution
>> ================
>>
>> The new technology that makes all this work is MSYS2. Here are some of
>> its features:
>>
>> * Installing MSYS2 on Windows couldn't be easier. [2]
>> * It supports proper Windows exception handling protocols.
>> * It has an amazing command line package manager called Pacman, which
>> Just Works TM.
>> * Unlike Cygwin, it's very minimal, and takes little time to install.
>> * It's fast. Very fast.
>> * Parallel make works.
>> * The resulting binaries are fast, sometimes faster on my laptop than on
>> the Linux server I usually use for development.
>> * MSYS2 provides a posix layer! (Applications can be distributed with an
>> MSYS2 dll for this.)
>> * Multithreading using pthreads Just Works TM (Applications can be
>> distributed with a winthreads dll for this. I've actually used this, no
>> patching required, so I am not speaking theoretically here.)
>> * Like native Windows, sizeof(long) = 4 and sizeof(long long) = 8 on
>> MSYS2. This means interop with native Windows applications or assembly
>> objects is a cynch.
>> * The resulting applications can be run on Windows as essentially native
>> applications. They don't have to be run from within the MSYS2 shell. They
>> can also be distributed as binary packages for those that don't care about
>> the source code. But here's the thing: it's not *necessary* to distribute
>> them as binary packages. It's now quite feasible for developers to actually
>> *build* packages on Windows. And let's face it, this is the type of
>> developer we actually want on the Windows platform. Simply supporting users
>> and not developers is just going to increase the maintenance burden for
>> people who work on other platforms. We want actual Windows developers, not
>> just users!
>>
>> The only bad thing about MSYS2 is that autotools is still incredibly slow
>> (Flint roll-your-own configure takes 5s at most on Windows, whereas
>> autotools can take some minutes to configure a package). Note: make is not
>> slow (at least not on Windows 64 -- it's not as fast on 32 bit Windows, but
>> fortunately almost no one is using Windows 32 any more anyway).
>>
>
> If MSYS2 has (Cygwin's?) fork(), it should not differ much from Cygwin,
> in the sense of porting of Sage.
> fork() was the major pain on Win32, as building Python extensions results
> in huge number of dlls that have hard time
> fitting into the (32bit) address space, so (semi)automatic running of
> rebase during the build was necessary...
>
I am really, really uninterested in Windows 32, as are, I imagine, most
Windows developers these days. Given that Sage is so very far behind the
ball game on Windows, I would skip it entirely. You will likely have to
visit a museum to get a working Win32 box by the time the port is complete.
(This was already the case back when Sage started its Cygwin port).
>
> (Note that on Cygwin (like elsewhere) Sage does not use Cygwin's python,
> it uses its own Python, which
> seems to be quite different from Cygwin's Python)
>
Seriously? So SageMath also risks not even having a supported Python in the
future. Wowsers.
>
> OK, let me try to see if one gets anywhere with GAP on MSYS2...
>
I'll give it a go myself, just to see what happens. It is probably
completely broken, but only one way to find out.
Bill.
>
> Cheers,
> Dima
>
>
> Proposal
>> =======
>>
>> What I propose is to begin porting Sage dependencies to Windows (note
>> that by Windows, I mean Windows 64, i.e. what everyone is using). What this
>> would mean is porting them to MSYS2 (at least initially). Note it's not a
>> big step from there to MinGW or MSVC if people want to do that later.
>> Microsoft have made MSVC much, much more friendly to GNU conventions,
>> recently. (For some users, porting to Windows natively is synonymous with
>> supporting MSVC using Visual Solution files. But I'm not suggesting we do
>> that.)
>>
>> Since we know that many upstream projects simply will not accept
>> thousands of patches to their repositories for supporting Windows 64, much
>> less do they have developer resources to support them on an ongoing basis,
>> here is what I think needs to be done:
>>
>> 1) Automagically patch their source trees at build time so that long
>> becomes long long and unsigned long becomes unsigned long long on Windows.
>> 2) Automagically patch their source trees at build time so that
>> printf/scanf("%ld") becomes printf/scanf("%lld") on Windows.
>> 3) Send a very small number of "Windows" patches upstream for the few
>> instances of nonportable code that probably doesn't work anywhere except
>> Linux, let alone on Windows.
>>
>> Is this feasible?
>>
>> Yes it is. Note that libpari still works today on Windows 64 (actually
>> natively with MinGW, not just with MSYS). On Windows 64 their build
>> automagically patches their source tree. As far as I know, this hasn't been
>> touched in years, and it still works today as well as the day it was
>> written (modulo a couple of trivial bugs in their actual build system)!
>>
>> [Optionally, we should investigate the feasibility of providing Makefiles
>> for Windows, automatically generated by autotools, but stored on a server
>> so that they do not have to be generated at build time (of course the user
>> still has the option of doing that if they really want). This may not be
>> feasible, but it's worth investigating. Obviously paths would have to be
>> patched on a per user basis. But Windows is a very homogeneous environment,
>> so most of what autotools actually does on Windows is pretty pointless.]
>>
>> What we've done so far
>> ==================
>>
>> So far, the following packages (and probably many more) work fine on
>> Windows 64, with the provisions noted:
>>
>> * MPIR-2.7.0
>> * MPFR-3.1.3
>> * Flint-2.4.5
>> * libpari-2.7.4 -- GP is not patched, and there are some bugs in the Pari
>> build script, but I've been able to hack them to get libpari to build on
>> Windows 64. The %ld -> %lld patching is not done yet.
>> * Antic
>> * OpenBLAS (the Julia people already build this on Windows, since they
>> use it heavily)
>>
>> Python must also work, since I use IJulia on my Windows 64 laptop, which
>> is basically the same thing as the IPython notebook. This I download as
>> part of the Anaconda distribution currently.
>>
>> You can find Windows 32 and 64 dlls for most of the packages mentioned
>> above, here:
>>
>> http://nemocas.org/binaries/
>>
>> My intention is to publish a note every time I update this collection of
>> dlls and to gradually expand them to cover more packages. We'd eventually
>> want to use dlltray, or the OpenSUSE cross platform build service [3] for
>> hosting these (the latter is not trivial at the moment, since their package
>> management system doesn't handle some of the complexities of building
>> packages that Sage would rely on).
>>
>> It would be fantastic if others were to join me in extending this list
>> over time. I'd be happy to host the dlls there (actually the machine is
>> provided by William Stein), at least until the hosting requirements start
>> to become nontrivial.
>>
>> The only requirement would be to send dlls for Window 32 and 64 and a
>> tarball of the source used to generate them (or alternatively a git repo I
>> can clone), so that we aren't violating anyone's GPL. We'd also have to
>> agree on what exception handling protocol to use. It would be very
>> convenient for me should that match the one used by the Julia project.
>>
>> We will shortly be porting Arb to Windows 64. I'll post an update when
>> that is done.
>>
>> Note that Julia itself works on Windows and they have an interest in
>> porting lots of stuff to work natively on Windows. So it's possible some
>> joint effort could be mutually beneficial.
>>
>> MPIR vs GMP
>> ==========
>>
>> One of MPIR's dirty little secrets is we abandon the GMP interface on
>> Windows 64, for good reason. They make all their _ui and _si function take
>> an unsigned long/long instead of an unsigned long long/long long, even
>> though limbs are 64 bits on Windows.
>>
>> In some timings I did recently, this slows some operations down by a
>> factor of 6, and in Flint we have to supply a GMP compatibility header, so
>> that we can use GMP in the same way we use MPIR on Windows 64 (which is
>> pretty time consuming to write and obviously a performance sink).
>>
>> As I'm not interested in just having something that works on Windows but
>> doesn't care about performance, I personally believe that such an effort
>> should be based on MPIR not GMP for the time being.
>>
>> Final note
>> ========
>>
>> The Julia project targeted Windows very early on, and they have attracted
>> serious, extremely knowledgeable Windows developers (who have been very
>> helpful to me). I think this speaks for itself.
>>
>> Bill.
>>
>> [1] https://plus.google.com/115360165819500279592/posts/HiXvz9bjumu
>> [2] https://msys2.github.io/
>> [3] https://build.opensuse.org/
>>
>
--
You received this message because you are subscribed to the Google Groups
"sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-devel+unsubscr...@googlegroups.com.
To post to this group, send email to sage-devel@googlegroups.com.
Visit this group at http://groups.google.com/group/sage-devel.
For more options, visit https://groups.google.com/d/optout.
