On Thursday, October 8, 2015, David Joyner <wdjoy...@gmail.com> wrote:
> On Thu, Oct 8, 2015 at 2:58 PM, William Stein <wst...@gmail.com > <javascript:;>> wrote: > > > > > > On Thursday, October 8, 2015, David Joyner <wdjoy...@gmail.com > <javascript:;>> wrote: > >> > >> Before this thread drops off the radar, I have a question. How hard > >> would it be to rebuild (not port) Sage starting with windows Python, > >> then adding windows GAP, windows SIngular, networkxx, and > >> SymPy+friends, of which GAP+Singular communicate with the Sage > >> terminal via pexpect? Call it WinSage 1.0:-) > > > > > > Just curious - what do you mean by "the sage terminal"? > > > > I was referring to some sort of cygwin command line (since I hardly > ever use anything else), but if people prefer an ipython notebook-like > interface then that's fine too. This whole thread is about creating such a thing - in any way at all that actually *works* - in the first place. There is no usable sage command line for Windows. Native or not. > > Also to stop people jumping on pexpect which doesn't at all make sense or > > work natively on Windows, replace that by "some sort of IPC". > > > > > > > >> > >> > >> I don't even use windows so really this is a totally naive question. > >> > >> > >> On Thu, Oct 8, 2015 at 7:01 AM, Bill Hart <goodwillh...@googlemail.com > <javascript:;>> > >> wrote: > >> > > >> > > >> > On Wednesday, 7 October 2015 17:38:00 UTC+2, bluescarni wrote: > >> >> > >> >> Some random thoughts: > >> >> > >> >> - I am not so convinced the strategy of automatic long -> long long > >> >> patching is actually feasible, I think in practice this is gonna be a > >> >> big > >> >> can of worms. Pushing upstream to fix their code is a much better > >> >> long-term > >> >> solution IMO (and I'd rather have nothing to do with projects which > >> >> refuse > >> >> portability and code-quality patches, but that's just me :) > >> >> - mingw-w64 is a high quality compiler. I recently had to develop > for a > >> >> while on a Windows 64 machine, and had to recompile a full stacks of > >> >> dependencies (MPFR, GMP, Boost, etc.). It was not so easy (most > issues > >> >> were > >> >> related to build systems), but the end result was pretty impressive > >> >> performance-wise. > >> >> - While for C code you can probably interoperate with libraries > >> >> compiled > >> >> with MSVC, for C++ you will end up having issues related, e.g., to > >> >> different > >> >> implementations of exception handling. My suggestion would be to > forget > >> >> about MSVC and compile the full stack of dependencies exclusively > with > >> >> mingw. > >> >> - Python on windows 64 does not play properly with mingw due to some > >> >> long-standing issues in some header files. If you want to compile > >> >> Python > >> >> C/C++ extensions on Win64 you will have to patch slightly the Python > >> >> headers > >> >> and re-generate the definitions of the Python library. The issue is > >> >> explored, e.g., here: > >> >> > >> >> > >> >> > >> >> > http://ascend4.org/Setting_up_a_MinGW-w64_build_environment#Setup_Python_for_compilation_of_extensions > >> > > >> > > >> > Someone has asked me off list to point out the following: > >> > > >> > "the MSYS2 project have ported CPython and have provided four packages > >> > that > >> > don't use MSVC: mingw-w64-{i686,x86_64}-ptython{2,3} (as well as ones > >> > that > >> > link to msys-2.0.dll)." > >> > > >> > I see that there is a list of packages here: > >> > > >> > http://sourceforge.net/p/msys2/wiki/Packages/ > >> > > >> >> > >> >> > >> >> Generally speaking, mingw-w64 is a really good option for C/C++ > >> >> development on Windows. The biggest problem is the proliferation of > >> >> distributions (mingw-w64, TDM mingw, mingw-build, msys2, etc.), but > the > >> >> toolchain is solid. clang is getting there as well, so it's worth > >> >> keeping it > >> >> in mind for the future. > >> >> > >> >> Cheers, > >> >> > >> >> Francesco. > >> >> > >> >> > >> >> On 7 October 2015 at 16:35, Bill Hart <goodwi...@googlemail.com > <javascript:;>> wrote: > >> >>> > >> >>> HI all, > >> >>> > >> >>> William Stein recently bemoaned the fact that SageMath currently > only > >> >>> runs natively on some brands of Linux, and not natively on the > latest > >> >>> Windows or OSX (that is to say nothing of BSD). [1] > >> >>> > >> >>> Until recently, a port of SageMath to Windows has seemed like a pipe > >> >>> dream. However, things have changed a lot, and I think it would now > be > >> >>> possible to achieve this (for some definition of the word "native"). > >> >>> > >> >>> I've tried VM's before and it has always ended in disaster. They > >> >>> either > >> >>> screw up my networking, they have performance issues, or don't > support > >> >>> my > >> >>> mouse properly, or change my keyboard layout, or it's impossible to > >> >>> get > >> >>> files from my hard drive into the system easily, or they take up too > >> >>> much > >> >>> disk space or need to be constantly downloaded to update them, or > some > >> >>> features don't work, or people stop supporting them, etc. The > >> >>> disadvantages > >> >>> of VMs are so numerous I hardly need to enumerate them. And even if > it > >> >>> is a > >> >>> solution for users, it's hardly a solution for serious Windows > >> >>> developers. > >> >>> > >> >>> As some people know, I've been recently working on a Julia based > >> >>> "computer algebra system" for a much more limited subset of > "computer > >> >>> algebra" than Sage targets. What people may not know is that that > >> >>> entire > >> >>> technology stack works natively on Windows. > >> >>> > >> >>> I can't express how fantastic it was to be sitting on a train > >> >>> recently, > >> >>> with no web access whatsoever and to be able to do work on my > Windows > >> >>> 10 (64 > >> >>> bit) laptop on the train. And everything ran as fast, or in some > cases > >> >>> faster, than my Linux server. That's the first time that has > happened > >> >>> since > >> >>> I started doing computational stuff about 10 years ago! > >> >>> > >> >>> MSYS2 as a solution > >> >>> ================ > >> >>> > >> >>> The new technology that makes all this work is MSYS2. Here are some > of > >> >>> its features: > >> >>> > >> >>> * Installing MSYS2 on Windows couldn't be easier. [2] > >> >>> * It supports proper Windows exception handling protocols. > >> >>> * It has an amazing command line package manager called Pacman, > which > >> >>> Just Works TM. > >> >>> * Unlike Cygwin, it's very minimal, and takes little time to > install. > >> >>> * It's fast. Very fast. > >> >>> * Parallel make works. > >> >>> * The resulting binaries are fast, sometimes faster on my laptop > than > >> >>> on > >> >>> the Linux server I usually use for development. > >> >>> * MSYS2 provides a posix layer! (Applications can be distributed > with > >> >>> an > >> >>> MSYS2 dll for this.) > >> >>> * Multithreading using pthreads Just Works TM (Applications can be > >> >>> distributed with a winthreads dll for this. I've actually used this, > >> >>> no > >> >>> patching required, so I am not speaking theoretically here.) > >> >>> * Like native Windows, sizeof(long) = 4 and sizeof(long long) = 8 on > >> >>> MSYS2. This means interop with native Windows applications or > assembly > >> >>> objects is a cynch. > >> >>> * The resulting applications can be run on Windows as essentially > >> >>> native > >> >>> applications. They don't have to be run from within the MSYS2 shell. > >> >>> They > >> >>> can also be distributed as binary packages for those that don't care > >> >>> about > >> >>> the source code. But here's the thing: it's not *necessary* to > >> >>> distribute > >> >>> them as binary packages. It's now quite feasible for developers to > >> >>> actually > >> >>> *build* packages on Windows. And let's face it, this is the type of > >> >>> developer we actually want on the Windows platform. Simply > supporting > >> >>> users > >> >>> and not developers is just going to increase the maintenance burden > >> >>> for > >> >>> people who work on other platforms. We want actual Windows > developers, > >> >>> not > >> >>> just users! > >> >>> > >> >>> The only bad thing about MSYS2 is that autotools is still incredibly > >> >>> slow > >> >>> (Flint roll-your-own configure takes 5s at most on Windows, whereas > >> >>> autotools can take some minutes to configure a package). Note: make > is > >> >>> not > >> >>> slow (at least not on Windows 64 -- it's not as fast on 32 bit > >> >>> Windows, but > >> >>> fortunately almost no one is using Windows 32 any more anyway). > >> >>> > >> >>> Proposal > >> >>> ======= > >> >>> > >> >>> What I propose is to begin porting Sage dependencies to Windows > (note > >> >>> that by Windows, I mean Windows 64, i.e. what everyone is using). > What > >> >>> this > >> >>> would mean is porting them to MSYS2 (at least initially). Note it's > >> >>> not a > >> >>> big step from there to MinGW or MSVC if people want to do that > later. > >> >>> Microsoft have made MSVC much, much more friendly to GNU > conventions, > >> >>> recently. (For some users, porting to Windows natively is synonymous > >> >>> with > >> >>> supporting MSVC using Visual Solution files. But I'm not suggesting > we > >> >>> do > >> >>> that.) > >> >>> > >> >>> Since we know that many upstream projects simply will not accept > >> >>> thousands of patches to their repositories for supporting Windows > 64, > >> >>> much > >> >>> less do they have developer resources to support them on an ongoing > >> >>> basis, > >> >>> here is what I think needs to be done: > >> >>> > >> >>> 1) Automagically patch their source trees at build time so that long > >> >>> becomes long long and unsigned long becomes unsigned long long on > >> >>> Windows. > >> >>> 2) Automagically patch their source trees at build time so that > >> >>> printf/scanf("%ld") becomes printf/scanf("%lld") on Windows. > >> >>> 3) Send a very small number of "Windows" patches upstream for the > few > >> >>> instances of nonportable code that probably doesn't work anywhere > >> >>> except > >> >>> Linux, let alone on Windows. > >> >>> > >> >>> Is this feasible? > >> >>> > >> >>> Yes it is. Note that libpari still works today on Windows 64 > (actually > >> >>> natively with MinGW, not just with MSYS). On Windows 64 their build > >> >>> automagically patches their source tree. As far as I know, this > hasn't > >> >>> been > >> >>> touched in years, and it still works today as well as the day it was > >> >>> written > >> >>> (modulo a couple of trivial bugs in their actual build system)! > >> >>> > >> >>> [Optionally, we should investigate the feasibility of providing > >> >>> Makefiles > >> >>> for Windows, automatically generated by autotools, but stored on a > >> >>> server so > >> >>> that they do not have to be generated at build time (of course the > >> >>> user > >> >>> still has the option of doing that if they really want). This may > not > >> >>> be > >> >>> feasible, but it's worth investigating. Obviously paths would have > to > >> >>> be > >> >>> patched on a per user basis. But Windows is a very homogeneous > >> >>> environment, > >> >>> so most of what autotools actually does on Windows is pretty > >> >>> pointless.] > >> >>> > >> >>> What we've done so far > >> >>> ================== > >> >>> > >> >>> So far, the following packages (and probably many more) work fine on > >> >>> Windows 64, with the provisions noted: > >> >>> > >> >>> * MPIR-2.7.0 > >> >>> * MPFR-3.1.3 > >> >>> * Flint-2.4.5 > >> >>> * libpari-2.7.4 -- GP is not patched, and there are some bugs in the > >> >>> Pari > >> >>> build script, but I've been able to hack them to get libpari to > build > >> >>> on > >> >>> Windows 64. The %ld -> %lld patching is not done yet. > >> >>> * Antic > >> >>> * OpenBLAS (the Julia people already build this on Windows, since > they > >> >>> use it heavily) > >> >>> > >> >>> Python must also work, since I use IJulia on my Windows 64 laptop, > >> >>> which > >> >>> is basically the same thing as the IPython notebook. This I download > >> >>> as part > >> >>> of the Anaconda distribution currently. > >> >>> > >> >>> You can find Windows 32 and 64 dlls for most of the packages > mentioned > >> >>> above, here: > >> >>> > >> >>> http://nemocas.org/binaries/ > >> >>> > >> >>> My intention is to publish a note every time I update this > collection > >> >>> of > >> >>> dlls and to gradually expand them to cover more packages. We'd > >> >>> eventually > >> >>> want to use dlltray, or the OpenSUSE cross platform build service > [3] > >> >>> for > >> >>> hosting these (the latter is not trivial at the moment, since their > >> >>> package > >> >>> management system doesn't handle some of the complexities of > building > >> >>> packages that Sage would rely on). > >> >>> > >> >>> It would be fantastic if others were to join me in extending this > list > >> >>> over time. I'd be happy to host the dlls there (actually the machine > >> >>> is > >> >>> provided by William Stein), at least until the hosting requirements > >> >>> start to > >> >>> become nontrivial. > >> >>> > >> >>> The only requirement would be to send dlls for Window 32 and 64 and > a > >> >>> tarball of the source used to generate them (or alternatively a git > >> >>> repo I > >> >>> can clone), so that we aren't violating anyone's GPL. We'd also have > >> >>> to > >> >>> agree on what exception handling protocol to use. It would be very > >> >>> convenient for me should that match the one used by the Julia > project. > >> >>> > >> >>> We will shortly be porting Arb to Windows 64. I'll post an update > when > >> >>> that is done. > >> >>> > >> >>> Note that Julia itself works on Windows and they have an interest in > >> >>> porting lots of stuff to work natively on Windows. So it's possible > >> >>> some > >> >>> joint effort could be mutually beneficial. > >> >>> > >> >>> MPIR vs GMP > >> >>> ========== > >> >>> > >> >>> One of MPIR's dirty little secrets is we abandon the GMP interface > on > >> >>> Windows 64, for good reason. They make all their _ui and _si > function > >> >>> take > >> >>> an unsigned long/long instead of an unsigned long long/long long, > even > >> >>> though limbs are 64 bits on Windows. > >> >>> > >> >>> In some timings I did recently, this slows some operations down by a > >> >>> factor of 6, and in Flint we have to supply a GMP compatibility > >> >>> header, so > >> >>> that we can use GMP in the same way we use MPIR on Windows 64 (which > >> >>> is > >> >>> pretty time consuming to write and obviously a performance sink). > >> >>> > >> >>> As I'm not interested in just having something that works on Windows > >> >>> but > >> >>> doesn't care about performance, I personally believe that such an > >> >>> effort > >> >>> should be based on MPIR not GMP for the time being. > >> >>> > >> >>> Final note > >> >>> ======== > >> >>> > >> >>> The Julia project targeted Windows very early on, and they have > >> >>> attracted > >> >>> serious, extremely knowledgeable Windows developers (who have been > >> >>> very > >> >>> helpful to me). I think this speaks for itself. > >> >>> > >> >>> Bill. > >> >>> > >> >>> [1] https://plus.google.com/115360165819500279592/posts/HiXvz9bjumu > >> >>> [2] https://msys2.github.io/ > >> >>> [3] https://build.opensuse.org/ > >> >>> > >> >>> -- > >> >>> You received this message because you are subscribed to the Google > >> >>> Groups > >> >>> "sage-devel" group. > >> >>> To unsubscribe from this group and stop receiving emails from it, > send > >> >>> an > >> >>> email to sage-devel+...@googlegroups.com <javascript:;>. > >> >>> To post to this group, send email to sage-...@googlegroups.com > <javascript:;>. > >> >>> Visit this group at http://groups.google.com/group/sage-devel. > >> >>> For more options, visit https://groups.google.com/d/optout. > >> >> > >> >> > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "sage-devel" group. > >> > To unsubscribe from this group and stop receiving emails from it, send > >> > an > >> > email to sage-devel+unsubscr...@googlegroups.com <javascript:;>. > >> > To post to this group, send email to sage-devel@googlegroups.com > <javascript:;>. > >> > Visit this group at http://groups.google.com/group/sage-devel. > >> > For more options, visit https://groups.google.com/d/optout. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "sage-devel" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to sage-devel+unsubscr...@googlegroups.com <javascript:;>. > >> To post to this group, send email to sage-devel@googlegroups.com > <javascript:;>. > >> Visit this group at http://groups.google.com/group/sage-devel. > >> For more options, visit https://groups.google.com/d/optout. > > > > > > > > -- > > Sent from my massive iPhone 6 plus. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "sage-devel" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to sage-devel+unsubscr...@googlegroups.com <javascript:;>. > > To post to this group, send email to sage-devel@googlegroups.com > <javascript:;>. > > Visit this group at http://groups.google.com/group/sage-devel. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com <javascript:;>. > To post to this group, send email to sage-devel@googlegroups.com > <javascript:;>. > Visit this group at http://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. > -- Sent from my massive iPhone 6 plus. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at http://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
