On Sun, Jun 14, 2009 at 9:22 AM, Dr. David Kirkby<david.kir...@onetel.net> wrote: > > I know from experience that trying to build programs in parallel, with > things like > > $ make -j 10 > > will often break them, so I tend not to do it. It's a nice idea, and > does work in some cases, but in others one just ends up with a mess. > But Sage is a big program, and takes a *long* time to compile. The T5240 > (hostname 't2') has a couple of T2+ processors each with 8 cores, (16 > cores in total). Building 20-30 Sage .spkg files in parallel on 't2' > would speed up the compilation process *considerably*. With the ready > availability of multi-core processors (some 'home' machine now have 4 > cores), it would seem to me some way of building Sage more quickly would > be a really good idea.
Same here. That's why since the beginning, in Sage if you type export MAKE="make -j10" then the individual spkg's for which parallel make is known to work will use "make -j10" and the ones for which it doesn't work will just use "make". Note that two spkg's will never be built at the same time though. > > I don't know how long it will take to build Sage on 't2' in its present > form, but I believe it would be more than one day, as that machine is > currently not being used to its best (mainly since I'm the only one > using it most of the time). I regularly build Sage on sparc lately, and indeed it does take more than a day. > > Another point is that by doing things in parallel, it might allow a more > extensive test suite. If you do export SAGE_CHECK="yes" then build Sage, it will run the spkg-check scripts in each spkg. This is not documented in the Sage README.txt, since my understanding is that it never passes (that's what Michael Abshoff told me), and that nobody decided to fix the issues that arise. I just started a build of 4.0.2.alpha3 (available at http://sage.math.washington.edu/home/wstein/farm/src/) on sage.math with SAGE_CHECK="yes" to see what happens. > Clearly there are dependencies among the .spkg files. There's no point > trying to build model A if it depends on a library B which is not yet > built. Those are encoded in the makefile spkg/standard/deps. I don't know if all of them are encoded, though I hope so. > I can see this would take some work, but the rewards in a reduction of > compilation time, could be very large. You can be sure processors are Actually, you can already build all the spkg's themselves in parallel with dependencies being tracked as follows. In the file SAGE_ROOT/spkg/install change the line time make -f standard/deps $1 to time make -j10 -f standard/deps $1 That's it. Obviously we don't do this now for a reason, which is that last time I tried I think something went wrong, maybe do to lack of some dependency being properly listed in the deps file. > only going to become available with more cores. At one time it was one, > my laptop has two, several home computers have 4 and 't2' has 16. > > I would suggest if this was done, it would be wise to have create files > 'install-<spkg_name>.txt' so there is a record of each. package name, > and not use one 'install.log'. > > > Thoughts? --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to sage-devel-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---
