Benjamin Lutz wrote:
Hello,
Since Multi-core processors are becoming popular (or, more
egocentrically, since I've acquired one), I've become interested in
parallel compilation. Unfortunately, it seems that parallel builds of
any kind are completely unsupported by the ports framework at the
moment.
Not for lack of interest, but for lack of person hours to do the work.
As you've already ascertained, the amount of work required is
substantial, and would amount to a constantly moving target, since new
ports are always being added, and "vendor code" is always changing.
o Leave the ports framework as it is, and implement support for
parallel building in add-on tool, eg., portupgrade. The tool would
support automatic parallelism ("portupgrade -a" would automatically
build ports in parallel where possible), or having several
user-created instances running at the same time. I'll call this
"tool-based macro-parallelism".
I specifically designed portmaster to be able to do this. Separate
instances do not share anything in common, other than the ports tree
and package database. Assuming that you were updating two (or more)
ports that you were positive did not share any dependencies in common,
you could run as many portmaster instances as you wanted to, and not
have any problems.
Where this gets really sticky is in (for example) the -a case. If you
run 'portmaster -l' on a typical system, you'll see that there are
some root ports (no dependencies, not depended on), some leaf ports
(not depended on), but most of the installed ports are in the
"middle," where they are depended on by something else, and most of
those have dependencies of their own. It would be a SMOP to have
portmaster track the ports that are "safe" to upgrade in parallel.
Then of course you have to keep track of your child processes, create
a queue for what ports to run next, etc. etc. It's all doable of
course, but I think the return on investment for this project would be
very small. Patches are always welcome however. :)
Disadvantages:
- Moderately difficult to implement.
*cough*
Locking, build failures and interruptions would have to be taken
care of. I don't see problems that can't be solved though.
Yeah, like I said, it's all about having the time to do it.
- No speed gain when updating single large ports, eg. gcc. (To be
fair, it must be said that some of the large ports, eg.
OpenOffice.org, don't support micro-parallelism either. Macro-
parallelism would at least allow the otherwise unused CPUs to
do something sometimes.)
My gut feeling is that the cases where the gains outweigh the effort
required are very few, and far between. Maybe someday when 64-way
desktops are the norm, sure, but not now. :) I would of course not
ever discourage someone who wanted to work on this, it's just not on
my TODO list atm.
hth,
Doug
--
This .signature sanitized for your protection
_______________________________________________
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
