Re: Parallel builds, build locks, pkg_dbdir locked

2009-04-09 Thread Jeremy Lea
Hi, On Thu, Apr 02, 2009 at 05:57:05PM -0400, David Forsythe wrote: > patch: http://dmz2.khome.utcorp.net/~dforsyth/port.mk-locks.diff Since no one else seems to have commented, a few quick thoughts. I have not actually tried your patch. 1. Is there any reason not to do the locking? Removing N

Re: parallel builds revisited

2007-04-14 Thread Ulrich Spoerlein
Pav Lucistnik wrote: > Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > > > Some time ago, after buying a Core 2 Duo system, I've become interested > > in doing something about the inherent single-threadedness of the ports. > > Even though I have a dualcore machine, ports builds only ever u

Re: parallel builds revisited

2007-04-13 Thread Kris Kennaway
On Sat, Apr 14, 2007 at 07:12:21AM +0200, [LoN]Kamikaze wrote: > Kris Kennaway wrote: > > On Thu, Apr 12, 2007 at 09:31:37PM +0200, [LoN]Kamikaze wrote: > > > >>> Btw, do you think it's possible that a port can only be built with, n > >>> parallel make jobs, but will fail with n+1? > >> No. I do

Re: parallel builds revisited

2007-04-13 Thread [LoN]Kamikaze
Kris Kennaway wrote: > On Thu, Apr 12, 2007 at 09:31:37PM +0200, [LoN]Kamikaze wrote: > >>> Btw, do you think it's possible that a port can only be built with, n >>> parallel make jobs, but will fail with n+1? >> No. I do not think this can be the case. > > It certainly is the case. If a makefi

Re: parallel builds revisited

2007-04-13 Thread Kris Kennaway
On Fri, Apr 13, 2007 at 04:51:33PM -0700, [EMAIL PROTECTED] wrote: > My opinion is that there should be a threshold value empirically derived by > the developer / retrieved by bug reports, as well as a knob, to specify the > maximum number of parallel jobs to be used for a particular port, that

Re: parallel builds revisited

2007-04-13 Thread Kris Kennaway
On Thu, Apr 12, 2007 at 09:31:37PM +0200, [LoN]Kamikaze wrote: > > Btw, do you think it's possible that a port can only be built with, n > > parallel make jobs, but will fail with n+1? > > No. I do not think this can be the case. It certainly is the case. If a makefile has incorrectly specifie

Re: parallel builds revisited

2007-04-13 Thread youshi10
On Fri, 13 Apr 2007, David Nečas (Yeti) wrote: On Fri, Apr 13, 2007 at 05:10:47PM +0200, Pav Lucistnik wrote: Peter Pentchev píše v pá 13. 04. 2007 v 18:06 +0300: > > I was thinking about having it embedded in every port's Makefile > directly, instead. Something like > > USE_MAKE_JOBS= 2

Re: parallel builds revisited

2007-04-13 Thread [LoN]Kamikaze
Peter Jeremy wrote: > Can anyone come up with either examples of ports that fall into the > second category above or counter-examples to my first paragraph? I have used 'make -j' for quite some time, and to my experience it either works or doesn't. I've never seen a port that works sometimes. __

Re: parallel builds revisited

2007-04-13 Thread Peter Jeremy
On 2007-Apr-13 17:43:54 +0200, "David Ne?as (Yeti)" <[EMAIL PROTECTED]> wrote: >implements it. How many ports exist that can fail with N+1 >jobs yet cannot break with N jobs (for N > 1)? As far as I can see, for a port to be safely built with N>1 jobs then it needs to have proper dependency relat

Re: parallel builds revisited

2007-04-13 Thread Yeti
On Fri, Apr 13, 2007 at 05:10:47PM +0200, Pav Lucistnik wrote: > Peter Pentchev píše v pá 13. 04. 2007 v 18:06 +0300: > > > > > > I was thinking about having it embedded in every port's Makefile > > > directly, instead. Something like > > > > > > USE_MAKE_JOBS=2 > > > > IMHO, hardcoding the

Re: parallel builds revisited

2007-04-13 Thread Pav Lucistnik
Peter Pentchev píše v pá 13. 04. 2007 v 18:06 +0300: > On Tue, Apr 10, 2007 at 07:44:47PM +0200, Pav Lucistnik wrote: > > Benjamin Lutz p??e v ?t 10. 04. 2007 v 04:52 +0200: > [snip] > > > 3) Save this to /usr/local/etc/parallel_builds.conf: > > > http://www.maxlor.com/temp/parallel_builds.c

Re: parallel builds revisited

2007-04-13 Thread Peter Pentchev
On Tue, Apr 10, 2007 at 07:44:47PM +0200, Pav Lucistnik wrote: > Benjamin Lutz p??e v ?t 10. 04. 2007 v 04:52 +0200: [snip] > > 3) Save this to /usr/local/etc/parallel_builds.conf: > > http://www.maxlor.com/temp/parallel_builds.conf . > > This is a list of ports as stored in PKGORIGIN,

Re: parallel builds revisited

2007-04-12 Thread Coleman Kane
On Thu, 2007-04-12 at 21:07 +0200, Benjamin Lutz wrote: > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > > I dunno how you want to approach this, but gmake does recommend 2 > > jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a > > dual core machines. > > -Garrett > > S

Re: parallel builds revisited

2007-04-12 Thread youshi10
On Thu, 12 Apr 2007, Benjamin Lutz wrote: On Thursday 12 April 2007 22:20, Pav Lucistnik wrote: Benjamin Lutz píše v čt 12. 04. 2007 v 21:07 +0200: On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > I dunno how you want to approach this, but gmake does recommend 2 > jobs be run in parall

Re: parallel builds revisited

2007-04-12 Thread youshi10
On Thu, 12 Apr 2007, [LoN]Kamikaze wrote: Benjamin Lutz wrote: On Thursday 12 April 2007 11:06, Garrett Cooper wrote: I dunno how you want to approach this, but gmake does recommend 2 jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a dual core machines. -Garrett So far the

Re: parallel builds revisited

2007-04-12 Thread Benjamin Lutz
On Thursday 12 April 2007 22:20, Pav Lucistnik wrote: > Benjamin Lutz píše v čt 12. 04. 2007 v 21:07 +0200: > > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > > > I dunno how you want to approach this, but gmake does recommend 2 > > > jobs be run in parallel for HTT enabled chips, and 3 o

Re: parallel builds revisited

2007-04-12 Thread youshi10
On Thu, 12 Apr 2007, Benjamin Lutz wrote: On Thursday 12 April 2007 18:32, [LoN]Kamikaze wrote: Robert Noland wrote: Have any of you looked at sysutils/bsdadminscripts, it's buildflags options allow for parallel builds as well as ccache / distcc use. I have a reasonable list of ports that m

Re: parallel builds revisited

2007-04-12 Thread Pav Lucistnik
Benjamin Lutz píše v čt 12. 04. 2007 v 21:07 +0200: > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > > I dunno how you want to approach this, but gmake does recommend 2 > > jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a > > dual core machines. > > -Garrett > > So fa

Re: parallel builds revisited

2007-04-12 Thread [LoN]Kamikaze
Benjamin Lutz wrote: > On Thursday 12 April 2007 11:06, Garrett Cooper wrote: >> I dunno how you want to approach this, but gmake does recommend 2 >> jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a >> dual core machines. >> -Garrett > > So far the approach is one job per CPU.

Re: parallel builds revisited

2007-04-12 Thread Benjamin Lutz
On Thursday 12 April 2007 11:06, Garrett Cooper wrote: > I dunno how you want to approach this, but gmake does recommend 2 > jobs be run in parallel for HTT enabled chips, and 3 or 4 jobs for a > dual core machines. > -Garrett So far the approach is one job per CPU. I'll do some benchmarks lateron

Re: parallel builds revisited

2007-04-12 Thread [LoN]Kamikaze
Benjamin Lutz wrote: > I've looked at your patches and programs, and I'm starting to have a > fairly clear idea of solution should look like. I would like to see: > > * Integration into the existing ports framework. No new scripts or files > should be required. The whitelist file needs to go.

Re: parallel builds revisited

2007-04-12 Thread Benjamin Lutz
On Thursday 12 April 2007 18:32, [LoN]Kamikaze wrote: > Robert Noland wrote: > > Have any of you looked at sysutils/bsdadminscripts, it's buildflags > > options allow for parallel builds as well as ccache / distcc use. > > I have a reasonable list of ports that must have some or all of > > these o

Re: parallel builds revisited

2007-04-12 Thread [LoN]Kamikaze
Robert Noland wrote: > On Thu, 2007-04-12 at 05:43 +0200, Benjamin Lutz wrote: >> Hello Pav, >> >> On Tuesday 10 April 2007 19:44, Pav Lucistnik wrote: >>> Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: Some time ago, after buying a Core 2 Duo system, I've become interested in doing

Re: parallel builds revisited

2007-04-12 Thread Robert Noland
On Thu, 2007-04-12 at 05:43 +0200, Benjamin Lutz wrote: > Hello Pav, > > On Tuesday 10 April 2007 19:44, Pav Lucistnik wrote: > > Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > > > Some time ago, after buying a Core 2 Duo system, I've become > > > interested in doing something about the inh

Re: parallel builds revisited

2007-04-12 Thread Garrett Cooper
Pav Lucistnik wrote: Benjamin Lutz píše v čt 12. 04. 2007 v 05:43 +0200: Is there any detailed information available on what's planned here that isn't in your description on SoC page? I don't know if I can forward you the proposal text, they might not be public. But it's basically co

Re: parallel builds revisited

2007-04-12 Thread Pav Lucistnik
Benjamin Lutz píše v čt 12. 04. 2007 v 05:43 +0200: > Is there any detailed information available on what's planned here > that > isn't in your description on SoC page? I don't know if I can forward you the proposal text, they might not be public. But it's basically copy&paste from the Ideas pag

Re: parallel builds revisited

2007-04-11 Thread Benjamin Lutz
Hello Pav, On Tuesday 10 April 2007 19:44, Pav Lucistnik wrote: > Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > > Some time ago, after buying a Core 2 Duo system, I've become > > interested in doing something about the inherent > > single-threadedness of the ports. Even though I have a dua

Re: parallel builds revisited

2007-04-10 Thread Pav Lucistnik
Benjamin Lutz píše v út 10. 04. 2007 v 04:52 +0200: > Some time ago, after buying a Core 2 Duo system, I've become interested > in doing something about the inherent single-threadedness of the ports. > Even though I have a dualcore machine, ports builds only ever use one > core. I started think

Re: Parallel Builds

2006-10-21 Thread Ulrich Spoerlein
Doug Barton wrote: > > o Leave the ports framework as it is, and implement support for > > parallel building in add-on tool, eg., portupgrade. The tool would > > support automatic parallelism ("portupgrade -a" would automatically > > build ports in parallel where possible), or having several

Re: Parallel Builds

2006-10-19 Thread Doug Barton
Benjamin Lutz wrote: Hello, Since Multi-core processors are becoming popular (or, more egocentrically, since I've acquired one), I've become interested in parallel compilation. Unfortunately, it seems that parallel builds of any kind are completely unsupported by the ports framework at the m

Re: Parallel Builds

2006-10-19 Thread RW
On Thursday 19 October 2006 10:24, Benjamin Lutz wrote: >.. > o Mark the ports that allow parallel building by adding a new flag >that can be used in ports makefiles, eg. PARALLEL_BUILDING=yes. >With such a port, the build target would call, say >"gmake -j${PARALLEL_NUM}" instead of ju

Re: Parallel Builds

2006-10-19 Thread [LoN]Kamikaze
Benjamin Lutz wrote: > ... > > I'm sure I'm not the only person that has thought about this. Maybe > there already is an effort to allow for parallelism in port builds. German speaking people might be interested in the following articles: a) http://wiki.bsdforen.de/index.php/Bsdadminscripts#buil