Re: pmap memory hogging

2009-01-21 Thread Mark Fredrickson
> It could be that the two threads are contending over the Array object > reference (Java arrays aren't pointers). Is there a nice way to > create "subvector" objects that only reference the underlying memory > and not the parent array object? You might be interested in the *Buffer classes (e.g.

Re: pmap memory hogging

2009-01-21 Thread chris
Is the mmap interface a possibility? You could bulkget an array for each thread. I guess that would probably not gain much against just copying an array for each thread. I couldn't find a way to create a subvector of a java array. Chris On Jan 21, 12:20 pm, "Mark H." wrote: > On Jan 21, 8:56

Re: pmap memory hogging

2009-01-21 Thread Mark H.
On Jan 21, 8:56 am, Perry Trolard wrote: > If I understand what you did correctly, the reason it worked for you > is that Rich committed a fix to SVN in the meantime! Awesome, real-time fix! > > The last (count result-p) call takes a few seconds (probably > > because there's a one-to-one mappin

Re: pmap memory hogging

2009-01-21 Thread Perry Trolard
Thanks for testing, Mark. If I understand what you did correctly, the reason it worked for you is that Rich committed a fix to SVN in the meantime! > The last (count result-p) call takes a few seconds (probably > because there's a one-to-one mapping of tasks to sequence > elements) but it finish

Re: pmap memory hogging

2009-01-20 Thread Mark H.
On Jan 20, 8:36 am, Perry Trolard wrote: > Yesterday I had a strange case of pmap (parallel map) eating up all of > the heap space I'd make available -- from 256M up to a Gig. I track > SVN, & I know for sure this wasn't happening last Wednesday (the > 14th). Calling map in place of pmap caused t

Re: pmap memory hogging

2009-01-20 Thread Mark H.
On Jan 20, 10:38 am, Perry Trolard wrote: > But I don't think lazyness is the problem: > > user=> (def r (doall (map inc (range 100 > #'user/r > user=> (count r) > 100 > user=> (pmap inc [0]) > java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:0) Hm, that's funny, the latte

Re: pmap memory hogging

2009-01-20 Thread Rich Hickey
On Jan 20, 1:38 pm, Perry Trolard wrote: > > Doesn't pmap have to construct the whole sequence explicitly in order > > to map its execution across more than one processor? or does it take > > in a lazy fashion? > > Semi-lazy, according to the doc: > > Like map, except f is applied in parallel

Re: pmap memory hogging

2009-01-20 Thread Mark H.
On Jan 20, 10:38 am, Perry Trolard wrote: > > Doesn't pmap have to construct the whole sequence explicitly in order > > to map its execution across more than one processor?  or does it take > > in a lazy fashion? > > Semi-lazy, according to the doc: > >  Like map, except f is applied in parallel.

Re: pmap memory hogging

2009-01-20 Thread Perry Trolard
> Doesn't pmap have to construct the whole sequence explicitly in order > to map its execution across more than one processor?  or does it take > in a lazy fashion? Semi-lazy, according to the doc: Like map, except f is applied in parallel. Semi-lazy in that the parallel computation stays ahe

Re: pmap memory hogging

2009-01-20 Thread Mark H.
On Jan 20, 8:36 am, Perry Trolard wrote: > Yesterday I had a strange case of pmap (parallel map) eating up all of > the heap space I'd make available -- from 256M up to a Gig. I track > SVN, & I know for sure this wasn't happening last Wednesday (the > 14th). Calling map in place of pmap caused t