On 31 May 2010 06:12, Zak Wilson <zak.wil...@gmail.com> wrote: > The trouble with pmap is that it only works well with a slow function > and a short sequence. In trivial tests, it seems to be best if the > sequence has as many elements as you do cores. > > I've been experimenting with things that are like pmap, but work > better in situations that I care about. I'm having trouble getting > full performance out of a quad-core machine, but dual-core is working > pretty well so far. Here's some discussion with a link to sample code: > http://groups.google.com/group/clojure/browse_thread/thread/963cdb76f0c3c178 > > And here's my personal utility file containing mostly re- > implementations of things I didn't know where to find. The zpmap > function is an eager parallel map that works well on large vectors and > maps with a fairly fast function: > http://github.com/zakwilson/zutil-clj/blob/master/util.clj
Interesting! Reducing the number of parallel threads in my test case from 10 to 2 improved the speed of the test from 12 seconds to 10, which is about as good as can be expected. And it immediately steps back up to 12 seconds when I use 3 threads. So for CPU intensive tasks, it seems that having precisely the same number of threads as CPUs gives the best performance. It looks like Java schedules 1 thread per CPU, until it can't, and at that point scheduling overhead kicks in (costing something like 30% performance). How did you arrive at the "number of CPUs + 2" approach in zpmap? I'd have thought that the +2 would hurt performance as it did in my case. So even when Clojure makes threading as easy as possible, it's still complicated in subtle ways. Ah, well :-) Thanks for the help. Paul. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
