On Aug 18, 2012, at 7:27 AM, Jim - FooBar(); wrote:
> As far as pmap goes, I originally thought of starting a new future for each
> starting branch but that is what pmap does essentially, so it looked very
> handy at first...
Yes, pmap is essentially a trap in that it looks like the perfect tool for some
light parallelism, but it works so poorly in the vast majority of cases that
it's basically useless. There have been a few discussions on this list about it
in the past.
> I don't see how reducers can help here because I'm not reducing anything
> really.
You should read the link I posted about reducers. It's good stuff.
> I just wish pmap worked!
So do I. Until the reducers library is ready, you could try something like this
(no guarantees that this is an optimal implementation!):
(defn with-thread-pool* [num-threads f]
(let [pool (java.util.concurrent.Executors/newFixedThreadPool num-threads)]
(try (f pool)
(finally
(when pool (.shutdown pool))))))
(defmacro with-thread-pool [[name num-threads] & body]
`(with-thread-pool* ~num-threads (fn [~name] ~@body)))
(defn pmap-pool [f coll]
(with-thread-pool [pool (.. Runtime getRuntime availableProcessors)]
(doall
(map #(.get %)
(.invokeAll pool
(map (partial partial f)
coll))))))
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
