Hi all,
I've been using the following fn quite extensively lately and I'm just
wondering if you can see any flaws with it...or perhaps something that
can be done better? I generally find that this performs better than a
bare pmap and in fact it often performs better than my 'pool-map' (which
I'm not showing here) which is backed by executors.
(defn mapr
"A basic map-reduce style mapping-function. Will partition the data
according to p-size and assign a future to each partition (per pmap)."
([f coll p-size]
(->> coll
(partition-all p-size)
(pmap (fn [p] (reduce #(conj %1 (f %2)) [] p)) )
(apply concat)) ) ;;concat the inner vectors that represent the
partitions
([f coll]
(mapr f coll (+ 2 cpu-no))))
thanks,
Jim
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
