Please evaluate this on the OSX box and report here what it says: (.availableProcessors (Runtime/getRuntime))
On Wed, May 1, 2013 at 1:58 PM, kuba roth <kuba.r...@gmail.com> wrote: > Hi there, I'm a bit nervous, this is my first post here :) > > I'm currently testing a map/pmap performance and run into weird behavior > in Clojure running on a OSX (compered to Linux). > Running a 'slow' (processor intensive pmap function on OSX suffers from > very poor performance whereas the same function on Linux runs just fine. > At the first glance looks like there is a oversubsription of threads > taking place on OSX and I don't fully understand if the problem is part of > my code or is this perhaps a bug on OSX platform. (Clojure 1.4) > > Here is the slow function, I need to stress it that using Thread/sleep to > create dummy long running job doesn't seem to cause this problem. Realizing > long sequence made a trick for me and force all procsc to run on 100%. > > ; Processor intensive process which will run in parallel > ; Notice Thread/sleep is not used - since it doesn't use processor (more > appropriate for imitating I/O stuff) > (defn myFunc [input] > ;(Thread/sleep 500) ;; this doesn't use processor > (doseq [n (take 10000000 (range))]) ;; some slow dummy process > (println (str "---" input)) ;; some side effect to track > down roughly the progress > ) > > ...and the results: > OSX - dual-core: > (time (doall (pmap #(myFunc %) (take 4 (range))))) "Elapsed time: > 30263.3 msecs" > (time (doall (map #(myFunc %) (take 4 (range))))) "Elapsed time: > 9239.3 msecs" > > Linux 12core - the vector to iterate over has been increased to avoid tha > case where there are more cores then tasks: > (time (doall (pmap #(myFunc %) (take 100 (range))))) "Elapsed time: > 16408.8 msecs" > (time (doall (map #(myFunc %) (take 100 (range))))) "Elapsed time: > 150310.1 msecs" > > > I've got more examples for OSX which clearly shows that as soon as the > number of tasks exceeds number of cores pmap performance suffers. It seems > to me like there is no blocking taking place on threads and all the tasks > are started at the same time. > > I've also run a test using Agents (pardon my code style) but the problem > seems to show up there too. All tasks in the range gets submitted instantly > causing oversubscribtion, so no task blocking is taking a place. > > > ;forced single threaded > (time (doseq [x (take 100 (range))] > (send (agent 0) (myFunc x)) ; still single > threaded... - I wonder if there is a better way to do that? > )) > > ; parallel > (time (doseq [x (take 100 (range))] > ;(send (agent x) myFunc ) ; > )) > > > With Agents, on linux parallel was way faster whereas on OSX a magnitude > slower: > linux parallel thr - 1min 52s > linux single thr - 9min 35s > > > Thanks, > Kuba > > -- > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
