Hi there, I'm currently testing a map/pmap performance and run into weird behavior in Clojure running on a OSX (compered to Linux). Running a 'slow' (processor intensive pmap function on OSX suffers from very poor performance whereas the same function on Linux runs just fine. At the first glance looks like there is a oversubsription of threads taking place on OSX and I don't fully understand if the problem is part of my code or is this perhaps a bug on OSX platform. (Clojure 1.4)
Here is the slow function, I need to stress it that using Thread/ sleep to create dummy long running job doesn't seem to cause this problem. Realizing long sequence made a trick for me and force all procsc to run on 100%. ; Processor intensive process which will run in parallel ; Notice Thread/sleep is not used - since it doesn't use processor (more appropriate for imitating I/O stuff) (defn myFunc [input] ;(Thread/sleep 500) ;; this doesn't use processor (doseq [n (take 10000000 (range))]) ;; some slow dummy process (println (str "---" input)) ;; some side effect to track down roughly the progress ) ...and the results: OSX - dual-core: (time (doall (pmap #(myFunc %) (take 4 (range))))) "Elapsed time: 30263.3 msecs" (time (doall (map #(myFunc %) (take 4 (range))))) "Elapsed time: 9239.3 msecs" Linux 12core - the vector to iterate over has been increased to avoid tha case where there are more cores then tasks: (time (doall (pmap #(myFunc %) (take 100 (range))))) "Elapsed time: 16408.8 msecs" (time (doall (map #(myFunc %) (take 100 (range))))) "Elapsed time: 150310.1 msecs" I've got more examples for OSX which clearly shows that as soon as the number of tasks exceeds number of cores pmap performance suffers. It seems to me like there is no blocking taking place on threads and all the tasks are started at the same time. I've also run a test using Agents (pardon my code style) but the problem seems to show up there too. All tasks in the range gets submitted instantly causing oversubscribtion, so no task blocking is taking a place. ;forced single threaded (time (doseq [x (take 100 (range))] (send (agent 0) (myFunc x)) ; still single threaded... - I wonder if there is a better way to do that? )) ; parallel (time (doseq [x (take 100 (range))] ;(send (agent x) myFunc ) ; )) With Agents, on linux parallel was way faster whereas on OSX a magnitude slower: linux parallel thr - 1min 52s linux single thr - 9min 35s Thanks, Kuba -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
