Thanks all for advise and sorry for the double post. Will check this out tonight and get back.
Cheers, kuba On Wednesday, May 1, 2013 11:09:05 AM UTC-7, kuba roth wrote: > > Hi there, > > I'm currently testing a map/pmap performance and run into weird > behavior in Clojure running on a OSX (compered to Linux). > Running a 'slow' (processor intensive pmap function on OSX suffers > from very poor performance whereas the same function on Linux runs > just fine. > At the first glance looks like there is a oversubsription of threads > taking place on OSX and I don't fully understand if the problem is > part of my code or is this perhaps a bug on OSX platform. (Clojure > 1.4) > > Here is the slow function, I need to stress it that using Thread/ > sleep to create dummy long running job doesn't seem to cause this > problem. Realizing long sequence made a trick for me and force all > procsc to run on 100%. > > ; Processor intensive process which will run in parallel > ; Notice Thread/sleep is not used - since it doesn't use processor > (more appropriate for imitating I/O stuff) > (defn myFunc [input] > ;(Thread/sleep 500) ;; this doesn't use processor > (doseq [n (take 10000000 (range))]) ;; some slow dummy process > (println (str "---" input)) ;; some side effect to > track down roughly the progress > ) > > ...and the results: > OSX - dual-core: > (time (doall (pmap #(myFunc %) (take 4 (range))))) "Elapsed time: > 30263.3 msecs" > (time (doall (map #(myFunc %) (take 4 (range))))) "Elapsed > time: 9239.3 msecs" > > Linux 12core - the vector to iterate over has been increased to avoid > tha case where there are more cores then tasks: > (time (doall (pmap #(myFunc %) (take 100 (range))))) "Elapsed time: > 16408.8 msecs" > (time (doall (map #(myFunc %) (take 100 (range))))) "Elapsed time: > 150310.1 msecs" > > > I've got more examples for OSX which clearly shows that as soon as the > number of tasks exceeds number of cores pmap performance suffers. It > seems to me like there is no blocking taking place on threads and all > the tasks are started at the same time. > > I've also run a test using Agents (pardon my code style) but the > problem seems to show up there too. All tasks in the range gets > submitted instantly causing oversubscribtion, so no task blocking is > taking a place. > > > ;forced single threaded > (time (doseq [x (take 100 (range))] > (send (agent 0) (myFunc x)) ; still single > threaded... - I wonder if there is a better way to do that? > )) > > ; parallel > (time (doseq [x (take 100 (range))] > ;(send (agent x) myFunc ) ; > )) > > > With Agents, on linux parallel was way faster whereas on OSX a > magnitude slower: > linux parallel thr - 1min 52s > linux single thr - 9min 35s > > > Thanks, > Kuba > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
