On Aug 4, 2010, at 12:38 PM, David Nolen wrote:
>
> Why not just replace burn with a large number of arithmetic operations?
> Memory issues have a less chance of coming into play then.
>
> Primitive arithmetic operations. Like (+ (int a) (int b))
In some of my previous attempts to do that the arithmetic was being optimized
away or cached somewhere, and it would get much faster on subsequent
iterations. But this seems to be reliably slow enough:
(defn burn
([] (dotimes [i 1E4]
(* (int i)
(+ (int i)
(- (int i)
(/ (int i)
(inc (int i))))))))
([_] (burn)))
Running that with my tests (results below) seems to indicate that most of the
weirdest problems are indeed allocation/garbage-related. I now get better
better performance from the concurrent version than from the sequential
versions in all cases, even if it's not as good as I might hope for.
I'm still concerned about a couple of things, though, like:
- How to avoid bizarre slowdowns in the context of concurrency with garbage,
some of which seems to depend on the way in which the threads are launched and
some of which doesn't? I
- How can I get more performance out of my 48 core machine than my 16 core
machine? I basically see no advantage in these tests even for the arithmetic
burn function, even though for my largest ones I should have ~48 concurrent
threads on the 48-core machine. Granted, the CPUs in my 48-core machine are a
little slower -- 48 x 1.86 GHz as opposed to 16 x 2.21 GHz -- but the clock
rate difference is small compared to the core-count difference in the opposite
direction, so for my runs with 48 concurrent burns I should be getting
noticeably better overall performance on the 48 core machine than on the 16
core machine. Instead it's about the same, or maybe just a tiny bit better.
- Overall, is there a best way -- idiomatically and with respect to performance
--to launch the kinds of processes I'm launching here (independent,
individually expensive, and potentially many more of them than there are cores
in a large-N multicore environment)?
Thanks,
-Lee
TESTS with arithmetic burn function:
16 core machine:
4 sequential burns: "Elapsed time: 2023.716 msecs"
4 sequential burns: "Elapsed time: 1367.664 msecs"
4 sequential burns: "Elapsed time: 1361.962 msecs"
4 burns via pmap: "Elapsed time: 522.651 msecs"
4 burns via pmap: "Elapsed time: 491.507 msecs"
4 burns via pmap: "Elapsed time: 460.616 msecs"
4 burns via futures: "Elapsed time: 451.83 msecs"
4 burns via futures: "Elapsed time: 444.816 msecs"
4 burns via futures: "Elapsed time: 461.541 msecs"
4 burns via agents: "Elapsed time: 505.476 msecs"
4 burns via agents: "Elapsed time: 515.438 msecs"
4 burns via agents: "Elapsed time: 516.342 msecs"
16 sequential burns: "Elapsed time: 5701.441 msecs"
16 sequential burns: "Elapsed time: 4954.157 msecs"
16 sequential burns: "Elapsed time: 5564.551 msecs"
16 burns via pmap: "Elapsed time: 1020.446 msecs"
16 burns via pmap: "Elapsed time: 1020.503 msecs"
16 burns via pmap: "Elapsed time: 1009.662 msecs"
16 burns via futures: "Elapsed time: 994.616 msecs"
16 burns via futures: "Elapsed time: 978.923 msecs"
16 burns via futures: "Elapsed time: 978.161 msecs"
16 burns via agents: "Elapsed time: 1005.983 msecs"
16 burns via agents: "Elapsed time: 1010.817 msecs"
16 burns via agents: "Elapsed time: 950.932 msecs"
48 sequential burns: "Elapsed time: 17909.693 msecs"
48 sequential burns: "Elapsed time: 18366.431 msecs"
48 sequential burns: "Elapsed time: 18475.038 msecs"
48 burns via pmap: "Elapsed time: 2990.162 msecs"
48 burns via pmap: "Elapsed time: 2734.936 msecs"
48 burns via pmap: "Elapsed time: 2559.694 msecs"
48 burns via futures: "Elapsed time: 2520.915 msecs"
48 burns via futures: "Elapsed time: 2500.447 msecs"
48 burns via futures: "Elapsed time: 2482.992 msecs"
48 burns via agents: "Elapsed time: 2505.813 msecs"
48 burns via agents: "Elapsed time: 2528.694 msecs"
48 burns via agents: "Elapsed time: 2495.15 msecs"
48 core machine:
4 sequential burns: "Elapsed time: 2906.824 msecs"
4 sequential burns: "Elapsed time: 2262.275 msecs"
4 sequential burns: "Elapsed time: 2157.746 msecs"
4 burns via pmap: "Elapsed time: 1184.676 msecs"
4 burns via pmap: "Elapsed time: 1093.262 msecs"
4 burns via pmap: "Elapsed time: 1040.358 msecs"
4 burns via futures: "Elapsed time: 1065.38 msecs"
4 burns via futures: "Elapsed time: 1076.381 msecs"
4 burns via futures: "Elapsed time: 1122.641 msecs"
4 burns via agents: "Elapsed time: 1038.869 msecs"
4 burns via agents: "Elapsed time: 1030.148 msecs"
4 burns via agents: "Elapsed time: 1032.669 msecs"
16 sequential burns: "Elapsed time: 9083.289 msecs"
16 sequential burns: "Elapsed time: 8876.55 msecs"
16 sequential burns: "Elapsed time: 8886.605 msecs"
16 burns via pmap: "Elapsed time: 1716.749 msecs"
16 burns via pmap: "Elapsed time: 1328.807 msecs"
16 burns via pmap: "Elapsed time: 1247.66 msecs"
16 burns via futures: "Elapsed time: 1126.193 msecs"
16 burns via futures: "Elapsed time: 1169.753 msecs"
16 burns via futures: "Elapsed time: 1145.393 msecs"
16 burns via agents: "Elapsed time: 1206.185 msecs"
16 burns via agents: "Elapsed time: 1349.857 msecs"
16 burns via agents: "Elapsed time: 1232.605 msecs"
48 sequential burns: "Elapsed time: 36872.745 msecs"
48 sequential burns: "Elapsed time: 34877.353 msecs"
48 sequential burns: "Elapsed time: 34493.804 msecs"
48 burns via pmap: "Elapsed time: 2899.733 msecs"
48 burns via pmap: "Elapsed time: 2387.106 msecs"
48 burns via pmap: "Elapsed time: 2317.284 msecs"
48 burns via futures: "Elapsed time: 2418.022 msecs"
48 burns via futures: "Elapsed time: 2346.071 msecs"
48 burns via futures: "Elapsed time: 2314.774 msecs"
48 burns via agents: "Elapsed time: 2407.47 msecs"
48 burns via agents: "Elapsed time: 2319.749 msecs"
48 burns via agents: "Elapsed time: 2401.674 msecs"
--
Lee Spector, Professor of Computer Science
School of Cognitive Science, Hampshire College
893 West Street, Amherst, MA 01002-3359
lspec...@hampshire.edu, http://hampshire.edu/lspector/
Phone: 413-559-5352, Fax: 413-559-5438
Check out Genetic Programming and Evolvable Machines:
http://www.springer.com/10710 - http://gpemjournal.blogspot.com/
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
