On Sat, Jun 26, 2010 at 7:01 PM, toddg <t.greenwoodg...@gmail.com> wrote: > (running clojure 1.2 snapshot) > > Q1: Why does pmap use the number of available processors + 2? I would > have thought it would just use the number of avail processors...
I'm not entirely sure, but I think the idea is to prevent too much context-switching on each core. Perhaps +2 helps fill in gaps in the processor's pipelines or something. The agent pool for 'send' has a similar limit formula. > Q2: Could someone clear up my misunderstanding of pmap w/ respect to > the code snippets below? Pmap doesn't seem to be limiting the number > of threads to the number of processors + 2... > > I've created an anon func that does't return, so I wouldn't expect the > pmap step function to advance beyond 4 (I have 2 processors): > > #1: Limit pmap to # of processors > ---------------------------------- > [snip] > > --> just two threads running, as expected > > > #2: Limit pmap to # of processors * 10 > -------------------------------------- > > user=> (pmap #(while true (do (println "Thread: " (.getId (Thread/ > currentThread)) "item: " %)(Thread/sleep 500))) (range (* 10 > (.availableProcessors (Runtime/getRuntime))))) > Thread: Thread: 12 item: 0 > (Thread: 25 item: 13 > Thread: 24 item: 12 > Thread: 23 item: 11 > Thread: 22 item: 10 > Thread: 21 item: 9 > Thread: 20 item: 8 > Thread: 19 item: 7 > Thread: 18 item: 6 > Thread: 17 item: 5 > Thread: 16 item: 4 > Thread: 15 item: 3 > > --> In this short snippet, you can see > 4 threads running...expected? Range produces a chunked seq, which if my answer to Q1 is correct, I guess pmap isn't expecting. So pmap creates one 'future' per element, but instead of only doing this for procs+2 elements at a time, it does so for every element of the first chunk. If you use range of more than 32 elements, you'll see your example doesn't go beyond the first 32 (range's chunk size). ...or you can use a PersistentList instead of a range, which produces unchunked seqs, and see that it only uses procs+2 threads. --Chouser http://joyofclojure.com/ -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
