I don't know whether there are similar limitations of parallelism when launching threads via sends to agents. I haven't looked at that yet. If you have an example program you can share, preferably trimmed down to the core of the issue, I might be able to look at it.
I only know about pmap performance in such detail from using it on computer language benchmarks game Clojure programs, and trying to figure out why they weren't finishing sooner. It helped to print messages when threads started and ended, examine that output, and then read pmap source code to figure out why it explained that output. Andy On Sat, Sep 24, 2011 at 9:16 AM, Lee Spector <lspec...@hampshire.edu> wrote: > > Thanks for this info -- I didn't realize quite how pmap worked. > > I often launch parallel threads with pmap and have sometimes been puzzled > by dips in processor utilization that I can't trace to memory resource > contention, etc. > > I have similar issues sometimes when I launch parallel threads via sends to > agents. Will this behave similarly to pmap? If so, is there a > straightforward way to get the same kind of benefit as medusa-pmap in an > agent context? > > -Lee > > > On Sep 22, 2011, at 11:34 PM, Andy Fingerhut wrote: > > > pmap will limit the maximum number of simultaneous threads. So will the > medusa library's medusa-pmap. > > > > The difference is that if one job early in the list takes, e.g., 100 > times longer than the next 20, and you have 4 cpus available, pmap will > start the first (4+2)=6 in parallel threads, let jobs 2 through 6 complete, > and then wait for the first one to finish before starting number 7. Thus > most of the time will be spent running one thread. This has the advantage > of limiting the memory required to store intermediate results, but the > disadvantage of low usage of multiple cpu cores. > > > > Medusa has a medusa-pmap that will also limit the parallelism, but it > lets you pick the level of parallelism (it isn't fixed at (# cpus + 2)), and > it will continue starting new threads, even if that means using more memory > when one job takes much longer than the following jobs. > > > > If you like pmap's behavior except for the number of threads, you can > always copy its source code into your own program and change that number > pretty easily. > > > > Andy > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
