One benefit would be convenience of enabling parallelism on nested data structures. One function at the top level could use parallelism, and the pieces, perhaps handled by separate functions, and perhaps nested several levels deep in function calls, could also use parallelism.
If it were implemented not by creating Java Threads for each task, but submitting a task to an ExecutorService, then the actual number of active Java Threads could be kept reasonably low (e.g. maybe 2 times the number of physical CPU cores), whereas the number of parallel tasks the work is divided into could be limited only by memory for storing the tasks scheduled for future execution. Andy On Tue, Oct 11, 2011 at 10:55 AM, j-g-faustus <johannes.fries...@gmail.com>wrote: > On Tuesday, October 11, 2011 3:55:09 AM UTC+2, Lee wrote: >> >> >> Does your pmap-pool permit nesting? (That is, does it permit passing >> pmap-pool a function which itself calls pmap-pool?). If so then that would >> be a reason to prefer it over my pmapall. >> > I expect it would be possible to nest it (possible as in "no exceptions or > deadlocks"), but I can't see any scenario where you would want to - you > would get an exponentially increasing number of threads. If 48 cores each > start 48 threads, each of those threads start another 48 etc., it doesn't > take long before you have enough threads to bog down even the most powerful > server. > > But what would be the purpose of a nested "run this on all cores" > construct? You are already running on all cores, there are no spare > resources, so in terms of CPU time I can't see how it would differ from > merely having the pmapped function use a plain same-thread map? > > There are no particular advantages to pmap-pool over pmapall that I can > see, except that pmap-pool lets you control the number of threads more > easily. (E.g. for debugging "where does it stop scaling linearly" problems.) > I thought creating thousands of agents (as pmapall does) would be more > expensive, so I tried an alternative, but in practice they seem to be > equally fast; at least on this test and the kind of hardware I have access > to. > So pmapall wins by having fewer lines of code. > > > jf > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
