I think the suggestion from the OP is to change the computational executor pool in Agent from an Executors to a ForkJoinPool to get better scalability. Indeed, the work Doug has been in doing is in response to many libs using fork/join as an alternative to Executors to schedule lightweight computational tasks, GPars in Groovy and Scala actors being the obvious examples. We're using fork/join similarly in our processing engine at Revelytix (in Clojure).
The benefit of using fork/join in this scenario is to eliminate the possible bottleneck of a single entry queue for all agent task executions, which is an inherent part of using Executors. On chunkier tasks or smaller core counts (the original design space for Executors), this is not an issue, but I believe the pool under agent send (but probably not send-off) would be a good match. The pool under send-off (and future) is probably not as good for this as it is assumed to be IO-bound and may block, which fork/join does not generally like. We have used the fork/join ManagedBlocker api to get around this in some cases though. If someone wanted to work on it, the changes seem isolated to Agent.java so this is a very tractable problem. I think that David Liebke's f/j work has largely been to look at using f/j for parallel execution over data structures ala pmap, which is an entirely different problem (but also one of great interest). On Jan 29, 7:34 pm, Stuart Sierra <the.stuart.sie...@gmail.com> wrote: > There was some excellent work on ForkJoin in Clojure by David Liebke and > others last year, but it never made it into the main Clojure branch. It's > waiting for someone else to finish it. Search the wiki and mailing list for > notes. > > -S -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
