On Tuesday, July 2, 2013 1:41:14 PM UTC-5, Jim foo.bar wrote: > > On 02/07/13 18:30, Brian Kirkbride wrote: > > In the past I had used promises that were fulfilled by a FixedThreadPool. > But that's a lot of setup and teardown that distracts from what you're > actually trying to achieve. > > > > no it's not...you define it once you use it forever! :) > > (defn pool-map > "A saner, more disciplined version of pmap. > Submits jobs eagerly but polls for results lazily. > Don't use if original ordering of 'coll' matters." > ([f coll threads] > (let [exec (Executors/newFixedThreadPool threads) > pool (ExecutorCompletionService. exec) > futures (try (mapv (fn [x] (.submit pool #(f x))) coll) > (finally (.shutdown exec)))] > (repeatedly (count < > span class="nv" style="color: rgb(0, 128, 128);">futures) #(.. pool take > get)))) > ([f coll] > (pool-map f coll (+ 2 cpu-no)))) > > > Jim >
Absolutely true! I do like relying on future rather than creating and shutting down a ThreadPool for each run, though. And, as you mentioned, ordering is preserved with the Semaphore approach. One thing that I really like about blocking in acquire vs submitting everything to an ExecutorService/CompletionService right off the bat: you can bail out of the remainder of the computation when one part fails. I've got a function that creates a lazy Seq that will become empty if any application of (f x) fails. That means you don't even schedule the remaining items. You can, of course, check and abort inside of (f x) too. But there's a lot to like about not processing those items at all. Especially in composed pipelines of functions. -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
