On Tuesday, July 2, 2013 2:13:04 PM UTC-5, Jim foo.bar wrote: > On 02/07/13 20:05, Brian Kirkbride wrote: > > > > One thing that I really like about blocking in acquire vs submitting > > everything to an ExecutorService/CompletionService right off the bat: > > you can bail out of the remainder of the computation when one part > > fails. I've got a function that creates a lazy Seq that will become > > empty if any application of (f x) fails. That means you don't even > > schedule the remaining items. You can, of course, check and abort > > inside of (f x) too. But there's a lot to like about not processing > > those items at all. Especially in composed pipelines of functions. > > aaa ok now we're getting slightly off just 'mapping'. I thought all you > wanted to do was a parallel map - I didn't realise there is more logic > to it (aborting and stuff).... > now it makes sense why you won't like something like this either which > can use hundreds of threads no problem :) > > (->> coll > (map agent) > (map #(send-off % f))) > > Jim >
Forget that I mentioned the short-circuiting, it's nice but orthogonal to my question. :) Sure, there are a lot of ways to kick off 100,000 threads in Clojure. Most Clojure examples that I've found of non-CPU-bounded parallelism fire off a future or agent for each item. For reasons in addition to avoiding memory exhaustion, I'd like to avoid that. Dropping down to CompletionService and friends is definitely an option, but my goal was to find a simple and flexible solution implemented (as far as possible) using Clojure primitives. Are most people using Executors for this sort of thing? If so, I'm surprised there isn't some small lib wrapping it up. Any major faults in the Semaphore approach? Did I miss a simpler way to split the work of processing N items over K threads and wait for it? -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
