Recently I worked on an algorithm where a distributed tree is (sort of) flattened in a way that each node runs a commutative aggregation over all of its child nodes calculations.
1 A node can obviously not pmap over all the child nodes (would spawn exponential amount of threads). 2 If I want to limit the threads using a threadpool, child nodes will quickly run out of threads and we can't reuse their parents threads because they are blocking, waiting for the child nodes to complete, calculating nothing. It sure could be possible to implement this with an unlimited threadpool and some machinery that ensures that no more than N computations are happening in parallel. Nonetheless an insane amount of threads would be spawned. (2) Made me wonder why I couldn't use the go machinery for this. Parent nodes that would "wait" for their child nodes to complete would park, making their thread available to their child nodes. So of course I could just do it, but the per node computations are presumably too heavy for the lightweight go threadpool. How realistic/useful would it be to have sth. like a (go-with threadpool []) in this scenario? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
