On 21 March 2010 13:29, Andrzej <ndrwr...@googlemail.com> wrote: > Yesterday I looked at the implementation of the PersistentVector > class, trying to figure out how to exploit its internal structure to > decompose the vector. I hit several issues though:
One thing that comes to my mind after reading through your list is that when this sort of "tree parallelisation" is actually worth while, with large datasets and computationally expensive commutative operations, it probably doesn't matter much whether the chunks are of equal size or not, as long as they're roughly comparable. Even with equal sized chunks, with some interesting operations there is no guarantee that any two chunks will take roughly the same amount of time to process; even something simple like multiplication through the Clojure built-in * will take no time at all with small arguments and take forever when dealing with huge BigIntegers. Ultimately, in the general case, one needs to make sure that the number of chunks is large enough for differences in actual workload from chunk to chunk not to become problematic, yet small enough for bookkeeping overhead to stay significantly smaller then the expense of performing the main computation. Clearly there is room here for user-supplied parameters. Not sure if I want to draw any particular conclusion from this... Probably not, since I'm not yet done wondering to which degree I might be correct in thinking it. :-) I do want to stress once again the need for benchmarking with expensive tasks, though. Addition is so cheap that it's guaranteed not to be worth the bookkeeping cost of fancy work splitting solutions. All the best, Michał -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en To unsubscribe from this group, send email to clojure+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
