Hi Paul, This might be an interesting contribution to clojure.core.reducers. I haven't looked at your code in detail, so I can't say for sure, but being able to do parallel fold over semi-lazy sequences would be very useful.
-S On Tuesday, March 12, 2013 9:34:43 AM UTC-4, Paul Butcher wrote: > > So this turned out to be pretty easy. I've implemented a function called > "foldable-seq" that takes a lazy sequence and turns it into something that > can be folded in parallel. I've checked an example program that uses it to > count words in a Wikipedia XML dump into GitHub: > > https://github.com/paulbutcher/foldable-seq > > The code for foldable-seq is here: > > > https://github.com/paulbutcher/foldable-seq/blob/master/src/wordcount/reducers.clj#L60 > > On my 4-core MacBook Pro, I see a 40 second runtime without parallel-seq, > 13 seconds with. > > -- > paul.butcher->msgCount++ > > Snetterton, Castle Combe, Cadwell Park... > Who says I have a one track mind? > > http://www.paulbutcher.com/ > LinkedIn: http://www.linkedin.com/in/paulbutcher > MSN: pa...@paulbutcher.com <javascript:> > AIM: paulrabutcher > Skype: paulrabutcher > > On 11 Mar 2013, at 13:38, Paul Butcher <pa...@paulbutcher.com<javascript:>> > wrote: > > On 11 Mar 2013, at 11:00, Marko Topolnik <marko.t...@gmail.com<javascript:>> > wrote: > > The idea is to transform into a lazy sequence of eager chunks. That > approach should work. > > > Exactly. Right - I guess I should put my money where my mouth is and see > if I can get it working... > > -- > paul.butcher->msgCount++ > > Snetterton, Castle Combe, Cadwell Park... > Who says I have a one track mind? > > http://www.paulbutcher.com/ > LinkedIn: http://www.linkedin.com/in/paulbutcher > MSN: pa...@paulbutcher.com <javascript:> > AIM: paulrabutcher > Skype: paulrabutcher > > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
