Although this looks like it might work, it's not exactly a good idea. Look at what you're doing inside the reducer, a call to >!! will block a OS level thread until someone takes from the channel. Since reducers use fork-join pools, I wouldn't be surprised if it caused some serious side-effects. Since reducing over things like vectors and maps happen in parallel, you could end up blocking unknown numbers of threads until these seqs get realized. So the pool may go out and create more threads, just to have those blocked by this code. Either that, or the fork/join pool will just clog up, keeping other reducers from running properly.
Timothy Baldridge On Sun, Aug 25, 2013 at 8:43 PM, Alan Busby <thebu...@gmail.com> wrote: > Here is something similar pulled from bagotricks "1.5.2", using Java's > linked blocking queue; > > https://github.com/thebusby/bagotricks/blob/master/src/bagotricks.clj#L204-L238 > > It uses fold instead of reduce to run in parallel, so has a slightly > different use case than above. > > > On Mon, Aug 26, 2013 at 10:14 AM, Mikera <mike.r.anderson...@gmail.com>wrote: > >> Nice idea Jozef! >> >> Hmmm.... this is another example of why nil-as-end-of-channel is a >> slightly problematic design decision for core.async: it makes this kind of >> code much more fiddly. >> >> >> On Monday, 26 August 2013 01:47:14 UTC+8, Jozef Wagner wrote: >>> >>> Hi, >>> >>> A distinctive feature of reducers is that "reducing is a one-shot >>> thing". The common understanding is that reducers are fast for cases where >>> you want to process whole collection at once, but for infinite and lazy >>> seqs, you have to use good old seqs. >>> >>> With core.async, it is now easy to create a transformation which >>> produces lazy seq from any reducible collection. >>> >>> (defn lazy-seq* >>> [reducible] >>> (let [c (chan) >>> NIL (Object.) >>> encode-nil #(if (nil? %) NIL %) >>> decode-nil #(if (identical? NIL %) nil %)] >>> (thread >>> (reduce (fn [r v] (>!! c (encode-nil v))) nil reducible) >>> (close! c)) >>> (take-while (complement nil?) (repeatedly #(decode-nil (<!! c)))))) >>> >>> >>> (def s (lazy-seq* (clojure.core.reducers/map inc (range)))) >>> >>> (first s) >>> >>> (take 100 s) >>> >>> This approach can be also extended to produce chunked seqs and chan >>> buffer can also be used to further tune the performance. >>> >>> JW >>> >>> -- >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth) -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
