Just for fun, I ran the (dorun (sequence (comp (mapcat #(range %)) (mapcat #(range %))) (range 1000))) and eduction version with the CLJ-1515 patch. I saw ~20 seconds before and 15 seconds after.
But the new version also makes pure seqs better and I saw the full (dorun ...) drop from 6 seconds before to 3 seconds after too. Just a hint of the benefits coming in CLJ-1515. On Thursday, April 2, 2015 at 9:21:08 AM UTC-5, Michał Marczyk wrote: > > It may be worth noting that while the return value of range is wrapped in > lazy-seq and thus isn't itself a clojure.lang.IChunkedSeq, what you get > when you realize it is indeed chunked: > > (contains? (ancestors (class (seq (range 128)))) clojure.lang.IChunkedSeq) > true > > It doesn't implement c.l.IReduce, but > clojure.core.protocols/InternalReduce has an implementation for > c.l.IChunkedSeq. At least transduce should be able to take advantage of the > InternalReduce implementation (via CollReduce). transduce could be used for > short-circuiting search with (reduced …), so it might be a legitimate > contender here. > > Cheers, > Michał > > > On 2 April 2015 at 15:38, Tassilo Horn <ts...@gnu.org <javascript:>> > wrote: > >> Alex Miller <al...@puredanger.com <javascript:>> writes: >> >> Hi Alex, >> >> > If you're going to use expanding transformations and not realize all of >> the >> > results then I think sequences are likely a better choice for you. >> >> Ok, I see. >> >> >> However, at least I had expected that in the case where all elements >> >> are realized the transducer version should have been faster than the >> >> traditional version which also needs to fully realize all >> >> intermediate lazy seqs. Why is it still three times slower? >> > >> > I think my main suggestion here is that you are using a non-reducible >> > source (range) throughout these timings, so transducers have no >> > leverage on the input side. CLJ-1515 will make range reducible and >> > should help a lot on this particular example. >> >> Well, even if I revamp the (admittedly contrieved) example to have >> reducible vectors as source and also intermediates >> >> (let [v (vec (range 0 1000)) >> vs (zipmap (range 0 1000) >> (for [i (range 0 1000)] >> (vec (range i 1000))))] >> (time (dorun (sequence (comp (mapcat (fn [i] (vs i))) >> (mapcat (fn [i] (vs i)))) >> v)))) >> >> it still takes 18 seconds instead of 21 with lazy seqs produced by >> range, or just 7 seconds with normal lazy seq functions. >> >> In my real scenario, I think there's also no IReduces paths because the >> mapcat functions either return normal lazy seqs or Java Collections >> (which are not actually clojure collections). But usually, the >> transformations are not so freaking expanding as the example above. I >> benchmarked a bit, and there sometimes using transducers is faster and >> sometimes it is not. So I've made than configurable (with normal lazy >> seqs as default) so users can benchmark and then decide, and I don't >> need to choose for them. :-) >> >> Oh, and actually *you* have made that possible by making me aware of >> >> (sequence (comp xform*) start-coll) >> >> is almost identical to >> >> (->> start-coll xform*) >> >> that is, when my macro computes xforms as if they were meant for >> transducing, I can also use them "traditionally" with ->>. >> >> Until now, I've newer used ->> but before I had implemented the >> expansion for transducers, I used a for with gensyms for intermediates >> like: >> >> (for [G__1 start-coll >> G__2 (xform1 G__1) >> G__3 (xform2 G__2)] >> G__3) >> >> That's pretty much different to generate. But since the xforms for >> transducers and ->> are the same, switching between lazy seq fns and >> transducers is just changing how start-coll and xforms are composed. >> Awesome! >> >> >> So my conclusion is that you cannot use transducers as a kind of >> >> drop-in replacement of traditional sequence manipulation functions. >> >> They pay off only when you can make very strong assumptions about the >> >> sizes and compututation costs of intermediate collections, and I >> >> think you cannot do that in general. Or well, maybe you can when you >> >> program an application but you almost certainly cannot when you >> >> program a library and thus have no clue about how that's gonna be >> >> used by users. >> > >> > Transducers make different trade offs than sequences and there will >> > always be cases where one or the other is a better choice. I really >> > appreciate this thread as highlighting some of the nuances. >> >> Yes, thanks a lot for your patience. I appreciate that very much. >> >> > Transducers break transformations into three parts - source iteration, >> > composed transforms, and output collection. In the case of reducible >> > inputs, multiple transforms, and full realization, transducers can be >> > much faster. If not all of those are in play, then the results are >> > more subtle. One thing I've found in perf testing a lot of stuff is >> > that chunked sequences continually surprise me at how fast they can >> > be. >> >> Then maybe I should experiment with chunged seqs. >> >> Bye, >> Tassilo >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> <javascript:> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
