On Dec 12, 12:29 am, "Mark Engelberg" <mark.engelb...@gmail.com>
wrote:
> On Mon, Dec 8, 2008 at 6:51 PM, Rich Hickey <richhic...@gmail.com> wrote:
>
> I don't have the latest build of Clojure with atoms, so I
> reimplemented Rich's filter solution using refs, turning:
>
> > (defn filter
> > [pred coll]
> > (let [sa (atom (seq coll))
> > step (fn step []
> > (when-let [s @sa]
> > (let [x (first s)]
> > (if (pred x)
> > (lazy-cons x (do (swap! sa rest) (step)))
> > (do (swap! sa rest)
> > (recur))))))]
> > (step)))
>
> into:
>
> (defn safe-filter
> [pred coll]
> (let [sa (ref (seq coll)),
> step (fn step []
> (when-let [s @sa]
> (let [x (first s)]
> (if (pred x)
> (lazy-cons x (do
> (dosync (ref-set sa (rest s)))
> (step)))
> (do
> (dosync (ref-set sa (rest s)))
> (recur))))))]
> (step)))
>
> But splode still splodes!:
>
> (defn splode [n]
> (doseq [i (safe-filter #(= % 20) (map inc (range n)))]))
>
> Any idea why the ref version wouldn't be working?
>
Yes, because that is not the same - your version still closes over s.
swap! passes a fn to transform the current value, ref-set does not.
But alter and commute do:
(defn safe-filter
[pred coll]
(let [sa (ref (seq coll)),
step (fn step []
(when-let [s @sa]
(let [x (first s)]
(if (pred x)
(lazy-cons x (do
(dosync (commute sa rest))
(step)))
(do
(dosync (commute sa rest))
(recur))))))]
(step)))
> > But it's not pretty. Fortunately, this segues with work I have been
> > doing on I/O and generators/streams.
>
> I'm really looking forward to seeing how this all turns out. Cached
> lazy sequences seems to be a bad default for all the standard sequence
> operations. You have to be very careful to not retain the head of one
> of these sequences (can't give names to intermediate results, for
> example), and it's very hard to predict when the head of one of these
> sequences might be unintentionally held. This seems to make code more
> brittle. Probably the best solution is to default to sequences that
> always freshly generate the results, and you have to intentionally
> cache the sequence if that is what you want.
>
I'm appreciate the time you and others have spent on this, and will
improve filter, but I'm not sure where you are getting your
presumptions about lazy sequences. They are not a magic bullet that
makes working with data bigger than memory transparent. They do make
it possible in some cases, but that is not their intent. Latent
runtime memory consumption is a fact of life of laziness, and is
present in Haskell etc. I can make it go away for the specific case of
filter and the rest of the library, but not generally.
Not caching would let to excessive recalculation in many contexts, and
then people would be complaining about performance. There are many
benefits of the thread-safe once-and-only-once evaluation promise of
lazy-cons that you may not be considering. It is certainly not a bad
default.
There will soon be a streams/generator library, intended for i/o, that
will do one-pass non-cached iteration. It will offer a different set
of tradeoffs and benefits, including reduced ease-of use - not being
persistent removes a lot of things like first/rest/nth, destructuring
etc.
But the important thing is tradeoffs/benefits. They always come
together.
Rich
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
