On Dec 13, 2008, at 2:18 AM, Mark Engelberg wrote:
> > On Fri, Dec 12, 2008 at 9:28 PM, Rich Hickey <richhic...@gmail.com> > wrote: >> I think it's very important not to conflate different notions of >> sequences. Clojure's model a very specific abstraction, the Lisp >> list, >> originally implemented as a singly-linked list of cons cells. It is a >> persistent abstraction, first/second/third/rest etc, > > OK, I think I see where you're going with this. It sounds like you're > saying that one of the key ideas here is that the first/rest interface > is meant to guarantee a certain kind of persistence. If I say (first > (rest coll)), it should aways give me the same thing. If you designed > first/rest to work on uncached sequences, most would work this way, > but there is certainly no guarantee, depending on the nature of the > generating function. Since you want the interface to imply this sort > of guarantee, you have no choice but to cache anything involving > first/rest (not including things like "seq"ified vectors which are > inherently cached). This makes sense. > There are other attributes as well, like the concurrency properties of lazy-cons, which guarantees once-only evaluation of first and rest. > So the piece you're working on is to provide better support for things > that are determined by their generating functions. You are > intentionally avoiding using the terms first/rest for these "streams" > for the reasons above. Since streams will be easily convertable to > seqs, we'll be able to get the best of both words. We can manipulate > streams, and eventually when we pass the stream to a function that > requires seqs, it will do the conversion, and stability will be > guaranteed. > > It sounds like a very promising approach, and I'm looking forward to > seeing the results. It seems like ideally, it should be as easy as > possible to manipulate the streams in stream form, so if you're > working with a stream source you can go as long as possible without > converting a stream to seq. Having map/filter/mapcat/comprehensions > work fluidly over streams as well as seqs could be hugely beneficial. > Yes, those things will be available for streams. >> That said, I think there is certainly room for a stream/generator >> model, especially for I/O, but also for more efficient collection >> processing. Such a thing is explicitly one-pass and ephemeral. It >> will >> not have the interface of first/rest, nor Java's thread-unsafe >> hasNext/next iterator model (shared by Scala). You can obviously >> build >> seqs from streams/generators, and in my model, with a single >> definition you will get both a stream and seq version of functions >> like map and filter, as I showed here: >> http://groups.google.com/group/clojure/msg/53227004728d6c54 > > OK, looking back over your sneak preview example, I have a couple > quick comments/questions. > > 1. You are using stream-seq to convert streams to seqs. Can't you > just make the seq function work automatically on streams to produce > the sequence, just like it does on vectors, sets, etc., rather than > have a special name for converting streams to sequences? That way, > you can pass streams to all the functions that begin with (seq coll) > and you'll get the desired behavior. No you can't, for the same reasons you can't for Iterator or Enumeration seqs. Again it comes down to abstractions, and the abstraction for (seq x) is one on persistent collections. It presumes that (seq x) is referentially transparent, which it isn't for ephemeral sources - i.e. streams aren't colls. It would become quite fragile if people had to adopt call-once-only semantics for seq - ephemerality would pollute the world. That said, there are a growing number of these ephemeral-source-seq functions which could be folded into one multimethod. Also note that converting a stream to a seq may involve I/O (as may any consumption of a stream, and stream->seq will call next!), and thus will have different concurrency semantics than (seq coll). > > 2. After you've added streams, what will the range function produce? > Consider making range produce a stream rather than a sequence. > (Somewhat relatedly, in Python, range produced a concrete list and > xrange produced a generated stream. range was so rarely used that one > of the breaking changes they made in Python 3.0 was to get rid of the > list-producing range, and start using the name "range" for the > generator-variant rather than "xrange"). If you look in the svn you'll see that Range already implements Streamable, and so can be used in either context. > > 3. In your sneak-preview filter function, it always produces a seq. > In the spirit of making it easy to work with streams as long as > possible without conversion, how about making filter return a stream > in the case that the input is a stream, rather than having separate > filter-stream and filter public functions? Or is it not in the spirit > of Clojure to have functions overloaded in this way? I'm still on the fence about overloading by type here. I'm not fond of sticking "-stream" on everything either, though any prefix/suffix will be short. What you get back from (map f aseq) and (map f astream) would be two very different things, and without declared types you won't know from looking at the code what you are dealing with. That's tricky. Also, as discussed earlier, when transitioning from streams to the easier-to-use seqs there will need to be a once-only explicit conversion. There are also issues about being able to partition by stream(able)/seq(able) - will nothing be both? Anyway, these are just details, the design is pretty well along (e.g. the stream system will be queue and timeout savvy) and I don't see any impediments other than needing to cut a release soon :) Rich --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
