Just a quick thought I had as I was walking home. Given an infinite number of cores, the time to process a set of dataflow functions is equivalent to the the time that the longest function took to do its processing. The efficiency is the (sum of time that all the dataflow functions took) / ( (count of the dataflow functions) * (the time of the longest running dataflow function) ). Given this, optimization is really simple. Take the longest running dataflow function, and see if you can somehow split it into smaller functions. Nothing profound here, but I thought it was interesting how "evident" optimizations might be when you use a dataflow processing model.
On Tuesday, December 24, 2013 3:50:43 PM UTC-8, Stephen Cagle wrote: > > One thing that I am seeing on a re-read is that I conflated the notion of > the data flow function and the paths. I was sort of thinking that the data > flow functions "sit" at a particular path location. Similar to how a value > "sits" in a location in memory. It is more appropriate to say that the data > flow function is associated (referred isn't quite right) with a particular > path location. > > Taking this a bit further, a data flow function DOES NOT actually know who > its inputs are. It only knows that it takes input values in/of a certain > form. The schema that specifies how one set of paths map to another path > are possibly separate from the schema that specifies which dataflow > function is associated with which path. > > On Tuesday, December 24, 2013 1:36:16 PM UTC-8, Stephen Cagle wrote: >> >> Thank you. I only read the last two articles so far; some notes. >> >> >> http://my.opera.com/Vorlath/blog/2008/01/06/simple-example-of-the-difference-between-imperative-functional-and-data-flow >> >> I realized that I really wasn't getting what dataflow was about. I was >> viewing dataflow paths as a sort of hook that I could hang values on. I had >> never externalized the fact that each path only refers to the things it >> inputs upon. Specifically, I was modeling some of the patterns like path A >> passes a value to B, B does some computation, and puts the result on path >> A. This isn't neccessarily wrong, but it appears that I was using dataflow >> paths in a way similar to function evaluation. >> >> He makes a big deal out of not needing to know where his inputs come >> from, as well as not needing to invoke a function to create his inputs, but >> it still seems he must have a reference to his inputs. >> >> The effectively automatic parallellization of the code is pretty neat. >> Not so much because it is parallelized (which can be done in other >> systems), but mostly because it required no forethought or synchronization. >> It is automatically parallelized and pipelined. Neato. Of course, we aren't >> going to get much of that in js without some work. >> >> The second to last paragraph was another head turner. I had previously >> viewed every dataflow node/path as a loop that just waits for a change in >> its inputs and computes a new value when one of them changes. However, in >> his system recursive calls are also parallelized and pipelined. I am not >> quite sure how this would be implemented, but it seems neat. Again though, >> probably not relevant to js. >> >> http://my.opera.com/Vorlath/blog/2008/07/25/hierarchies-and-equivalence >> >> This one is more of a philosophy/notice of intent piece. This one should >> probably be read after the previous article. If I hadn't read the previous >> article, I would understand the words he is saying, but not their >> implications. There were many parts in this article that I disagreed with, >> but it seems wasteful to argue over individual sentences. >> >> Honestly, too many ideas came at me too fast. I would have to re-read at >> a later point to have much of an opinion. Having said that, the content may >> be good (or not), but I am not sure this article would change any opinions >> unless the person was already predisposed. >> >> On Tuesday, December 24, 2013 9:39:58 AM UTC-8, Dan Kersten wrote: >>> >>> Here's some resources to get you started learning about dataflow as a >>> paradigm. From this you should be able to figure out how Pedestal's >>> dataflow system fits in. >>> >>> A list of existing dataflow languages and systems: >>> http://stackoverflow.com/questions/461796/dataflow-programming-languages/949771#949771 >>> I >>> would suggest looking into a few of these to learn the concepts. I'd also >>> suggest trying out one or more of the visual dataflow languages to get a >>> feel for how problems can be solved in this paradigm and how non-pure >>> functions fit in. I've personally used both Max/MSP and Synthmaker and >>> found the concepts really clicked after using them for a while. >>> >>> A work in progress book on which you may also want to keep an eye: >>> http://dataflowbook.com/cms/ >>> >>> Also, if you can ignore the tone, this article has helped me better >>> understand how code fits together in a dataflow system (a flat hierarchy): >>> http://my.opera.com/Vorlath/blog/2008/07/25/hierarchies-and-equivalence and >>> this one talks about how dataflow differs from imperative programming: >>> http://my.opera.com/Vorlath/blog/2008/01/06/simple-example-of-the-difference-between-imperative-functional-and-data-flow >>> But >>> be warned that the author of those is very opinionated and feels that he is >>> writing about the "one true way" ;-) >>> >>> >>> >>> On 23 December 2013 19:32, Stephen Cagle <sam...@gmail.com> wrote: >>> >>>> < Cross posted from pedestal-users group, as many people who are not >>>> using pedestal may still know about dataflow, terms like effect are used >>>> in >>>> the context of pedestal > >>>> >>>> Pedestal seems strongly based on dataflow based programming. The >>>> gigantic tutorial just sort of jumps in on it. >>>> >>>> Based on my usage thus far, dataflow seems really good at modeling >>>> problems that are self contained. In their pure forms (no external inputs >>>> or internal effects), dataflow seems almost like a circuit diagram. >>>> >>>> However, I quickly get confused once we start dealing with effects. >>>> Seems that there is a large "Here be Dragons" area of code in that region >>>> (services.cljs). >>>> >>>> I feel that this may not be dataflow's fault; I just haven't got my >>>> head around it. When I look at dataflow, I feel like it is constraining me >>>> to a particular way of solving a problem. The upside of this is that I >>>> have >>>> made my logic declarative and potentially easier to reason about. This is >>>> a >>>> trade-offs that seems similar to the trade-offs one makes when moving from >>>> a mutable procedural programming model to a immutable functional model. I >>>> have yet to personally get substantial benefit from dataflow, but that >>>> does >>>> not mean I will not with more mastery. >>>> >>>> I am wondering if there are any any getting started guides for dataflow >>>> programming that you (the community) would recommend. I would be >>>> especially >>>> interested in "recipe books" for dataflow based programming. How do you >>>> really do asynchronous processing with dataflow? What if your asyncs may >>>> return in a random order but must be processed in a specified order? A few >>>> books/articles/whatever on how experts think through these problems could >>>> be quite beneficial. >>>> >>>> -- >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+u...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>> >>> -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
