To continue OT, There is pretty interesting work being done on incremental datalog computation, that some people might be interested in being aware of.
http://research.microsoft.com/en-us/projects/naiad/ http://channel9.msdn.com/posts/Frank-McSherry-Introduction-to-Naiad-and-Differential-Dataflow "Naiad is an investigation of data-parallel dataflow computation in the spirit of Dryad and DryadLINQ, but with a focus on incremental computation. Naiad introduces a new computational model, differential dataflow, operating over collections of differences rather than collections of records, and resulting in very efficient implementations of programming patterns that are expensive in existing systems." On Fri, Feb 8, 2013 at 1:56 AM, Edmund <edmundsjack...@gmail.com> wrote: > Hey Jason, > > > Going slightly OT here to describe my motivations, but you did ask > for more details ! :) > > I wrote a little blog post + lib last month using Stuart Sierra's Flow (much > like > Graph) to implement the steps in doing a very simple discrete inference > task. Its > demonstrating the same thing you talk about in your post about > declarative computation - here the point is that you can efficiently update > your > inference based new data on account of having the > graph structure to work over. Here's the post http://boss-level.com/?p=160 > and a > literate version of the code http://ejackson.github.com/inflow/ > This is my original motivation. > > What I'm doing there is 'building-up' the computation over time, > so I start with a graph with just the input data, and end up with a > graph with the input and output. But the 'building up' is not the principal > hangup I > have: I'm happy to have an in- and out- map. The problem is, when > dealing with a very nested graph, getting a fine grained syntax for > describing where to put the outputs. Ie how do I tell Graph to put just > the output x into > {:a {:b 1 :c x}} of the graph, without replacing the entire map of the :a > key ? > That's obviously no good if I need to have lots of things changing > beneath :a in a single graph flow due to uniqueness.[ > > With Flow what I did is flatten the nested graph such that the keys are > vectors of the path to each node, run the flow on that graph, and then > re-nest the the graph afterwards (the flow-> and ->flow functions). > So the map {:a {:b 1 :c x}} becomes {[:a :b] 1, [:a :c] x}, and I can refer > to > nested nodes this way. > > In Graph such a scheme would be possible too, but that Graph enforces > keywords as the keys for the graph. Having vectors for the keys still > allows you to > have uniqueness, topological ordering etc. So the example I gave in my > last email: > > > {:a {:b 1}} -> {:a {:b 1, :c 2}} > > could, hypothetically, be written for just the outputs as > > ((graph/eager-compile {[:a :c] (fnk [[:a b]] (inc b))}) > {:a {:b 1}}) > >>> {:a {:c 2}} > > So my question is: how do I refer to nested nodes in the output > selector using Graph? Conditionally, on that being impossible, would > there be any merit in the vector scheme that I have suggested ? > > Sorry for the long post, I hope I haven't obscured my question. I also > get the feeling that I must be missing something very obvious in your > code for how to do this ! > > Edmund > > > On Thursday, 7 February 2013 19:27:13 UTC, Jason Wolfe wrote: >> >> On Thu, Feb 7, 2013 at 10:54 AM, AtKaaZ <atk...@gmail.com> wrote: >> > Hello. >> > >> > https://github.com/Prismatic/plumbing/blob/master/test/plumbing/graph_examples_test.clj#L148 >> > Why do they return in a map instead of maybe a set ? do we ever get >> > {:key >> > false} ? >> > Thanks. >> >> The leaf value for output schemata is always 'true'. It's a bit odd, >> but to support specifications of functions/graphs that return nested >> maps, the outer layers need to be maps. It's true that the innermost >> layers could be represented as sets, but we chose to keep things >> uniform instead. >> >> > >> > >> > On Thu, Feb 7, 2013 at 7:22 PM, Jason Wolfe <ja...@w01fe.com> wrote: >> >> >> >> We've just posted a blog post with more high-level context on what >> >> we're >> >> trying to accomplish with Graph (plus more examples!) >> >> >> >> >> >> >> >> http://blog.getprismatic.com/blog/2013/2/1/graph-abstractions-for-structured-computation >> >> >> >> We're also answering questions and reading comments in the Hacker News >> >> thread, if that's your thing: >> >> >> >> http://news.ycombinator.com/item?id=5183236 >> >> >> >> Cheers, Jason >> >> >> >> >> >> On Tuesday, January 29, 2013 10:46:54 AM UTC-8, Aria Haghighi wrote: >> >>> >> >>> Hey all, >> >>> >> >>> Prismatic has open-sourced our Plumbing and Graph library on github. >> >>> Jason Wolfe gave a talk about how we use graph for systems composition >> >>> at >> >>> Strange loop last year. Please give the library >> >>> a whirl and let us know if you're using it and if you find any issues >> >>> or >> >>> feature requests. We use this library very heavily throughout our code >> >>> and >> >>> hope others find it useful as well. >> >>> >> >>> Best, Aria >> >> >> >> -- >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups "Clojure" group. >> >> To post to this group, send email to clo...@googlegroups.com >> >> Note that posts from new members are moderated - please be patient with >> >> your first post. >> >> To unsubscribe from this group, send email to >> >> clojure+u...@googlegroups.com >> >> For more options, visit this group at >> >> http://groups.google.com/group/clojure?hl=en >> >> --- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Clojure" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> >> an >> >> email to clojure+u...@googlegroups.com. >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> > >> > >> > >> > >> > -- >> > Please correct me if I'm wrong or incomplete, >> > even if you think I'll subconsciously hate it. >> > >> > -- >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Clojure" group. >> > To post to this group, send email to clo...@googlegroups.com >> > Note that posts from new members are moderated - please be patient with >> > your >> > first post. >> > To unsubscribe from this group, send email to >> > clojure+u...@googlegroups.com >> > For more options, visit this group at >> > http://groups.google.com/group/clojure?hl=en >> > --- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Clojure" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to clojure+u...@googlegroups.com. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > > > -- > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to the Google Groups > "Clojure" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
