On Friday, February 8, 2013 1:56:27 AM UTC-8, Edmund wrote:
>
> Hey Jason,
>
> Going slightly OT here to describe my motivations, but you did ask
> for more details ! :)
>
> I wrote a little blog post + lib last month using Stuart Sierra's Flow
> (much like
> Graph) to implement the steps in doing a very simple discrete inference
> task. Its
> demonstrating the same thing you talk about in your post about
> declarative computation - here the point is that you can efficiently
> update your
> inference based new data on account of having the
> graph structure to work over. Here's the post
> http://boss-level.com/?p=160 and a
> literate version of the code http://ejackson.github.com/inflow/
> This is my original motivation.
>
I think I understand what you're trying to do from the example, thanks!
> What I'm doing there is 'building-up' the computation over time,
> so I start with a graph with just the input data, and end up with a
> graph with the input and output. But the 'building up' is not the
> principal hangup I
> have: I'm happy to have an in- and out- map. The problem is, when
> dealing with a very nested graph, getting a fine grained syntax for
> describing where to put the outputs. Ie how do I tell Graph to put just
> the output x into
> {:a {:b 1 :c x}} of the graph, without replacing the entire map of the :a
> key ?
>
Like I said earlier, it's a requirement of Graph that node names must be
unique, and distinct from top-level input keys.
Thus, there cannot be a way to 'put x into the map under :a at key :c' ,
because there is no way to in-place modify the inputs or other node values
of the graph. The output is just a map of all the node results, nothing
more and nothing less.
That said, you can use a hierarchical graph to specify different node
functions for the different sub-keys:
;; As a hierarchical graph
user> ((eager-compile {:a2 {:b (fnk [[:a1 b]] b)
:c (fnk [[:a1 b]] (inc b))}})
{:a1 {:b 1}})
{:a2 {:b 1, :c 2}}
So for an example like the one you gave in the above link, you can
represent your nested stats directly as a nested graph without resorting to
key flattening. And if you want to compute just a subset of the graph for
updated downstream values and then merge this into your previous map, you
can do this in two steps:
1. Pick out the subset of leaf functions that may have changed, using the
same strategy you currently use with Flow, and construct a Graph with only
those sub-keys. For example:
user> ((eager-compile {:a2 {:b (fnk [[:a1 b]] b)}})
{:a1 {:b 2}})
{:a2 {:b 2}}
, then
2. deep-merge these results with the original result map.
i..e, (deep-merge-with (fn [x y] y) original-results updated-results)
where deep-merge-with
is
http://clojuredocs.org/clojure_contrib/clojure.contrib.map-utils/deep-merge-with
This logic could be wrapped up into a single higher-order function that
takes a graph and knows how to incrementally update the results given
incremental changes in input. Does that make sense / seem like it will
satisfy your need?
Cheers,
Jason
> That's obviously no good if I need to have lots of things changing
> beneath :a in a single graph flow due to uniqueness.[
>
> With Flow what I did is flatten the nested graph such that the keys are
> vectors of the path to each node, run the flow on that graph, and then
> re-nest the the graph afterwards (the flow-> and ->flow functions).
> So the map {:a {:b 1 :c x}} becomes {[:a :b] 1, [:a :c] x}, and I can
> refer to
> nested nodes this way.
>
> In Graph such a scheme would be possible too, but that Graph enforces
> keywords as the keys for the graph. Having vectors for the keys still
> allows you to
> have uniqueness, topological ordering etc. So the example I gave in my
> last email:
>
> {:a {:b 1}} -> {:a {:b 1, :c 2}}
>
> could, hypothetically, be written for just the outputs as
>
> ((graph/eager-compile {[:a :c] (fnk [[:a b]] (inc b))})
> {:a {:b 1}})
>
> >> {:a {:c 2}}
>
> So my question is: how do I refer to nested nodes in the output
> selector using Graph? Conditionally, on that being impossible, would
> there be any merit in the vector scheme that I have suggested ?
>
> Sorry for the long post, I hope I haven't obscured my question. I also
> get the feeling that I must be missing something very obvious in your
> code for how to do this !
>
> Edmund
>
>
> On Thursday, 7 February 2013 19:27:13 UTC, Jason Wolfe wrote:
>>
>> On Thu, Feb 7, 2013 at 10:54 AM, AtKaaZ <[email protected]> wrote:
>> > Hello.
>> >
>> https://github.com/Prismatic/plumbing/blob/master/test/plumbing/graph_examples_test.clj#L148
>>
>> > Why do they return in a map instead of maybe a set ? do we ever get
>> {:key
>> > false} ?
>> > Thanks.
>>
>> The leaf value for output schemata is always 'true'. It's a bit odd,
>> but to support specifications of functions/graphs that return nested
>> maps, the outer layers need to be maps. It's true that the innermost
>> layers could be represented as sets, but we chose to keep things
>> uniform instead.
>>
>> >
>> >
>> > On Thu, Feb 7, 2013 at 7:22 PM, Jason Wolfe <[email protected]> wrote:
>> >>
>> >> We've just posted a blog post with more high-level context on what
>> we're
>> >> trying to accomplish with Graph (plus more examples!)
>> >>
>> >>
>> >>
>> http://blog.getprismatic.com/blog/2013/2/1/graph-abstractions-for-structured-computation
>>
>> >>
>> >> We're also answering questions and reading comments in the Hacker News
>> >> thread, if that's your thing:
>> >>
>> >> http://news.ycombinator.com/item?id=5183236
>> >>
>> >> Cheers, Jason
>> >>
>> >>
>> >> On Tuesday, January 29, 2013 10:46:54 AM UTC-8, Aria Haghighi wrote:
>> >>>
>> >>> Hey all,
>> >>>
>> >>> Prismatic has open-sourced our Plumbing and Graph library on github.
>> >>> Jason Wolfe gave a talk about how we use graph for systems
>> composition at
>> >>> Strange loop last year. Please give the library
>> >>> a whirl and let us know if you're using it and if you find any issues
>> or
>> >>> feature requests. We use this library very heavily throughout our
>> code and
>> >>> hope others find it useful as well.
>> >>>
>> >>> Best, Aria
>> >>
>> >> --
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups "Clojure" group.
>> >> To post to this group, send email to [email protected]
>> >> Note that posts from new members are moderated - please be patient
>> with
>> >> your first post.
>> >> To unsubscribe from this group, send email to
>> >> [email protected]
>> >> For more options, visit this group at
>> >> http://groups.google.com/group/clojure?hl=en
>> >> ---
>> >> You received this message because you are subscribed to the Google
>> Groups
>> >> "Clojure" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> an
>> >> email to [email protected].
>> >> For more options, visit https://groups.google.com/groups/opt_out.
>> >>
>> >>
>> >
>> >
>> >
>> >
>> > --
>> > Please correct me if I'm wrong or incomplete,
>> > even if you think I'll subconsciously hate it.
>> >
>> > --
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Clojure" group.
>> > To post to this group, send email to [email protected]
>> > Note that posts from new members are moderated - please be patient with
>> your
>> > first post.
>> > To unsubscribe from this group, send email to
>> > [email protected]
>> > For more options, visit this group at
>> > http://groups.google.com/group/clojure?hl=en
>> > ---
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Clojure" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to [email protected].
>> > For more options, visit https://groups.google.com/groups/opt_out.
>> >
>> >
>>
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
