You can always have the vertex value being a tuple3 and let the user set only a part of it (the value).
It would work the same way as with the vertex key - the user function can see it, but not change it. On Sat, Mar 21, 2015 at 8:42 PM, Andra Lungu <lungu.an...@gmail.com> wrote: > Hi Stephan, > > You got the idea! I wanted to make the inDeg and outDeg part of the value > itself(that was one of my first versions), but then I thought it would be > too annoying for a user who does not want to access the degrees in > vertexUpdate/ sendMessages to carry them around. > > Because in this scenario, when you do setNewVertexValue() you would need a > new Tuple3(inDeg, outDeg, value). Also, this approach can introduce a ton > of bugs; imagine that the user can him/herself change the degree values. > One workaround I see is to make setNewVertexValue update just the third > value and keep the degrees from the previous iteration. But I am not sure > that's possible. > > If you say that's a good approach I will keep the degrees in the value. > There seems to be no other way... > > Thanks! > Andra > > On Sat, Mar 21, 2015 at 8:26 PM, Stephan Ewen <se...@apache.org> wrote: > > > Hi Andra! > > > > I am still not 100% sure about the cause of the problem, but here is a > bit > > of background on how we model the computation as a delta iteration and > how > > you can extend the vertex state. Maybe that helps answer your question. > > > > When using delta iterations for vertex-centric iterations, we keep the > > vertices in the solution set as a two-tuple (vertex key, vertexvalue). > The > > key identifies the element in the solution set, so if we add a new > version > > of the vertex (same key, different value), the previous entry gets > > replaced. > > > > The result of the iteration is the latest version of the solution set > after > > the last superstep. > > > > Making the degree part of the vertex value should work (this makes the > > vertex a nested tuple (key (inDeg, outDeg, value)). Also changing the > > solution set types to be a 4-tuple (vertex key, inDegree, outDegree, > value) > > should work. > > > > Greetings, > > Stephan > > > > > > On Fri, Mar 20, 2015 at 12:00 PM, Andra Lungu <lungu.an...@gmail.com> > > wrote: > > > > > Hi Stephan, > > > > > > What I am trying to do, among other things, is to make the vertex's > > > inDegree and outDegree available in the vertexUpdate and sendMessages > > > functions via vertex.getInDegrees(). > > > That is almost done: > https://github.com/andralungu/flink/tree/spargelExt > > > > > > Only the problem is that the inDegrees are computed once, per graph > > before > > > starting a vertex centric iteration. So, what I decided to do was to > > store > > > the inDegree for each vertex within the Vertex object as an attribute. > So > > > far so good, > > > > > > DataSet<Vertex<K, VV>> newVertices = > > > verticesWithInDegrees.runOperation(iteration); > > > > > > The vertices with in degrees are the initial vertices with the degrees > > set > > > as values. > > > When runOperation() gets called, precisely when > > > > > > createResult() > > > > > > from VertexCentricIteration gets called, the inDegrees stored are lost. > > > This happens because basically in each superstep a new DataSet is > > created. > > > > > > I wanted to know how to keep the degrees there. In other words, > > > how/where/what are the steps for vertex value updates and how to > include > > > the degrees there? > > > > > > Thanks! > > > Andra > > > > > > > > > > > > On Fri, Mar 20, 2015 at 11:28 AM, Stephan Ewen <se...@apache.org> > wrote: > > > > > > > Hi Andra! > > > > > > > > I am not sure I am getting exactly what the question is. The code you > > > > pasted is from the Spargel API - specifically just forwarding > > registered > > > > broadcast variables. > > > > > > > > What do you mean with "the vertex values get reset" ? > > > > > > > > Stephan > > > > > > > > PS: The delta iterations are based in this paper: > > > > http://arxiv.org/pdf/1208.0088.pdf > > > > > > > > > > > > On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lungu.an...@gmail.com> > > > > wrote: > > > > > > > > > Hello, > > > > > > > > > > I've used delta iterations several times up until now, but I just > > > > realized > > > > > that I never fully understood what happens inside. And the > > > documentation > > > > > only explains things from a user's perspective. Which is why I > could > > > > really > > > > > use your help :). > > > > > > > > > > Here goes nothing: > > > > > In Gelly, Graph.java, there is a nice > runVertexCentricIteration(...) > > > > > method, which is in fact Spargel's - that disguises a delta > > iteration. > > > > What > > > > > I am trying to do is to set the vertex value before running > > > > > > > > > > DataSet<Vertex<K, VV>> newVertices = > > > > > verticesWithInDegrees.runOperation(iteration); > > > > > > > > > > The problem is that after this runOperation, the vertex values get > > > reset. > > > > > Now, when I looked in VertexCentricIteration.java's createResult(). > > > > > > > > > > It's a plain delta iteration that (more or less) looks like this: > > > > > > > > > > final DeltaIteration<Vertex<VertexKey, VertexValue>, > > Vertex<VertexKey, > > > > > VertexValue>> iteration = > > > > > this.initialVertices.iterateDelta(this.initialVertices, > > > > > this.maximumNumberOfIterations, zeroKeyPos); > > > > > > > > > > ................ > > > > > > > > > > // configure coGroup update function with name and broadcast > > variables > > > > > updates = updates.name("Vertex State Updates"); > > > > > for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) { > > > > > updates = updates.withBroadcastSet(e.f1, e.f0); > > > > > } > > > > > > > > > > // let the operator know that we preserve the key field > > > > > > updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0"); > > > > > > > > > > return iteration.closeWith(updates, updates); > > > > > > > > > > The > > > > > > > > > > DataSet<?> > > > > > > > > > > in the for is the problem. Vertex values get reset. Can someone > give > > > me a > > > > > hint on how to propagate the vertex value throughout the iteration? > > > > > > > > > > Thanks! > > > > > Andra > > > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------------------------------- > > > > > P.S. Could someone please subscribe me to the @user mailing list? > For > > > > some > > > > > reason, sending the mail to that recipient fails. Thanks! > > > > > > > > > > > > > > >
