Hi Stephan, You got the idea! I wanted to make the inDeg and outDeg part of the value itself(that was one of my first versions), but then I thought it would be too annoying for a user who does not want to access the degrees in vertexUpdate/ sendMessages to carry them around.
Because in this scenario, when you do setNewVertexValue() you would need a new Tuple3(inDeg, outDeg, value). Also, this approach can introduce a ton of bugs; imagine that the user can him/herself change the degree values. One workaround I see is to make setNewVertexValue update just the third value and keep the degrees from the previous iteration. But I am not sure that's possible. If you say that's a good approach I will keep the degrees in the value. There seems to be no other way... Thanks! Andra On Sat, Mar 21, 2015 at 8:26 PM, Stephan Ewen <se...@apache.org> wrote: > Hi Andra! > > I am still not 100% sure about the cause of the problem, but here is a bit > of background on how we model the computation as a delta iteration and how > you can extend the vertex state. Maybe that helps answer your question. > > When using delta iterations for vertex-centric iterations, we keep the > vertices in the solution set as a two-tuple (vertex key, vertexvalue). The > key identifies the element in the solution set, so if we add a new version > of the vertex (same key, different value), the previous entry gets > replaced. > > The result of the iteration is the latest version of the solution set after > the last superstep. > > Making the degree part of the vertex value should work (this makes the > vertex a nested tuple (key (inDeg, outDeg, value)). Also changing the > solution set types to be a 4-tuple (vertex key, inDegree, outDegree, value) > should work. > > Greetings, > Stephan > > > On Fri, Mar 20, 2015 at 12:00 PM, Andra Lungu <lungu.an...@gmail.com> > wrote: > > > Hi Stephan, > > > > What I am trying to do, among other things, is to make the vertex's > > inDegree and outDegree available in the vertexUpdate and sendMessages > > functions via vertex.getInDegrees(). > > That is almost done: https://github.com/andralungu/flink/tree/spargelExt > > > > Only the problem is that the inDegrees are computed once, per graph > before > > starting a vertex centric iteration. So, what I decided to do was to > store > > the inDegree for each vertex within the Vertex object as an attribute. So > > far so good, > > > > DataSet<Vertex<K, VV>> newVertices = > > verticesWithInDegrees.runOperation(iteration); > > > > The vertices with in degrees are the initial vertices with the degrees > set > > as values. > > When runOperation() gets called, precisely when > > > > createResult() > > > > from VertexCentricIteration gets called, the inDegrees stored are lost. > > This happens because basically in each superstep a new DataSet is > created. > > > > I wanted to know how to keep the degrees there. In other words, > > how/where/what are the steps for vertex value updates and how to include > > the degrees there? > > > > Thanks! > > Andra > > > > > > > > On Fri, Mar 20, 2015 at 11:28 AM, Stephan Ewen <se...@apache.org> wrote: > > > > > Hi Andra! > > > > > > I am not sure I am getting exactly what the question is. The code you > > > pasted is from the Spargel API - specifically just forwarding > registered > > > broadcast variables. > > > > > > What do you mean with "the vertex values get reset" ? > > > > > > Stephan > > > > > > PS: The delta iterations are based in this paper: > > > http://arxiv.org/pdf/1208.0088.pdf > > > > > > > > > On Thu, Mar 19, 2015 at 9:25 AM, Andra Lungu <lungu.an...@gmail.com> > > > wrote: > > > > > > > Hello, > > > > > > > > I've used delta iterations several times up until now, but I just > > > realized > > > > that I never fully understood what happens inside. And the > > documentation > > > > only explains things from a user's perspective. Which is why I could > > > really > > > > use your help :). > > > > > > > > Here goes nothing: > > > > In Gelly, Graph.java, there is a nice runVertexCentricIteration(...) > > > > method, which is in fact Spargel's - that disguises a delta > iteration. > > > What > > > > I am trying to do is to set the vertex value before running > > > > > > > > DataSet<Vertex<K, VV>> newVertices = > > > > verticesWithInDegrees.runOperation(iteration); > > > > > > > > The problem is that after this runOperation, the vertex values get > > reset. > > > > Now, when I looked in VertexCentricIteration.java's createResult(). > > > > > > > > It's a plain delta iteration that (more or less) looks like this: > > > > > > > > final DeltaIteration<Vertex<VertexKey, VertexValue>, > Vertex<VertexKey, > > > > VertexValue>> iteration = > > > > this.initialVertices.iterateDelta(this.initialVertices, > > > > this.maximumNumberOfIterations, zeroKeyPos); > > > > > > > > ................ > > > > > > > > // configure coGroup update function with name and broadcast > variables > > > > updates = updates.name("Vertex State Updates"); > > > > for (Tuple2<String, DataSet<?>> e : this.bcVarsUpdate) { > > > > updates = updates.withBroadcastSet(e.f1, e.f0); > > > > } > > > > > > > > // let the operator know that we preserve the key field > > > > updates.withForwardedFieldsFirst("0").withForwardedFieldsSecond("0"); > > > > > > > > return iteration.closeWith(updates, updates); > > > > > > > > The > > > > > > > > DataSet<?> > > > > > > > > in the for is the problem. Vertex values get reset. Can someone give > > me a > > > > hint on how to propagate the vertex value throughout the iteration? > > > > > > > > Thanks! > > > > Andra > > > > > > > > > > > > > > ---------------------------------------------------------------------------------------- > > > > P.S. Could someone please subscribe me to the @user mailing list? For > > > some > > > > reason, sending the mail to that recipient fails. Thanks! > > > > > > > > > >
