Re: [gelly] Spargel model rework

2015-11-03 Thread Andra Lungu
I also think a Giraph-like model could be added, but we shouldn't remove Spargel in favour of it! On Tue, Nov 3, 2015 at 2:35 AM, Stephan Ewen wrote: > When creating the original version of Spargel I was pretty much thinking in > GSA terms, more than in Pregel terms. There are some fundamental >

Re: Caching information from a stream

2015-10-29 Thread Andra Lungu
gt; > > } > > > > The output is the following on my machine (discarded the output of the > > print): > > > > Key: 2 Previous state was: -1 Update state to: 5 > > Key: 1 Previous state was: -1 Update state to: 3 > > Key: 6 Previous state was: -1 Update state to: 7 &

Caching information from a stream

2015-10-28 Thread Andra Lungu
Hey guys! I've been thinking about this one today: Say you have a stream of data in the form of (id, value) - This will evidently be a DataStream of Tuple2. I need to cache this data in some sort of static stream (perhaps even a DataSet). Then, if in the input stream, I see an id that was previou

[jira] [Created] (FLINK-2926) Add a Strongly Connected Components Library Method

2015-10-27 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2926: -- Summary: Add a Strongly Connected Components Library Method Key: FLINK-2926 URL: https://issues.apache.org/jira/browse/FLINK-2926 Project: Flink Issue Type

[jira] [Created] (FLINK-2899) The groupReduceOn* methods which take types as a parameter fail with TypeErasure

2015-10-22 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2899: -- Summary: The groupReduceOn* methods which take types as a parameter fail with TypeErasure Key: FLINK-2899 URL: https://issues.apache.org/jira/browse/FLINK-2899 Project

Re: Design document for FLINK-2254

2015-10-22 Thread Andra Lungu
Hi Saumitra, As you already noticed, the first version (with duplicates) is highly inefficient and consumes a lot of memory. So, I suggest we drop it for now. The version with the label makes a lot of modifications on the base Graph class, and this, in my opinion would make it more difficult to gr

Using Flink Streaming to write to multiple output files in HDFS

2015-10-21 Thread Andra Lungu
Hey guys, Long time, no see :). I recently started a new job and it involves performing a set of real-time data analytics using Apache Kafka, Storm and Flume. What happens, on a very high level, is that set of signals is collected, stored into a Kafka topic and then Storm is used to filter certai

[jira] [Created] (FLINK-2715) Benchmark Triangle Count methods

2015-09-20 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2715: -- Summary: Benchmark Triangle Count methods Key: FLINK-2715 URL: https://issues.apache.org/jira/browse/FLINK-2715 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-2714) Port the Flink DataSet Triangle Count example to the Gelly library

2015-09-20 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2714: -- Summary: Port the Flink DataSet Triangle Count example to the Gelly library Key: FLINK-2714 URL: https://issues.apache.org/jira/browse/FLINK-2714 Project: Flink

[jira] [Created] (FLINK-2661) Add a Node Splitting Technique to Overcome the Limitations of Skewed Graphs

2015-09-12 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2661: -- Summary: Add a Node Splitting Technique to Overcome the Limitations of Skewed Graphs Key: FLINK-2661 URL: https://issues.apache.org/jira/browse/FLINK-2661 Project: Flink

[jira] [Created] (FLINK-2634) Add a Vertex-centric Version of the Tringle Count Library Method

2015-09-08 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2634: -- Summary: Add a Vertex-centric Version of the Tringle Count Library Method Key: FLINK-2634 URL: https://issues.apache.org/jira/browse/FLINK-2634 Project: Flink

[jira] [Created] (FLINK-2570) Add a Triangle Count Library Method

2015-08-25 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2570: -- Summary: Add a Triangle Count Library Method Key: FLINK-2570 URL: https://issues.apache.org/jira/browse/FLINK-2570 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-2563) Gelly's Graph Algorithm Interface is limites

2015-08-23 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2563: -- Summary: Gelly's Graph Algorithm Interface is limites Key: FLINK-2563 URL: https://issues.apache.org/jira/browse/FLINK-2563 Project: Flink Issue

Re: Gelly Blog Post

2015-08-22 Thread Andra Lungu
On Tue, May 26, 2015 at 2:36 PM, Stephan Ewen wrote: > > > Wow, this is impressive :-) > > > > > > Amazing work, Gelly folks! > > > > > > On Tue, May 26, 2015 at 10:03 AM, Andra Lungu > > wrote: > > > > > >> Hey everyone, > > >&g

Re: Join hints for the Gelly functions

2015-08-22 Thread Andra Lungu
ly slower than letting the > system decide. > > Best, > Martin > > > On 22.08.2015 09:51, Andra Lungu wrote: > >> Hey everyone, >> >> When coding for my thesis, I observed that half of the current Gelly >> functions (the ones that use join operators) fai

Join hints for the Gelly functions

2015-08-22 Thread Andra Lungu
Hey everyone, When coding for my thesis, I observed that half of the current Gelly functions (the ones that use join operators) fail on a cluster environment with the following exception: java.lang.IllegalArgumentException: Too few memory segments provided. Hash Join needs at least 33 memory segm

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Andra Lungu
Congrats Chesnay! :) On Thu, Aug 20, 2015 at 8:51 PM, Márton Balassi wrote: > Welcome Chesnay! > > On Thu, Aug 20, 2015 at 7:29 PM, Henry Saputra > wrote: > > > Welcome Chesnay! > > > > On Thu, Aug 20, 2015 at 2:18 AM, Robert Metzger > > wrote: > > > The Project Management Committee (PMC) for

[jira] [Created] (FLINK-2546) The webclient does not allow users to save the execution plan

2015-08-19 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2546: -- Summary: The webclient does not allow users to save the execution plan Key: FLINK-2546 URL: https://issues.apache.org/jira/browse/FLINK-2546 Project: Flink

Re: Nested iterations not supported?

2015-08-18 Thread Andra Lungu
Hello Pieter, Nested iterations are indeed not supported in Flink. http://mail-archives.apache.org/mod_mbox/flink-user/201504.mbox/%3Cop.xw24u7fhf7e33m@vaio-sb%3E The problem is not in your code. On Tue, Aug 18, 2015 at 11:27 AM, Pieter-Jan Van Aeken < pieterjan.vanae...@euranova.eu> wrote: >

Re: [Proposal] Addition to Gelly

2015-08-12 Thread Andra Lungu
I would love to get some feedback from the guys at data Artisans about this one. So far, the comments originated and spread in the Stockholm area :) On Tue, Aug 11, 2015 at 6:33 PM, Andra Lungu wrote: > Hi Samia, > > A good method to statistically determine skewed vertices was b

Re: [Proposal] Addition to Gelly

2015-08-11 Thread Andra Lungu
hold to separate the high degree vertices > from the low degree vertices. > > Regards, > Samia > > On Tue, Aug 11, 2015 at 3:41 PM, Andra Lungu > wrote: > > > Hi Paris, > > > > Nice to virtually meet you too :) > > > > Maybe it makes s

Re: [Proposal] Addition to Gelly

2015-08-11 Thread Andra Lungu
ers should be aware when they will get benefits by using it since node > splitting doesn’t come with no cost I guess. > I am really eager to see how this will evolve, I think it’s good effort. > > cheers > Paris > > > > On 11 Aug 2015, at 14:58, Andra Lungu wrote: > >

Re: [Proposal] Addition to Gelly

2015-08-11 Thread Andra Lungu
ect (if any). > > Cheers, > Vasia. > On Aug 10, 2015 10:47 AM, "Andra Lungu" wrote: > > > Hey, > > > > Before actually opening a PR, I wanted to hear your opinion. So, here > goes > > nothing :). > > > > I'd like to add the core

[Proposal] Addition to Gelly

2015-08-10 Thread Andra Lungu
Hey, Before actually opening a PR, I wanted to hear your opinion. So, here goes nothing :). I'd like to add the core of my master thesis to Gelly. That is, a series of operators that take a skewed graph, split its high degree vertices into subvertices and redistribute the edges accordingly (thus

Re: A soft reminder

2015-07-30 Thread Andra Lungu
ately appreciate it. On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay wrote: > Yes, in a VertexCentricIteration with a few million nodes, running > locally on my laptop with about 10 GB of memory given to java. > > Best, > Gabor > > > > 2015-07-30 18:32 GMT+02:00 Andra Lungu : >

Re: A soft reminder

2015-07-30 Thread Andra Lungu
> Hi, > > > > I also encountered the EOF exception for a delta iteration with "more > > data". With less data it works ... > > > > Best regards, > > Felix > > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" : > > > >> Hi S

Re: [Gelly] Re-organize / cleanup examples and library

2015-07-29 Thread Andra Lungu
streaming, gelly) both Java and > Scala? > > Would help us reduce the plethora of maven projects a bit ;-) > > On Wed, Jul 29, 2015 at 2:20 PM, Andra Lungu > wrote: > > > Hi Vasia, > > > > We should document the examples in the Gelly guide, I totally agr

Re: [Gelly] Re-organize / cleanup examples and library

2015-07-29 Thread Andra Lungu
Hi Vasia, We should document the examples in the Gelly guide, I totally agree, perhaps something similar to what Giraph offers. However, if you recall, the examples are also there for test purposes. The library methods contain just the run method; no main method. For the tests, the nicest way to d

[jira] [Created] (FLINK-2412) Index Out of Bounds Exception

2015-07-27 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2412: -- Summary: Index Out of Bounds Exception Key: FLINK-2412 URL: https://issues.apache.org/jira/browse/FLINK-2412 Project: Flink Issue Type: Bug Components

Re: A soft reminder

2015-07-27 Thread Andra Lungu
p one of them? > > Greetings, > Stephan > > > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu > wrote: > > > Hi guys, > > > > I opened these JIRAs a while ago and, unfortunately, no one had time to > > look at them :|. > > > > [1] https://issues.ap

A soft reminder

2015-07-24 Thread Andra Lungu
Hi guys, I opened these JIRAs a while ago and, unfortunately, no one had time to look at them :|. [1] https://issues.apache.org/jira/browse/FLINK-2361 [2] https://issues.apache.org/jira/browse/FLINK-2360 [1] could probably be hacked, but [2] is a huge problem that was encountered by other people

Theoretical complexity of a coGroup

2015-07-22 Thread Andra Lungu
Hi everyone, I am not 100% sure about this one, so I thought that I could set my thoughts straight via the mailing list. Here's the use case. You coGroup a data set of vertices with a data set of edges. That gives you a complexity of* O(|V| * |E|)*, where |V| is the total number of vertices and |

[jira] [Created] (FLINK-2361) flatMap + distict gives eroneous results for big data sets

2015-07-14 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2361: -- Summary: flatMap + distict gives eroneous results for big data sets Key: FLINK-2361 URL: https://issues.apache.org/jira/browse/FLINK-2361 Project: Flink Issue

[jira] [Created] (FLINK-2360) EOFException

2015-07-14 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2360: -- Summary: EOFException Key: FLINK-2360 URL: https://issues.apache.org/jira/browse/FLINK-2360 Project: Flink Issue Type: Bug Components: Local Runtime

[jira] [Created] (FLINK-2352) [Graph Visualization] Integrate Gelly with Gephi

2015-07-13 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2352: -- Summary: [Graph Visualization] Integrate Gelly with Gephi Key: FLINK-2352 URL: https://issues.apache.org/jira/browse/FLINK-2352 Project: Flink Issue Type: New

Re: Documentation Webpage Rendering Problem

2015-07-13 Thread Andra Lungu
I just checked for Firefox 38.0.5, still reproducible. But I guess the fix for 39.0 will do the trick for the previous version as well :) On Mon, Jul 13, 2015 at 10:21 AM, Maximilian Michels wrote: > As far as I know this occurs only in the latest Firefox 39. > > https://issues.apache.org/jira/b

Re: The documentation site is cut on the top

2015-07-13 Thread Andra Lungu
The bug is also reproducible in Chrome... The page renders similarly to the way it does in Firefox :( On Mon, Jul 13, 2015 at 10:14 AM, Maximilian Michels wrote: > Hey Henry! > > Are you on Firefox? This only shows up for me after a recent Firefox > update. Safari still shows it correctly. The p

Passing around huge hash sets

2015-07-10 Thread Andra Lungu
Hey! It appears that my jobs have the same memory issue disguised in different Exceptions. It's expected, I am passing around hash sets of neighbors and for skewed graphs (i.e. with a lot of neighbors) it's bound to fail one way or the other. This time Kryo hates me :( Caused by: com.esotericsoft

[jira] [Created] (FLINK-2310) Add an Adamic-Adar Similarity example

2015-07-01 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2310: -- Summary: Add an Adamic-Adar Similarity example Key: FLINK-2310 URL: https://issues.apache.org/jira/browse/FLINK-2310 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-2299) The slot on which the task maanger was scheduled was killed

2015-06-30 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2299: -- Summary: The slot on which the task maanger was scheduled was killed Key: FLINK-2299 URL: https://issues.apache.org/jira/browse/FLINK-2299 Project: Flink Issue

Re: [Runtime] Division by Zero Exception

2015-06-30 Thread Andra Lungu
Sure! FLINK-2293 On Tue, Jun 30, 2015 at 10:22 AM, Fabian Hueske wrote: > That looks like a serious bug. :-( > Can you open a JIRA for that? > > Thanks, Fabian > > 2015-06-29 23:24 GMT+02:00 Andra Lungu : > > > From the same series of experiments: > > > &

[jira] [Created] (FLINK-2293) Division by Zero Exception

2015-06-30 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2293: -- Summary: Division by Zero Exception Key: FLINK-2293 URL: https://issues.apache.org/jira/browse/FLINK-2293 Project: Flink Issue Type: Bug Components

[Runtime] Division by Zero Exception

2015-06-29 Thread Andra Lungu
>From the same series of experiments: I am basically running an algorithm that simulates a Gather Sum Apply Iteration that performs Traingle Count (Why simulate it? Because you just need a superstep -> useless overhead if you use the runGatherSumApply function in Graph). What happens, at a high le

Re: Monitoring a Flink Job

2015-06-29 Thread Andra Lungu
9, 2015 at 1:58 PM, Fabian Hueske wrote: > Have you tried to use a custom accumulator that just appends to a list? > > 2015-06-29 12:59 GMT+02:00 Andra Lungu : > > > Hey Fabian, > > > > I am aware of the way open, preSuperstep(), postSuperstep() etc can help > me &

Re: Monitoring a Flink Job

2015-06-29 Thread Andra Lungu
gt; Of course, dataset.append() will be a very convenient API to add (IMHO). > > > > Best, > > Flavio > > > > > > On Sat, Jun 27, 2015 at 4:19 PM, Andra Lungu > > wrote: > > > > > Hey guys, > > > > > > Me again :) So

Monitoring a Flink Job

2015-06-27 Thread Andra Lungu
Hey guys, Me again :) So now that my wonderful job finishes, I would like to monitor it a bit (i.e. build some charts on the number of messages per vertex, compute the total amount of time elapsed per computation per vertex, etc). The main computational-intensive operation is a coGroup. There, wi

[jira] [Created] (FLINK-2254) Add Bipartite Graph Support for Gelly

2015-06-21 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2254: -- Summary: Add Bipartite Graph Support for Gelly Key: FLINK-2254 URL: https://issues.apache.org/jira/browse/FLINK-2254 Project: Flink Issue Type: New Feature

Re: Flink Runtime Exception

2015-06-19 Thread Andra Lungu
t; > On Fri, Jun 19, 2015 at 2:18 PM Andra Lungu wrote: > > > Yes, so I am using flink-0.9.0-milestone-1. Was it a problem for this > > version? > > I'll just fetch the latest master if this is the case. > > > > On Fri, Jun 19, 2015 at 2:12 PM, Till Rohr

Re: Flink Runtime Exception

2015-06-19 Thread Andra Lungu
ase try to run your example with > the latest code? > > Cheers, > Till > > On Fri, Jun 19, 2015 at 1:42 PM Andra Lungu wrote: > > > Hi everyone, > > > > I ran a job this morning on 30 wally nodes. DOP 224. Worked like a charm. > > > > Then, I ran a

Flink Runtime Exception

2015-06-19 Thread Andra Lungu
Hi everyone, I ran a job this morning on 30 wally nodes. DOP 224. Worked like a charm. Then, I ran a similar job, on the exact same configuration, on the same input data set. The only difference between them is that the second job computes the degrees per vertex and, for vertices with degree high

Re: The correct location for zipWithIndex and zipWithUniqueId

2015-06-12 Thread Andra Lungu
aSetUtils class in flink-java. > > > > > > We can easily add it to the API later, if necessary. We should however, > > > extend the documentation such that users are aware of the DataSetUtils. > > > > > > Cheers, Fabian > > > > > > 2015-0

Re: Run scala.App on Cluster

2015-06-10 Thread Andra Lungu
Hi Felix, You need to also specify the path to your main class. For example: ./flink run -p10-v --class org.apache.flink.graph.example.MusicProfiles then jar and arguments -p is the desired degree of parallelism and --class needs to be followed by the path I was talking about, Beast of luck :) A

The correct location for zipWithIndex and zipWithUniqueId

2015-06-10 Thread Andra Lungu
Hey everyone, We needed to assign unique labels as vertex values in Gelly at some point. We got a nice suggestion on how to do that in parallel (Implemented in https://github.com/apache/flink/pull/801#issuecomment-110654447). Now the question is where should these two functions go? Should they be

[jira] [Created] (FLINK-2178) groupReduceOnNeighbors throws NoSuchElementException

2015-06-06 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2178: -- Summary: groupReduceOnNeighbors throws NoSuchElementException Key: FLINK-2178 URL: https://issues.apache.org/jira/browse/FLINK-2178 Project: Flink Issue Type

Re: Planning the 0.9 Release

2015-06-05 Thread Andra Lungu
out implementing a compiler and translation test for the > > recently added Gather-Sum-Apply iteration. > > > > IMO, the second would be nice to have, but not a blocker. > > I could work on it after my paper deadline, in a week. But since I see > > you're eager to have th

Re: Gelly Roadmap

2015-06-05 Thread Andra Lungu
Hi Ufuk and Stephan, Help me out a bit! The issues currently being implemented have the corresponding JIRA issues attached to them. This is not the case for the two master theses, but there is a phrase at the end stating that there is someone working on them. Are you saying the JIRAs should point

Re: Planning the 0.9 Release

2015-06-05 Thread Andra Lungu
not that big a > deal. > > I am not very attached to the Spargel Stuff, I just want to make sure we do > not deprecate something that works well for something that is still work in > progress. > > > > > On Fri, Jun 5, 2015 at 1:46 PM, Andra Lungu wrote: > > > Hi

Re: Planning the 0.9 Release

2015-06-05 Thread Andra Lungu
Hi Stephan, I don't know if I have a saying in this, but I will give it a go :) The two unsolved issues don't affect the functionality at all. Gelly can, at the moment, support anything Spargel could. There is a guide in the documentation explaining how to migrate Spargel code to Gelly. I don't s

Re: ALS implementation

2015-06-04 Thread Andra Lungu
Hi Felix, Passing a JoinHint to your function should help. see: http://mail-archives.apache.org/mod_mbox/flink-user/201504.mbox/%3ccanc1h_vffbqyyiktzcdpihn09r4he4oluiursjnci_rwc+c...@mail.gmail.com%3E Cheers, Andra On Thu, Jun 4, 2015 at 7:07 PM, Felix Neutatz wrote: > after bug fix: > > for 1

Re: Gelly Roadmap

2015-06-03 Thread Andra Lungu
ed, May 20, 2015 at 8:58 PM, Andra Lungu > wrote: > > > The Roadmap is now available as a wiki page. > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly > > > > We're still happy to hear comments/suggestions, if any :) > > > > O

[jira] [Created] (FLINK-2141) Allow GSA's Gather to perform this operation in more than one direction

2015-06-03 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2141: -- Summary: Allow GSA's Gather to perform this operation in more than one direction Key: FLINK-2141 URL: https://issues.apache.org/jira/browse/FLINK-2141 Project:

[jira] [Created] (FLINK-2140) Access the number of vertices from within the GSA functions

2015-06-03 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2140: -- Summary: Access the number of vertices from within the GSA functions Key: FLINK-2140 URL: https://issues.apache.org/jira/browse/FLINK-2140 Project: Flink Issue

[jira] [Created] (FLINK-2127) The GSA Documentation has trailing s

2015-06-01 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2127: -- Summary: The GSA Documentation has trailing s Key: FLINK-2127 URL: https://issues.apache.org/jira/browse/FLINK-2127 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-2113) env.execute() is no longer needed after a call to print()

2015-05-29 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2113: -- Summary: env.execute() is no longer needed after a call to print() Key: FLINK-2113 URL: https://issues.apache.org/jira/browse/FLINK-2113 Project: Flink Issue

[jira] [Created] (FLINK-2093) Add a difference method to Gelly's Graph class

2015-05-26 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2093: -- Summary: Add a difference method to Gelly's Graph class Key: FLINK-2093 URL: https://issues.apache.org/jira/browse/FLINK-2093 Project: Flink Issue Type

Gelly Blog Post

2015-05-26 Thread Andra Lungu
Hey everyone, We are very excited to share the first stable draft of the Gelly blog post with you :D https://docs.google.com/document/d/1FMtpwKSE3kY7RfH082LzQpWrY6o-fdZVxqambIiC_rU/edit?usp=sharing *Feedback* is welcome, as usual! Andra

Re: Gelly Roadmap

2015-05-20 Thread Andra Lungu
The Roadmap is now available as a wiki page. https://cwiki.apache.org/confluence/display/FLINK/Flink+Gelly We're still happy to hear comments/suggestions, if any :) On Wed, May 20, 2015 at 8:43 PM, Andra Lungu wrote: > Thanks, Stephan! > > On Wed, May 20, 2015 at 8:42 PM, Steph

Re: Gelly Roadmap

2015-05-20 Thread Andra Lungu
Thanks, Stephan! On Wed, May 20, 2015 at 8:42 PM, Stephan Ewen wrote: > All right, you should have permissions now. > > On Wed, May 20, 2015 at 8:37 PM, Andra Lungu > wrote: > > > Sure, but first I need permissions! :) > > > > "*NOTE*: Due to spamming,

Re: Gelly Roadmap

2015-05-20 Thread Andra Lungu
s! On Wed, May 20, 2015 at 4:27 PM, Vasiliki Kalavri wrote: > Thank you for your feedback and ideas everyone! > > @Andra, how about moving the roadmap to the wiki? > > On 20 May 2015 at 15:48, Kostas Tzoumas wrote: > > > :-D > > > > Great! > > > &g

Re: Gelly Roadmap

2015-05-19 Thread Andra Lungu
he roadmap > > On Tue, May 19, 2015 at 10:55 AM, Andra Lungu > wrote: > > > Apache TinkerPop made it to the wish-list :) > > Thanks, Fabian! > > > > On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier < > pomperma...@okkam.it> > > wrote: > > &

Re: Gelly Roadmap

2015-05-19 Thread Andra Lungu
extending Gelly and our vision is > > to > > > > turn it into something much more than "yet another library for > running > > > > PageRank-like algorithms". The document that Andra has shared > contains > > > > ongoing work and our ideas on mid- and long-t

Gelly Roadmap

2015-05-18 Thread Andra Lungu
Dear Squirrels, The following document is intended to give you an insight on where Flink's Graph API, Gelly, is at the moment and especially on the direction in which it's heading in the near future. https://docs.google.com/document/d/11ovK1xZZTl3Qinw6Ud1rqpaDhTvRHDv8PWBzcHITFkM/edit?usp=sharing

[jira] [Created] (FLINK-2012) addVertices, addEdges, removeVertices, removeEdges methods

2015-05-14 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2012: -- Summary: addVertices, addEdges, removeVertices, removeEdges methods Key: FLINK-2012 URL: https://issues.apache.org/jira/browse/FLINK-2012 Project: Flink Issue

Re: [Question]Test failed in cluster mode

2015-05-13 Thread Andra Lungu
vironment. > >> > >> Stephan > >> > >> > >> > >> > >> On Wed, May 13, 2015 at 12:44 AM, Yi ZHOU > wrote: > >> > >> Hello, > >>> > >>> Thanks Andra for the gaussian sequence generation.

Re: Generat DataSet gaussian distribution

2015-05-08 Thread Andra Lungu
Hi Yi, To my knowledge, there is no simple way to generate this kind of DataSet(i.e. there is no env.generateGaussianSequence()). However, if you look in flink-perf, Till used something like this there: https://github.com/project-flink/flink-perf/blob/master/flink-jobs/src/main/scala/com/github/pr

Re: [Documentation] Broken links

2015-05-07 Thread Andra Lungu
> Sorry about that. I'll try to get to some time in the next days... > > > > Greetings, > > Stephan > > > > > > On Fri, Apr 24, 2015 at 5:00 PM, Andra Lungu > > wrote: > > > > > Hey everyone, > > > > > > I was follo

[jira] [Created] (FLINK-1987) Broken links in the add_operator section of the documentation

2015-05-07 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1987: -- Summary: Broken links in the add_operator section of the documentation Key: FLINK-1987 URL: https://issues.apache.org/jira/browse/FLINK-1987 Project: Flink

[jira] [Created] (FLINK-1976) Add ForwardedFields* hints for the optimizer

2015-05-05 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1976: -- Summary: Add ForwardedFields* hints for the optimizer Key: FLINK-1976 URL: https://issues.apache.org/jira/browse/FLINK-1976 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-1975) Graph getUndirected improvement

2015-05-05 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1975: -- Summary: Graph getUndirected improvement Key: FLINK-1975 URL: https://issues.apache.org/jira/browse/FLINK-1975 Project: Flink Issue Type: Improvement

Re: Adding a new operator

2015-04-27 Thread Andra Lungu
hes many core parts of the > > system. > > It would be good to start a discussion about that early in the process to > > make sure that the design is aligned with the system. > > Otherwise, duplicated work might be necessary before it can be added to > the > &

Re: Adding a new operator

2015-04-26 Thread Andra Lungu
> As you can see in the DriverStrategy enum there is several different > ReduceDrivers depending on the strategy the optimizer chooses. > > best, > Markus > > 2015-04-26 12:26 GMT+02:00 Andra Lungu : > > > Hey guys, > > > > I am trying to add a new runtime operato

Adding a new operator

2015-04-26 Thread Andra Lungu
Hey guys, I am trying to add a new runtime operator; To this end, I am following the guide here: http://ci.apache.org/projects/flink/flink-docs-master/internals/add_operator.html and the code itself. >From what I understood, the run() in ReduceDriver, for instance, should be called every time a

[Documentation] Broken links

2015-04-24 Thread Andra Lungu
Hey everyone, I was following the documentation on how to create a new runtime operator and I noticed that all the links to the classes on Github return 404. http://ci.apache.org/projects/flink/flink-docs-master/internals/add_operator.html Eventually, I started to check the code out directly fro

Re: Merge Python API

2015-04-20 Thread Andra Lungu
+1 from me too, the presentation of the Python API seemed okay, plus that having one more project in staging will not hurt. On Mon, Apr 20, 2015 at 2:49 PM, Gyula Fóra wrote: > +1 > > On Mon, Apr 20, 2015 at 2:41 PM, Fabian Hueske wrote: > > > +1 > > > > 2015-04-20 14:39 GMT+02:00 Maximilian Mi

Re: [Gelly] Vertex-centric iteration updateVertex does not get called

2015-04-16 Thread Andra Lungu
Hello Gabor, Yes, currently updateVertex only gets called when a new message was received. Could you please describe the logic behind your triangle count? The one I know is described at the beginning of page 1643 in this article: http://www.cc.gatech.edu/~bader/papers/GraphBSPonXMT-MTAAP2013.pdf

[jira] [Created] (FLINK-1883) Add Min Vertex ID Propagation Library Method and Example

2015-04-13 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1883: -- Summary: Add Min Vertex ID Propagation Library Method and Example Key: FLINK-1883 URL: https://issues.apache.org/jira/browse/FLINK-1883 Project: Flink Issue

Re: Memory segment error

2015-03-30 Thread Andra Lungu
ossible to increase > that a bit. > > 2015-03-30 23:25 GMT+02:00 Andra Lungu : > > > Hi Fabian, > > > > I'll see what I can do :). > > I am just a bit shocked. If this set of coGroups and joins was too much > for > >

Re: Memory segment error

2015-03-30 Thread Andra Lungu
TM). > > Do you see a way to reduce the number of operators in your testcase, maybe > by splitting it in half? > > 2015-03-30 11:01 GMT+02:00 Andra Lungu : > > > Sure, > > > > It was in the first mail but that was sent a while ago :) > > > > This is the cod

Re: Memory segment error

2015-03-30 Thread Andra Lungu
e that > gives the error? > > Then I have a look at it and try to figure out what's going wrong. > > Cheers, Fabian > > 2015-03-30 10:43 GMT+02:00 Andra Lungu : > > > Hello, > > > > I went on and did some further debugging on this issue. Even though th

Re: Memory segment error

2015-03-30 Thread Andra Lungu
e(); } collector.collect(new Vertex(vertex.getId(), sum)); } }); To me, nothing seems out of the ordinary here. This is regular user code. And the behaviour in the end is definitely not the one expected. Any idea why this might be happening? Thanks! Andra On Fri, Mar 27

Re: Memory segment error

2015-03-26 Thread Andra Lungu
Opps! Sorry! Did not know the mailing list does not support attachments :) https://gist.github.com/andralungu/fba36d77f79189daa183 On Fri, Mar 27, 2015 at 12:02 AM, Andra Lungu wrote: > Hi Fabian, > > I uploaded a file with my execution plan. > > On Thu, Mar 26, 2015 at 11:50 PM

Re: Memory segment error

2015-03-26 Thread Andra Lungu
d the size of a memory page is > 32KB, there should be more than 25k pages available. > > Can you post the execution plan of the program you execute ( > ExecutionEnvironment.getExecutionPlan() )? > > Best, Fabian > > 2015-03-26 23:31 GMT+01:00 Andra Lungu : > > > For

Re: Memory segment error

2015-03-26 Thread Andra Lungu
For 20 edges and 5 nodes, that should be more thank enough. On Thu, Mar 26, 2015 at 11:24 PM, Andra Lungu wrote: > Sure, > > 3470 [main] INFO org.apache.flink.runtime.taskmanager.TaskManager - > Using 820 MB for Flink managed memory. > > On Thu, Mar 26, 2015 at 4:48

Re: Memory segment error

2015-03-26 Thread Andra Lungu
:09,669 INFO org.apache.flink.runtime.taskmanager.TaskManager > - Using 1193 MB for Flink managed memory. > > Can you tell us how much memory Flink is managing in your case? > > > > On Thu, Mar 26, 2015 at 4:46 PM, Andra Lungu > wrote: > > > Hello everyone, > > > > I guess I need to revive thi

Memory segment error

2015-03-26 Thread Andra Lungu
Hello everyone, I guess I need to revive this old discussion: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Memory-segment-error-when-migrating-functional-code-from-Flink-0-9-to-0-8-td3687.html At that point, the fix was to kindly ask Alex to make his project work with

Re: Queries regarding RDFs with Flink

2015-03-22 Thread Andra Lungu
Hi Flavio, We don't have a specific example for generating RDF graphs using Gelly, but I will try to drop some lines of code here and hope you will find them useful. An RDF statement is formed of Subject - Predicate - Object triples. In Edge notation, the Subject and the Object will be the source

Re: [Delta Iterations] The dirty insides(insights)

2015-03-21 Thread Andra Lungu
rk. > > Greetings, > Stephan > > > On Fri, Mar 20, 2015 at 12:00 PM, Andra Lungu > wrote: > > > Hi Stephan, > > > > What I am trying to do, among other things, is to make the vertex's > > inDegree and outDegree available in the vertexUpdate and

Re: [Delta Iterations] The dirty insides(insights)

2015-03-20 Thread Andra Lungu
e you > pasted is from the Spargel API - specifically just forwarding registered > broadcast variables. > > What do you mean with "the vertex values get reset" ? > > Stephan > > PS: The delta iterations are based in this paper: > http://arxiv.org/pdf/1208.0088.

[Delta Iterations] The dirty insides(insights)

2015-03-19 Thread Andra Lungu
Hello, I've used delta iterations several times up until now, but I just realized that I never fully understood what happens inside. And the documentation only explains things from a user's perspective. Which is why I could really use your help :). Here goes nothing: In Gelly, Graph.java, there i

[jira] [Created] (FLINK-1741) Add Jaccard Similarity Metric Example

2015-03-18 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-1741: -- Summary: Add Jaccard Similarity Metric Example Key: FLINK-1741 URL: https://issues.apache.org/jira/browse/FLINK-1741 Project: Flink Issue Type: Task

  1   2   >