Re: streaming iteration

2015-07-27 Thread Gyula Fóra
Hey, The JobGraph that is executed cannot have cycles (due to scheduling reasons), that's why there is no edge between the head and tail operators. What we do instead is we add an extra source to the head and a sink to the tail (with the same parallelism), and the feedback data is passed outside o

streaming iteration

2015-07-27 Thread MaGuoWei
Hi,guysI am reading the code about streaming iteration.I find there is no StreamEdge the StreamIterationHead and StreamIterationTail.Why? Because the graph can't have any ring?Or any other concerns? thanks

RE: add some new api to the scheduler in the job manager

2015-07-27 Thread MaGuoWei
thanks all you guys.Now I know I can achieve this goal by creating a cluster per topology and estimating the cluster size by analyzing the JobGraph(or by some user input).But I think it can be more beautiful if Flink can abstract some common resource api such as get/release/deploy. :-) thanks.

[jira] [Created] (FLINK-2415) Link nodes in plan to vertices

2015-07-27 Thread Piotr Godek (JIRA)
Piotr Godek created FLINK-2415: -- Summary: Link nodes in plan to vertices Key: FLINK-2415 URL: https://issues.apache.org/jira/browse/FLINK-2415 Project: Flink Issue Type: Improvement Co

[jira] [Created] (FLINK-2414) Extend job data in job list

2015-07-27 Thread Piotr Godek (JIRA)
Piotr Godek created FLINK-2414: -- Summary: Extend job data in job list Key: FLINK-2414 URL: https://issues.apache.org/jira/browse/FLINK-2414 Project: Flink Issue Type: Improvement Compo

[jira] [Created] (FLINK-2413) Provide real vertex data

2015-07-27 Thread Piotr Godek (JIRA)
Piotr Godek created FLINK-2413: -- Summary: Provide real vertex data Key: FLINK-2413 URL: https://issues.apache.org/jira/browse/FLINK-2413 Project: Flink Issue Type: Improvement Componen

Travis build failure

2015-07-27 Thread Sachin Goel
A recent travis build[Job 3] on my forked repo failed with the following error: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project flink-tachyon: ExecutionException: java.lang.RuntimeException: The forked VM terminated without properly saying

[jira] [Created] (FLINK-2412) Index Out of Bounds Exception

2015-07-27 Thread Andra Lungu (JIRA)
Andra Lungu created FLINK-2412: -- Summary: Index Out of Bounds Exception Key: FLINK-2412 URL: https://issues.apache.org/jira/browse/FLINK-2412 Project: Flink Issue Type: Bug Components:

[jira] [Created] (FLINK-2411) Add basic graph summarization algorithm

2015-07-27 Thread Martin Junghanns (JIRA)
Martin Junghanns created FLINK-2411: --- Summary: Add basic graph summarization algorithm Key: FLINK-2411 URL: https://issues.apache.org/jira/browse/FLINK-2411 Project: Flink Issue Type: New F

[jira] [Created] (FLINK-2410) PojoTypeInfo is not completely serializable

2015-07-27 Thread Timo Walther (JIRA)
Timo Walther created FLINK-2410: --- Summary: PojoTypeInfo is not completely serializable Key: FLINK-2410 URL: https://issues.apache.org/jira/browse/FLINK-2410 Project: Flink Issue Type: Bug

Re: add some new api to the scheduler in the job manager

2015-07-27 Thread Robert Metzger
Hi MaGuoWei, would you like to have done automatically by Flink or based on some user input? Adding commands to the ./bin/yarn-session.sh to change the cluster size is quite easy. However, reducing the cluster size while a job is running will fail the job. Making this automatically is much harder

Re: Flink contributor list

2015-07-27 Thread Márton Balassi
manmat - Mátyás Manninger, aspired to be a GSoC student last year. On Mon, Jul 27, 2015 at 12:32 PM, Kostas Tzoumas wrote: > Cool, I added a link to the website. Keep the corrections coming > > On Mon, Jul 27, 2015 at 11:30 AM, Vasiliki Kalavri < > vasilikikala...@gmail.com> wrote: > > > Thanks

Re: Getting an eeror while running the code

2015-07-27 Thread Chiwan Park
Oh, I confused Streaming API with Batch API. :) Stephen’s comment will help you. Regards, Chiwan Park > On Jul 27, 2015, at 4:22 PM, Stephan Ewen wrote: > > Your program gives this exception: java.lang.UnsupportedClassVersionError: > > This usually means that a JVM tries to load code that has

[jira] [Created] (FLINK-2409) Old JM web interface is sending cancel messages w/o leader ID

2015-07-27 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2409: - Summary: Old JM web interface is sending cancel messages w/o leader ID Key: FLINK-2409 URL: https://issues.apache.org/jira/browse/FLINK-2409 Project: Flink

Re: Flink contributor list

2015-07-27 Thread Kostas Tzoumas
Cool, I added a link to the website. Keep the corrections coming On Mon, Jul 27, 2015 at 11:30 AM, Vasiliki Kalavri < vasilikikala...@gmail.com> wrote: > Thanks for compiling this list! It's impressive indeed :)) > > joey001 is Yi Zhou. He applied for GSoC and contributed to Gelly. > > +1 for add

[jira] [Created] (FLINK-2408) Some maven properties are defined in build profiles only

2015-07-27 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2408: - Summary: Some maven properties are defined in build profiles only Key: FLINK-2408 URL: https://issues.apache.org/jira/browse/FLINK-2408 Project: Flink Issu

Re: add some new api to the scheduler in the job manager

2015-07-27 Thread Maximilian Michels
Hi MaGuoWei, If I understand correctly, you're are looking for a way to have a job manager (master) contentiously running which requests YARN containers for the task managers (workers) on the fly. This is currently not supported by Flink although you can add or remove task managers while the clust

Re: A soft reminder

2015-07-27 Thread Felix Neutatz
Hi, I also encountered the EOF exception for a delta iteration with "more data". With less data it works ... Best regards, Felix Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" : > Hi Stephan, > > I tried to debug a bit around the EOF Exception. It seems that I am pretty > useless on my own. I h

Re: Flink contributor list

2015-07-27 Thread Vasiliki Kalavri
Thanks for compiling this list! It's impressive indeed :)) joey001 is Yi Zhou. He applied for GSoC and contributed to Gelly. +1 for adding this list to the website or wiki and link from the site. -Vasia. On 27 July 2015 at 11:40, Szabó Péter wrote: > Tofi007 - Kristóf Kovács (he's a hungarian

RE: add some new api to the scheduler in the job manager

2015-07-27 Thread MaGuoWei
It is great! Is there any document? I am very interested in this. thanks > From: aljos...@apache.org > Date: Mon, 27 Jul 2015 05:14:00 + > Subject: Re: add some new api to the scheduler in the job manager > To: dev@flink.apache.org > > Hi, > I think for more details on giving back resources

Re: Flink contributor list

2015-07-27 Thread Szabó Péter
Tofi007 - Kristóf Kovács (he's a hungarian student worked with us in summer 2014) 2015-07-27 10:25 GMT+02:00 Maximilian Michels : > Hi Kostas, > > Thanks for creating the list! Looks impressive. > > As for keeping the list up to date: We could add a CONTRIBUTORS file in the > repository. This is

Re: New Flink Runtime Monitor Dashboard

2015-07-27 Thread Maximilian Michels
We could open a pull request against the linguist repository which holds the exclusion rules for the graph. It already has rules to exclude minified Javascript files. We might be able to exclude ours too if its name follows a specific pattern. https://github.com/github/linguist/blob/master/lib/lin

Re: Flink contributor list

2015-07-27 Thread Maximilian Michels
Hi Kostas, Thanks for creating the list! Looks impressive. As for keeping the list up to date: We could add a CONTRIBUTORS file in the repository. This is common practice in many open source projects. Alternatively, we could also create a dedicated page on the web site. The ASF also has a contrib

Re: A soft reminder

2015-07-27 Thread Andra Lungu
Hi Stephan, I tried to debug a bit around the EOF Exception. It seems that I am pretty useless on my own. I have no idea how to fix it. And it occurs in quite a lot of experiments... I believe it's something behind a delta iteration [this is the common point between my experiments and the others'

Re: A soft reminder

2015-07-27 Thread Stephan Ewen
Hey! It is pretty much as you said: No one had the time to fix this so far. They are on the list, though, Do you want to pick up one of them? Greetings, Stephan On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu wrote: > Hi guys, > > I opened these JIRAs a while ago and, unfortunately, no one had

Re: Getting an eeror while running the code

2015-07-27 Thread Stephan Ewen
Your program gives this exception: java.lang.UnsupportedClassVersionError: This usually means that a JVM tries to load code that has been compiled with a newer Java version. For example, Java 7 running a Java 8 program. Per stack Overflow: http://stackoverflow.com/questions/10382929/how-to-fix-un