Re: Drafting a roadmap for Flink

2015-02-04 Thread Stephan Ewen
I agree, Henry. To me, this is a low priority feature and not one in the main line of what I see as the core Flink direction. I included it in there, because it is actually a fairly low hanging fruit. Once the stuff for the Zeppelin Integration is in place, it is a neat project for someone (extern

Re: Drafting a roadmap for Flink

2015-02-04 Thread Kostas Tzoumas
Yeah, this is something that nobody is working on AFAIK and not a major focus of the project. I moved it to the interesting projects page (and cleaned up the page a bit removing streaming, tez, and mahout as these are ongoing efforts). On Wed, Feb 4, 2015 at 9:34 AM, Stephan Ewen wrote: > I agre

Sorting of fields

2015-02-04 Thread Timo Walther
Hey, is it correct that we currently do not support sorting without any grouping? I had this question by 2 users in the last weeks and now I also need this functionality. Is it possible to sort e.g. a word count result Tuple2 by count? Regards, Timo

Cluster execution - Jobmanager unreachable

2015-02-04 Thread Chesnay Schepler
Hello, I'm trying to run python jobs with the latest master on a cluster and get the following exception: Error: The program execution failed: JobManager not reachable anymore. Terminate waiting for job answer. org.apache.flink.client.program.ProgramInvocationException: The program execution

Re: Sorting of fields

2015-02-04 Thread Timo Walther
Ok, I found an earlier discussion about it. Sorry for the mail. However, I think this is a very important feature and I should be added soon. On 04.02.2015 14:38, Timo Walther wrote: Hey, is it correct that we currently do not support sorting without any grouping? I had this question by 2 u

January 2015 in the Flink community

2015-02-04 Thread Kostas Tzoumas
Here is a digestible read on some January activity in the Flink community: http://flink.apache.org/news/2015/02/04/january-in-flink.html Highlights: - Flink 0.8.0 was released - The Flink community published a technical roadmap for 2015 - Flink was used to scale matrix factorization to extreme

Re: Sorting of fields

2015-02-04 Thread Fabian Hueske
I just merged support for local output sorting yesterday :-) This allows to sort the data before it is given to the OutputFormat. It is done like this: myData.write(myOF).sortLocalOutput(1, Order.ASCENDING); See the programming guide for details (only in master, not online). Full sorting can be

[jira] [Created] (FLINK-1473) Simplify SplittableIterator interface

2015-02-04 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1473: - Summary: Simplify SplittableIterator interface Key: FLINK-1473 URL: https://issues.apache.org/jira/browse/FLINK-1473 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-1474) Cross produces wrong results for non-tiny inputs

2015-02-04 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-1474: Summary: Cross produces wrong results for non-tiny inputs Key: FLINK-1474 URL: https://issues.apache.org/jira/browse/FLINK-1474 Project: Flink Issue Type: Bu

Re: Drafting a roadmap for Flink

2015-02-04 Thread Henry Saputra
Thanks Stephan, Kostas, sounds good to me On Wed, Feb 4, 2015 at 2:57 AM, Kostas Tzoumas wrote: > Yeah, this is something that nobody is working on AFAIK and not a major > focus of the project. I moved it to the interesting projects page (and > cleaned up the page a bit removing streaming, tez, a

[jira] [Created] (FLINK-1475) Minimize log output of yarn test cases

2015-02-04 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-1475: Summary: Minimize log output of yarn test cases Key: FLINK-1475 URL: https://issues.apache.org/jira/browse/FLINK-1475 Project: Flink Issue Type: Bug

suitable implementation tasks / student projects?

2015-02-04 Thread Adnan Khan
Hello! My name is Adnan and I'd like to learn more about Flink and get involved with the project. I'm an upper year cs/math student and I've been looking for an open source project in the large scale data processing, distributed systems, and databases space. So I'm excited to learn more about Flin

Re: suitable implementation tasks / student projects?

2015-02-04 Thread Fabian Hueske
Hi Adnan, as promised in IRC, here a few JIRAs that touch the internals. JIRAs are ordered from rather easy to more difficult (according to my subjective judgement ;-) ) - FLINK-1388: Support POJO data types with writeAsCsv (API) - FLINK-703: Use complete element as Join key (API) - FLINK-1418: F

[jira] [Created] (FLINK-1476) Flink VS Spark on loop test

2015-02-04 Thread xuhong (JIRA)
xuhong created FLINK-1476: - Summary: Flink VS Spark on loop test Key: FLINK-1476 URL: https://issues.apache.org/jira/browse/FLINK-1476 Project: Flink Issue Type: Test Affects Versions: 0.8, 0.7.0

Re: [jira] [Created] (FLINK-1462) Add documentation guide for the graph API

2015-02-04 Thread Ufuk Celebi
I think it is GitHub flavored markdown. GH has a reference with the additions. On Wednesday, February 4, 2015, Vasia Kalavri (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/FLINK-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306036#