[ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Robert Metzger
The Apache Flink team is proud to announce the next version of Apache Flink. Find the blogpost with the change log here: http://flink.apache.org/news/2015/01/21/release-0.8.html I would like to thank Márton Balassi for being the release manager of the first Apache Flink release as a top level pro

Re: [ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Fabian Hueske
Awesome! Thank you very much Marton and Robert! Cheers, Fabian 2015-01-22 9:04 GMT+01:00 Robert Metzger : > The Apache Flink team is proud to announce the next version of Apache > Flink. > > Find the blogpost with the change log here: > http://flink.apache.org/news/2015/01/21/release-0.8.html >

[jira] [Created] (FLINK-1432) CombineTaskTest.testCancelCombineTaskSorting sometimes fails

2015-01-22 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1432: - Summary: CombineTaskTest.testCancelCombineTaskSorting sometimes fails Key: FLINK-1432 URL: https://issues.apache.org/jira/browse/FLINK-1432 Project: Flink

Re: Very strange behaviour of groupBy() -> sort() -> first()

2015-01-22 Thread Fabian Hueske
BTW, I think as well that global sorting is an important feature and definitely missing in Flink (FLINK-598). Enabling local sorting for data sinks is one step on the way which can be rather easily solved (FLINK-1105). If you would like to contribute to make sorting possible, I would be very happy

[jira] [Created] (FLINK-1433) Add HADOOP_CLASSPATH to start scripts

2015-01-22 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1433: - Summary: Add HADOOP_CLASSPATH to start scripts Key: FLINK-1433 URL: https://issues.apache.org/jira/browse/FLINK-1433 Project: Flink Issue Type: Improvement

Re: Turn lazy operator execution off for streaming jobs

2015-01-22 Thread Till Rohrmann
I'm not sure whether it is currently possible to schedule first the receiver and then the sender. Recently, I had to fix the TaskManagerTest.testRunWithForwardChannel test case where this was exactly the case. Due to first scheduling the receiver, it happened sometimes that an IllegalQueueIteratorR

Re: Turn lazy operator execution off for streaming jobs

2015-01-22 Thread Ufuk Celebi
On 22 Jan 2015, at 11:37, Till Rohrmann wrote: > I'm not sure whether it is currently possible to schedule first the > receiver and then the sender. Recently, I had to fix the > TaskManagerTest.testRunWithForwardChannel test case where this was exactly > the case. Due to first scheduling the rece

[jira] [Created] (FLINK-1434) Web interface cannot be used to run streaming programs

2015-01-22 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-1434: - Summary: Web interface cannot be used to run streaming programs Key: FLINK-1434 URL: https://issues.apache.org/jira/browse/FLINK-1434 Project: Flink Issue Type: Bu

[jira] [Created] (FLINK-1435) TaskManager does not log missing memory error on start up

2015-01-22 Thread Malte Schwarzer (JIRA)
Malte Schwarzer created FLINK-1435: -- Summary: TaskManager does not log missing memory error on start up Key: FLINK-1435 URL: https://issues.apache.org/jira/browse/FLINK-1435 Project: Flink I

Re: [flink-streaming] Regarding loops in the Job Graph

2015-01-22 Thread Paris Carbone
Thanks for the quick answers! It is possible to use iterations, we could detect circles while building the samoa topology and convert them into iterations. It is perhaps the proper way to go. I just thought whether we could hack around it but we better avoid messing with cyclic dependences. Pa

ClassLoader issue when submitting Flink Streaming programs through the web cliend

2015-01-22 Thread Gyula Fóra
Hey, While trying to add support for plan visualisation I also realised that streaming programs cannot be run through the flink web client. It seems to be a ClassLoader issue, which is interesting because we use the classloader set in the environment to deserialize user defined objects. This work

Re: [jira] [Commented] (FLINK-1410) Integrate Flink version variables into website layout

2015-01-22 Thread Till Rohrmann
+1 On Thu, Jan 22, 2015 at 2:42 PM, Max Michels (JIRA) wrote: > > [ > https://issues.apache.org/jira/browse/FLINK-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287423#comment-14287423 > ] > > Max Michels commented on FLINK-1410: > -

[jira] [Created] (FLINK-1436) Command-line interface verbose option (-v)

2015-01-22 Thread Max Michels (JIRA)
Max Michels created FLINK-1436: -- Summary: Command-line interface verbose option (-v) Key: FLINK-1436 URL: https://issues.apache.org/jira/browse/FLINK-1436 Project: Flink Issue Type: Improvement

Re: ClassLoader issue when submitting Flink Streaming programs through the web cliend

2015-01-22 Thread Robert Metzger
Didn't we have a similar issue before the 0.7.0-incubating release as well? I thought I've tested submitting a streaming program with the web frontend for the 0.8 release and it worked. On Thu, Jan 22, 2015 at 2:31 PM, Gyula Fóra wrote: > Hey, > > While trying to add support for plan visualisat

Re: ClassLoader issue when submitting Flink Streaming programs through the web cliend

2015-01-22 Thread Gyula Fóra
Actually I might have found the problem. I was unaware of the OptimizerPlanEnvironment at the time I implemented the StreamExecutionEnvironment. So currently if the getExecutionEnvironment method of the StreamExecutionEnvironment does not detect a ContextEnvironment (with instanceOf check) it crea

[jira] [Created] (FLINK-1437) Bug in PojoSerializer's copy() method

2015-01-22 Thread Timo Walther (JIRA)
Timo Walther created FLINK-1437: --- Summary: Bug in PojoSerializer's copy() method Key: FLINK-1437 URL: https://issues.apache.org/jira/browse/FLINK-1437 Project: Flink Issue Type: Bug C

Re: [ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Kostas Tzoumas
Great job, thank you Marton, Robert and everyone! On Thu, Jan 22, 2015 at 12:50 AM, Fabian Hueske wrote: > Awesome! > > Thank you very much Marton and Robert! > > Cheers, Fabian > > 2015-01-22 9:04 GMT+01:00 Robert Metzger : > > > The Apache Flink team is proud to announce the next version of Ap

Re: [ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Henry Saputra
Awesome job guys! We should also send email to annou...@apache.org for the release too. It can be picked up by Sally and maybe get retweeted from official ASF Twitter handle =) - Henry On Thu, Jan 22, 2015 at 8:01 AM, Kostas Tzoumas wrote: > Great job, thank you Marton, Robert and everyone! >

Re: [ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Stephan Ewen
Thank you very much indeed! On Thu, Jan 22, 2015 at 8:56 AM, Henry Saputra wrote: > Awesome job guys! > > We should also send email to annou...@apache.org for the release too. > > It can be picked up by Sally and maybe get retweeted from official ASF > Twitter handle =) > > - Henry > > On Thu, J

Re: Turn lazy operator execution off for streaming jobs

2015-01-22 Thread Stephan Ewen
So the crux is that the JobManager has a location for the sender task (and tell that to the receivers) before the senders have registered their transfer queues. Can we just establish a "happens-before" there? - The TaskManager may send the "ack" to the deployment call only after all queues are re

Re: [ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Kostas Tzoumas
Great idea, I will send an email to announce@ On Thu, Jan 22, 2015 at 8:56 AM, Henry Saputra wrote: > Awesome job guys! > > We should also send email to annou...@apache.org for the release too. > > It can be picked up by Sally and maybe get retweeted from official ASF > Twitter handle =) > > - H

Re: [flink-streaming] Regarding loops in the Job Graph

2015-01-22 Thread Stephan Ewen
If this becomes a strong requirement, then we can look into relaxing the constraints (and then have some features not supported on cyclic flows). I just wanted to get a discussion started about the different angles of approach, and what may be the simplest way to do this... On Thu, Jan 22, 2015 a

[ANNOUNCE] Apache Flink 0.8.0 released

2015-01-22 Thread Kostas Tzoumas
The Apache Flink team is proud to announce the release of Apache Flink 0.8.0 This is the first release of Apache Flink as an Apache Top-Level Project Flink is building a system for distributed batch and real-time streaming data analysis that offers familiar collection-based programming APIs in Ja