[jira] [Created] (FLINK-1493) Support for streaming jobs preserving global ordering of records

2015-02-06 Thread JIRA
Márton Balassi created FLINK-1493: - Summary: Support for streaming jobs preserving global ordering of records Key: FLINK-1493 URL: https://issues.apache.org/jira/browse/FLINK-1493 Project: Flink

Streaming fault tolerance with event sending

2015-02-06 Thread Hermann Gábor
Hey, We've been implementing a simple Storm-like fault tolerance system with persisting source records and keeping track of all the records (whether they've been processed), and replaying them if they fail. The straightforward way to do this was creating a special AbstractJobVertex (and AbstractIn

[jira] [Created] (FLINK-1492) Exceptions on shutdown concerning BLOB store cleanup

2015-02-06 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1492: --- Summary: Exceptions on shutdown concerning BLOB store cleanup Key: FLINK-1492 URL: https://issues.apache.org/jira/browse/FLINK-1492 Project: Flink Issue Type:

[jira] [Created] (FLINK-1491) Inconsistent logging between Akka and other components

2015-02-06 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-1491: --- Summary: Inconsistent logging between Akka and other components Key: FLINK-1491 URL: https://issues.apache.org/jira/browse/FLINK-1491 Project: Flink Issue Type

[jira] [Created] (FLINK-1490) Incorrect local output sort for tuple position keys with nested tuples

2015-02-06 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-1490: Summary: Incorrect local output sort for tuple position keys with nested tuples Key: FLINK-1490 URL: https://issues.apache.org/jira/browse/FLINK-1490 Project: Flink

Re: Eclipse JDT, Java 8, lambdas

2015-02-06 Thread Robert Metzger
Sorry, I didn't see the pathID which is added in the map() method. Then your approach looks good for using Flink locally. On Fri, Feb 6, 2015 at 3:03 PM, Nam-Luc Tran wrote: > Thank you for your replies. > > @Stephen > Updating to 0.9-SNAPSHOT and using the "return" statement did the trick. I >

[jira] [Created] (FLINK-1489) Failing JobManager due to blocking calls in Execution.scheduleOrUpdateConsumers

2015-02-06 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-1489: Summary: Failing JobManager due to blocking calls in Execution.scheduleOrUpdateConsumers Key: FLINK-1489 URL: https://issues.apache.org/jira/browse/FLINK-1489 Project

Re: Eclipse JDT, Java 8, lambdas

2015-02-06 Thread Nam-Luc Tran
Thank you for your replies. @Stephen Updating to 0.9-SNAPSHOT and using the "return" statement did the trick. I will try the 4.5 M4 release and give a feedback on how it went. @Robert I launch the job right from the Eclipse IDE. Also, each file in the folder contains data for a different trajecto

Re: Google Summer of Code 2015 is coming

2015-02-06 Thread Robert Metzger
Some of the community members here have already some experience with GSoC because back in the Stratosphere days we were also participating in GSoC. In my experience its more work than one might think in the beginning. But it would be great if some committers in the projects agree to mentor some st

[jira] [Created] (FLINK-1488) JobManager web interface logfile access broken

2015-02-06 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1488: - Summary: JobManager web interface logfile access broken Key: FLINK-1488 URL: https://issues.apache.org/jira/browse/FLINK-1488 Project: Flink Issue Type: Bu

Re: Planning Release 0.8.1

2015-02-06 Thread Robert Metzger
It seems that quite a few important fixes still need some work until they are ready. I'll extend the "deadline" to Monday morning (CET), since we can not vote during the weekends anyways. On Fri, Feb 6, 2015 at 1:16 PM, Stephan Ewen wrote: > I am working on fixes for local split assignment and c

[jira] [Created] (FLINK-1487) Failing SchedulerIsolatedTasksTest.testScheduleQueueing test case

2015-02-06 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-1487: Summary: Failing SchedulerIsolatedTasksTest.testScheduleQueueing test case Key: FLINK-1487 URL: https://issues.apache.org/jira/browse/FLINK-1487 Project: Flink

Re: Eclipse JDT, Java 8, lambdas

2015-02-06 Thread Stephan Ewen
Hi Tran Nam-Luc! Java 8 lambdas are a bit tricky right now, because of Java's generic type erasure. Flink needs the type information of all Functions before sending programs to the execution engine. (for more details, see here: https://github.com/apache/flink/blob/master/docs/internal_types_serial

Re: Eclipse JDT, Java 8, lambdas

2015-02-06 Thread Stephan Ewen
Hey! I just saw that the fix was ported to the Eclipse 4.5 M4 release, which can be downloaded here: https://www.eclipse.org/downloads/index-developer.php I am eager to try that out for myself later. If the fix is really included, this Eclipse version should support lambdas elegantly out of the b

Re: Planning Release 0.8.1

2015-02-06 Thread Stephan Ewen
I am working on fixes for local split assignment and custom input split. Would be good to include those as well... On Fri, Feb 6, 2015 at 12:52 PM, Aljoscha Krettek wrote: > @robert, yes, will do > > On Fri, Feb 6, 2015 at 12:28 PM, Márton Balassi > wrote: > > Found a streaming bug, Gyula fixed

[jira] [Created] (FLINK-1486) Add a string to the print method to identify output

2015-02-06 Thread Max Michels (JIRA)
Max Michels created FLINK-1486: -- Summary: Add a string to the print method to identify output Key: FLINK-1486 URL: https://issues.apache.org/jira/browse/FLINK-1486 Project: Flink Issue Type: Imp

Re: Planning Release 0.8.1

2015-02-06 Thread Aljoscha Krettek
@robert, yes, will do On Fri, Feb 6, 2015 at 12:28 PM, Márton Balassi wrote: > Found a streaming bug, Gyula fixed it. Pushing it soon to both master and > branch-0.8. > > On Fri, Feb 6, 2015 at 11:51 AM, Robert Metzger wrote: > >> @Aljoscha, can you merge the backported fix to the "release-0.8"

Re: Eclipse JDT, Java 8, lambdas

2015-02-06 Thread Robert Metzger
Hi, looking at your code, it seems that you are creating a DataSet for each file in the "directory". Flink can also read entire directories. Now regarding the actual problem: How are you starting the Flink job? Out of your IDE, or using the "./bin/flink run" tool? Best, Robert On Fri, Feb 6,

Eclipse JDT, Java 8, lambdas

2015-02-06 Thread Nam-Luc Tran
Hello, I am trying to use Java 8 lambdas in my project and hit the following error: Exception in thread "main" org.apache.flink.api.common.functions.InvalidTypesException: The generic type parameters of 'Tuple2' are missing.  It seems that your compiler has not stored them into the .class file. 

Re: Planning Release 0.8.1

2015-02-06 Thread Márton Balassi
Found a streaming bug, Gyula fixed it. Pushing it soon to both master and branch-0.8. On Fri, Feb 6, 2015 at 11:51 AM, Robert Metzger wrote: > @Aljoscha, can you merge the backported fix to the "release-0.8" branch > when its ready? > > On Fri, Feb 6, 2015 at 11:39 AM, Aljoscha Krettek > wrote:

Re: Planning Release 0.8.1

2015-02-06 Thread Robert Metzger
@Aljoscha, can you merge the backported fix to the "release-0.8" branch when its ready? On Fri, Feb 6, 2015 at 11:39 AM, Aljoscha Krettek wrote: > I have a fix for this user-discovered bug: > > https://issues.apache.org/jira/browse/FLINK-1463?jql=project%20%3D%20FLINK%20AND%20assignee%20%3D%20cu

Re: Planning Release 0.8.1

2015-02-06 Thread Aljoscha Krettek
I have a fix for this user-discovered bug: https://issues.apache.org/jira/browse/FLINK-1463?jql=project%20%3D%20FLINK%20AND%20assignee%20%3D%20currentUser()%20AND%20resolution%20%3D%20Unresolved in this PR: https://github.com/apache/flink/pull/353 This should probably also be back-ported to 0.8.1