[jira] [Created] (FLINK-2556) Fix/Refactor pre-flight Key validation

2015-08-20 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-2556: --- Summary: Fix/Refactor pre-flight Key validation Key: FLINK-2556 URL: https://issues.apache.org/jira/browse/FLINK-2556 Project: Flink Issue Type: Bug

Re: [FAILING TEST] StateCheckpoinedITCase

2015-08-20 Thread Matthias J. Sax
Thanks for the info. Over the weeks I lost track which errors/failing/instable tests are know an which not. Should we start a wiki page or similar to collect know errors? If a test fails on a know error, it can just be ignored. This would avoid "spam" on the mailing list. Any thoughts about this?

Re: [FAILING TEST] StateCheckpoinedITCase

2015-08-20 Thread Robert Metzger
Sachin saw the error as well, as reported here: https://issues.apache.org/jira/browse/FLINK-2468 I also see it from time to time.I have a wip branch where I relaxed the constraints for the test to pass a bit. On Thu, Aug 20, 2015 at 10:05 PM, Matthias J. Sax < mj...@informatik.hu-berlin.de> wrote:

[FAILING TEST] StateCheckpoinedITCase

2015-08-20 Thread Matthias J. Sax
Error message is: > Failed tests: > StateCheckpoinedITCase>StreamFaultToleranceTestBase.runCheckpointedProgram:103->postSubmit:98 > Test inconclusive: failure occurred before first checkpoint See: https://travis-ci.org/mjsax/flink/jobs/76483093 -Matthias signature.asc Description: OpenPGP d

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Andra Lungu
Congrats Chesnay! :) On Thu, Aug 20, 2015 at 8:51 PM, Márton Balassi wrote: > Welcome Chesnay! > > On Thu, Aug 20, 2015 at 7:29 PM, Henry Saputra > wrote: > > > Welcome Chesnay! > > > > On Thu, Aug 20, 2015 at 2:18 AM, Robert Metzger > > wrote: > > > The Project Management Committee (PMC) for

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Márton Balassi
Welcome Chesnay! On Thu, Aug 20, 2015 at 7:29 PM, Henry Saputra wrote: > Welcome Chesnay! > > On Thu, Aug 20, 2015 at 2:18 AM, Robert Metzger > wrote: > > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > > Schepler to become a committer and we are pleased to announce

[jira] [Created] (FLINK-2555) Hadoop Input/Output Formats are unable to access secured HDFS clusters

2015-08-20 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2555: - Summary: Hadoop Input/Output Formats are unable to access secured HDFS clusters Key: FLINK-2555 URL: https://issues.apache.org/jira/browse/FLINK-2555 Project: Flink

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Henry Saputra
Welcome Chesnay! On Thu, Aug 20, 2015 at 2:18 AM, Robert Metzger wrote: > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > Schepler to become a committer and we are pleased to announce that they > have accepted. > > Chesnay has been very involved with the Flink project

[jira] [Created] (FLINK-2554) Add exception reporting handler

2015-08-20 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-2554: --- Summary: Add exception reporting handler Key: FLINK-2554 URL: https://issues.apache.org/jira/browse/FLINK-2554 Project: Flink Issue Type: Sub-task Co

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Paris Carbone
Awesome! Welcome Chesnay! Paris > On 20 Aug 2015, at 13:40, Chiwan Park wrote: > > Congrats Chesnay! > > Regards, > Chiwan Park > >> On Aug 20, 2015, at 7:39 PM, Gyula Fóra wrote: >> >> Welcome! :) >> >> On Thu, Aug 20, 2015 at 12:34 PM Matthias J. Sax < >> mj...@informatik.hu-berlin.de> w

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread Aljoscha Krettek
Thanks, changing the numerOfTaskSlots did it. On Thu, 20 Aug 2015 at 15:08 huangwei (G) wrote: > Hi Aljoscha, > > Once, I had the problem like yours. > What I did is to set the taskmanager.numberOfTaskSlots: 4 in > conf/flink-conf.yaml since the parallelism of > StormWordCountRemoteBySubmitter i

[jira] [Created] (FLINK-2553) Example Jars not build correctly

2015-08-20 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-2553: -- Summary: Example Jars not build correctly Key: FLINK-2553 URL: https://issues.apache.org/jira/browse/FLINK-2553 Project: Flink Issue Type: Bug

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread huangwei (G)
Hi Aljoscha, Once, I had the problem like yours. What I did is to set the taskmanager.numberOfTaskSlots: 4 in conf/flink-conf.yaml since the parallelism of StormWordCountRemoteBySubmitter is 4 by default. Maybe you can take a try. BTW: You are right. The StormWordCountLocal gets access in IDEA.

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread Matthias J. Sax
Hi, I just had a look into this. Currently, not all jars are build correctly. StormWordCountRemoteBySubmitter is build correctly though. It starts a program, sleeps for 5 seconds and tries to kill the program afterwards. The exception you see is "NotAliveException", meaning that there is no progr

[jira] [Created] (FLINK-2552) Sporadic failure of PartialConsumePipelinedResultTest

2015-08-20 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-2552: - Summary: Sporadic failure of PartialConsumePipelinedResultTest Key: FLINK-2552 URL: https://issues.apache.org/jira/browse/FLINK-2552 Project: Flink Issue T

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread Aljoscha Krettek
Hi, I'm afraid submitting the correct program also doesn't work right now. When I try to execute it I get this: bin/flink run --jarfile /Users/aljoscha/Dev/work/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/flink-storm-compatibility-examples-0.10-SNAPSHOT-W

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread Matthias J. Sax
Hi Huang, you are using the wrong jar file. The current built, does not assemble a jar file for StormWordCountLocal. You can extend pom.xml. Look at the example StormWordCountRemoteBySubmitter and the corresponding assembly file word-count-storm.xml. What you need to do is: - add a new tag sim

Re: Some new problems about flink-storm-compatibility

2015-08-20 Thread Aljoscha Krettek
Hi, could you please try running StormWordCountRemoteBySubmitter instead of StormWordCountLocal. I think the Local one only works when running inside an IDE (or executing by hand on the command line). The RemoteBySubmitter variant should be the correct one for executing on a cluster using "bin/flin

Some new problems about flink-storm-compatibility

2015-08-20 Thread huangwei (G)
Hi, I got some new problems about the storm compatibility currently. These occurred when I ran the “storm-wordcount” in the storm compatibility on a flink-0.10SNAPSHOT which I built it over a latest flink project. First, I start a local flink: $ cd bin $ ./start-local.sh Then I ran the exa

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Chiwan Park
Congrats Chesnay! Regards, Chiwan Park > On Aug 20, 2015, at 7:39 PM, Gyula Fóra wrote: > > Welcome! :) > > On Thu, Aug 20, 2015 at 12:34 PM Matthias J. Sax < > mj...@informatik.hu-berlin.de> wrote: > >> Congrats! The squirrel "army" is growing fast. :) >> >> On 08/20/2015 11:18 AM, Robert M

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Gyula Fóra
Welcome! :) On Thu, Aug 20, 2015 at 12:34 PM Matthias J. Sax < mj...@informatik.hu-berlin.de> wrote: > Congrats! The squirrel "army" is growing fast. :) > > On 08/20/2015 11:18 AM, Robert Metzger wrote: > > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > > Schepler to

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Matthias J. Sax
Congrats! The squirrel "army" is growing fast. :) On 08/20/2015 11:18 AM, Robert Metzger wrote: > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > Schepler to become a committer and we are pleased to announce that they > have accepted. > > Chesnay has been very involved

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Aljoscha Krettek
Very good. :D On Thu, 20 Aug 2015 at 11:23 Kostas Tzoumas wrote: > Welcome Chesnay! > > On Thu, Aug 20, 2015 at 11:18 AM, Robert Metzger > wrote: > > > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > > Schepler to become a committer and we are pleased to announce tha

[jira] [Created] (FLINK-2551) Fix Several Bugs in Input/Output Formats

2015-08-20 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-2551: --- Summary: Fix Several Bugs in Input/Output Formats Key: FLINK-2551 URL: https://issues.apache.org/jira/browse/FLINK-2551 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Kostas Tzoumas
Welcome Chesnay! On Thu, Aug 20, 2015 at 11:18 AM, Robert Metzger wrote: > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > Schepler to become a committer and we are pleased to announce that they > have accepted. > > Chesnay has been very involved with the Flink projec

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Till Rohrmann
Congrats and welcome on board Chesnay :-) On Thu, Aug 20, 2015 at 11:18 AM, Robert Metzger wrote: > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > Schepler to become a committer and we are pleased to announce that they > have accepted. > > Chesnay has been very invol

[ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Robert Metzger
The Project Management Committee (PMC) for Apache Flink has asked Chesnay Schepler to become a committer and we are pleased to announce that they have accepted. Chesnay has been very involved with the Flink project since its pre-ASF days. He has worked on several components including the Java API,

[jira] [Created] (FLINK-2550) Rework DataStream API

2015-08-20 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-2550: --- Summary: Rework DataStream API Key: FLINK-2550 URL: https://issues.apache.org/jira/browse/FLINK-2550 Project: Flink Issue Type: Improvement C

Re: DataStream API - What does "Beta" mean?

2015-08-20 Thread Aljoscha Krettek
Hi, on the wiki ( https://cwiki.apache.org/confluence/display/FLINK/Apache+Flink+Home) we have design documents for how we want to bring the streaming API into the final shape for the 0.10 release. The changes will change the names of some of the streaming API methods (and classes) and introduce s

Re: DataStream API - What does "Beta" mean?

2015-08-20 Thread Kostas Tzoumas
Stephan was very fast :-) I will just add that production readiness has a lot to do with the runtime code, which is largely shared by the DataSet and the Data Stream API. The beta status does not refer to production-readiness, but rather the ability to still make breaking changes. And the roadmap

Re: DataStream API - What does "Beta" mean?

2015-08-20 Thread Stephan Ewen
Hi! The beta status refers exactly to API stability and test coverage. Full status APIs, like the batch API, are very stable. For example, we have not made any breaking changes in the last two releases (except for rare corner case flags). The test coverage is also pretty good. Beta APIs (like st

Re: add some new api to the scheduler in the job manager

2015-08-20 Thread Robert Metzger
Hi Ankur, I am not aware of any up-to-date papers about the internals of Flink, but the links on this wiki page: https://cwiki.apache.org/confluence/display/FLINK/Flink+Internals contain a lot of helpful material. I'm very happy to see that you are interested in contributing and maintaining the f