[jira] [Created] (FLINK-3065) Can't cancel failing jobs

2015-11-24 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-3065: - Summary: Can't cancel failing jobs Key: FLINK-3065 URL: https://issues.apache.org/jira/browse/FLINK-3065 Project: Flink Issue Type: Bug Components: Comma

[jira] [Created] (FLINK-3066) Kafka source fails on leader change

2015-11-24 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-3066: - Summary: Kafka source fails on leader change Key: FLINK-3066 URL: https://issues.apache.org/jira/browse/FLINK-3066 Project: Flink Issue Type: Bug Compone

[jira] [Created] (FLINK-3067) Kafka source fails during checkpoint notifications with NPE

2015-11-24 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-3067: - Summary: Kafka source fails during checkpoint notifications with NPE Key: FLINK-3067 URL: https://issues.apache.org/jira/browse/FLINK-3067 Project: Flink Issue Ty

Re: Slinding Window Join (without duplicates)

2015-11-24 Thread Aljoscha Krettek
Hi, I’m not sure this is a problem. If a user specifies sliding windows then one element can (and will) end up in several windows. If these are joined then there will be multiple results. If the user does not want multiple windows then tumbling windows should be used. IMHO, this is quite straig

Re: Slinding Window Join (without duplicates)

2015-11-24 Thread Stephan Ewen
I understand Matthias' point. You want to join elements that occur within a time range of each other. In a tumbling window, you have strict boundaries and a pair of elements that arrives such that one element is before the boundary and one after, they will not join. Hence the sliding windows. Wha

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Vyacheslav Zholudev
I'm having trouble building release-0.10.1-rc1 with parameters: mvn clean install -Dhadoop.version=2.6.0.2.2.6.0-2800 -Pvendor-repos Env: maven 3, JDK 7, MacOS 10.10.5 Attached maven log when it started to produce failing tests. P.S. I had to kill the build process since it got stuck (probably

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Stephan Ewen
Hi Slava! I think the problem with your build is the file handles. It shows in various points: Exception in thread "main" java.lang.InternalError: java.io.FileNotFoundException: /Library/Java/JavaVirtualMachines/jdk1.8.0_60.jdk/Contents/Home/jre/lib/ext/localedata.jar (Too many open files in syst

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Gyula Fóra
Hi, I vote -1 for the RC due to the fact that the zookeeper deadlock issue was not completely solved. Robert could find the problem with the dependency management plugin and has opened a PR: [FLINK-3067] Enforce zkclient 0.7 for Kafka https://github.com/apache/flink/pull/1399 Cheers, Gyula Vy

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Vyacheslav Zholudev
Sorry, should have paid attention to the stacktraces. And actually I see than Java8 got picked up. Will try to fix the issue and rerun -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-Apache-Flink-0-10-1-release-0-10-0-rc1-tp9296p9318.

Re: Slinding Window Join (without duplicates)

2015-11-24 Thread Matthias J. Sax
Stephan is right. A tumbling window does not help. The last tuple of window n and the first tuple of window n+1 are "close" to each other and should be joined for example. From a SQL-like point of view this is a very common case expressed as: SELECT * FROM s1,s2 WHERE s1.key = s2.key AND |s1.ts -

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Gyula Fóra
Hi, Regarding my previous comment for the Kafka/Zookeeper issue, let's discuss if this is critical enough so we want to include it in this release or the next bugfix. I will try to further investigate the reason the job failed in the first place (we suspect broker failure) Cheers, Gyula Vyachesl

Re: Slinding Window Join (without duplicates)

2015-11-24 Thread Stephan Ewen
Since sessions are built per key, they have groups of keys that are close enough together in time. They will, however, treat the closeness transitively... On Tue, Nov 24, 2015 at 11:33 AM, Matthias J. Sax wrote: > Stephan is right. A tumbling window does not help. The last tuple of > window n an

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Robert Metzger
Hi Gyula, I'm undecided whether we should cancel the release on this or not. I understand that the issue is affecting you a lot. But I'm a bit hesitant delaying the release on this one because we are not 100% sure its really fixing the issue. You are running on a custom Flink branch anyways, right

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Maximilian Michels
I would rather not block the minor release on this issue. We don't know if we have a valid fix for it. Let's get out the minor release first and have another one when we have the fix. On Tue, Nov 24, 2015 at 11:34 AM, Gyula Fóra wrote: > Hi, > Regarding my previous comment for the Kafka/Zookeeper

[jira] [Created] (FLINK-3068) Add a Table API configuration to TableEnvironment

2015-11-24 Thread Timo Walther (JIRA)
Timo Walther created FLINK-3068: --- Summary: Add a Table API configuration to TableEnvironment Key: FLINK-3068 URL: https://issues.apache.org/jira/browse/FLINK-3068 Project: Flink Issue Type: New

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Gyula Fóra
Thanks Robert, This issue is not really a blocker for me because we are using the snapshot versions anyways. If you think this won't affect other production users too much at this stage, let's go ahead. Gyula Maximilian Michels ezt írta (időpont: 2015. nov. 24., K, 12:08): > I would rather not

[jira] [Created] (FLINK-3069) Make state materialization asynchronous

2015-11-24 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3069: --- Summary: Make state materialization asynchronous Key: FLINK-3069 URL: https://issues.apache.org/jira/browse/FLINK-3069 Project: Flink Issue Type: Improvement

Re: Null Pointer Exception in tests but only in COLLECTION mode

2015-11-24 Thread Maximilian Michels
Hi André, hi Martin, This looks very much like a bug. Martin, I would be happy if you opened a JIRA issue. Thanks, Max On Sun, Nov 22, 2015 at 12:27 PM, Martin Junghanns wrote: > Hi, > > What he meant was MultipleProgramsTestBase, not FlinkTestBase. > > I debugged this a bit. > > The NPE is thr

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Vyacheslav Zholudev
I can confirm that the build works fine when increasing a max number of opened files. Sorry for confusion. -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-Release-Apache-Flink-0-10-1-release-0-10-0-rc1-tp9296p9327.html Sent from the Apache Fl

[jira] [Created] (FLINK-3070) Create an asynchronous state handle interface

2015-11-24 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3070: --- Summary: Create an asynchronous state handle interface Key: FLINK-3070 URL: https://issues.apache.org/jira/browse/FLINK-3070 Project: Flink Issue Type: Sub-tas

[jira] [Created] (FLINK-3071) Add asynchronous materialization thread

2015-11-24 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3071: --- Summary: Add asynchronous materialization thread Key: FLINK-3071 URL: https://issues.apache.org/jira/browse/FLINK-3071 Project: Flink Issue Type: Sub-task

Re: [VOTE] Release Apache Flink 0.10.1 (release-0.10.0-rc1)

2015-11-24 Thread Stephan Ewen
@Gyula: I think it affects users, so should definitely be fixed very soon (either 0.10.1 or 0.10.2) Still checking whether Robert's current version fix solves it now, or not... On Tue, Nov 24, 2015 at 1:46 PM, Vyacheslav Zholudev < vyacheslav.zholu...@gmail.com> wrote: > I can confirm that the b

withParameters() for Streaming API

2015-11-24 Thread Timo Walther
Hi all, I want to set the Configuration of a streaming operator and access it via the open method of the RichFunction. There is no possibility to set the Configuration of the open method at the moment, right? Can I open an issue for a withParameters() equivalent for the Stremaing API? Regard

Re: withParameters() for Streaming API

2015-11-24 Thread Matthias J. Sax
We had this discussion a while ago. If I recall correctly, "withParameters()" is not encourage to be used in DataSet either. This is the thread: https://mail-archives.apache.org/mod_mbox/flink-dev/201509.mbox/%3C55EC69CD.1070003%40apache.org%3E -Matthias On 11/24/2015 02:14 PM, Timo Walther wro

Re: withParameters() for Streaming API

2015-11-24 Thread Timo Walther
Thanks for the hint Matthias. So actually the parameter of the open() method is useless? IMHO that does not look like a nice API design... We should try to keep DataSet and DataStream API in sync. Does it make sense to deprecate withParameters() for 1.0? Timo On 24.11.2015 14:31, Matthias J. S

Re: [DISCUSS] Include import statements in documentation code examples

2015-11-24 Thread Maximilian Michels
>Also, I'll not add the import statements to ALL examples, only to those >where people might copy paste them. That sounds ok to me for now. On Thu, Nov 19, 2015 at 6:38 PM, Ufuk Celebi wrote: > I think it's confusing to only have a subset of import statements provided. > But then again, the miss

Re: withParameters() for Streaming API

2015-11-24 Thread Stephan Ewen
I was also thinking of deprecating that. With that, RichFunctions should change "open(Configuration)" --> "open()". Would be heavily API breaking, so bit hesitant there... On Tue, Nov 24, 2015 at 2:48 PM, Timo Walther wrote: > Thanks for the hint Matthias. > So actually the parameter of the ope

Re: withParameters() for Streaming API

2015-11-24 Thread Till Rohrmann
If not API breaking before 1.0, then probably never? On Tue, Nov 24, 2015 at 3:06 PM, Stephan Ewen wrote: > I was also thinking of deprecating that. With that, RichFunctions should > change "open(Configuration)" --> "open()". > > Would be heavily API breaking, so bit hesitant there... > > On Tue

Re: withParameters() for Streaming API

2015-11-24 Thread Suneel Marthi
Agree with @till +1 to change this now On Tue, Nov 24, 2015 at 9:15 AM, Till Rohrmann wrote: > If not API breaking before 1.0, then probably never? > > On Tue, Nov 24, 2015 at 3:06 PM, Stephan Ewen wrote: > > > I was also thinking of deprecating that. With that, RichFunctions should > > change

Flink Master on Travis

2015-11-24 Thread Matthias J. Sax
Hi, I just observed that a couple of builds failed recently due to timeout... Is there anything we can do about this? Two recent build passed but took 1:56:25, 1:50:47 what is close to the 2h time limit, too. Just increasing the timeout seems no a good idea I guess. -Matthias signature.asc

Re: Flink Master on Travis

2015-11-24 Thread Robert Metzger
The problem is that the 2 hours timeout is from travis, we can not do anything about it. The build which is usually hitting the timeout is the one which is uploading the artifacts of the hadoop2 build to amazon s3 and the snapshot repository, for both the scala 2.10 and scala 2.11 build. I suspect

[jira] [Created] (FLINK-3072) DbStateBackendTest

2015-11-24 Thread Matthias J. Sax (JIRA)
Matthias J. Sax created FLINK-3072: -- Summary: DbStateBackendTest Key: FLINK-3072 URL: https://issues.apache.org/jira/browse/FLINK-3072 Project: Flink Issue Type: Bug Components: Te

Union a data stream with a product of itself

2015-11-24 Thread Vasiliki Kalavri
Hi squirrels, when porting the gelly streaming code from 0.9 to 0.10 today with Paris, we hit an exception in union: "*A DataStream cannot be unioned with itself*". The code raising this exception looks like this: stream.union(stream.map(...)). Taking a look into the union code, we see that it's

[jira] [Created] (FLINK-3073) Activate streaming mode by default

2015-11-24 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3073: - Summary: Activate streaming mode by default Key: FLINK-3073 URL: https://issues.apache.org/jira/browse/FLINK-3073 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-3074) Make ApplicationMaster/JobManager akka port configurable

2015-11-24 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3074: - Summary: Make ApplicationMaster/JobManager akka port configurable Key: FLINK-3074 URL: https://issues.apache.org/jira/browse/FLINK-3074 Project: Flink Issu

Re: withParameters() for Streaming API

2015-11-24 Thread Chesnay Schepler
should we do the same for IOFormats to be consistent? after FLINK-2351 they aren't used by our formats. On 24.11.2015 15:17, Suneel Marthi wrote: Agree with @till +1 to change this now On Tue, Nov 24, 2015 at 9:15 AM, Till Rohrmann wrote: If not API breaking before 1.0, then probably never?

[jira] [Created] (FLINK-3075) Rename Either creation methods to avoid name clash with projection methods

2015-11-24 Thread Gyula Fora (JIRA)
Gyula Fora created FLINK-3075: - Summary: Rename Either creation methods to avoid name clash with projection methods Key: FLINK-3075 URL: https://issues.apache.org/jira/browse/FLINK-3075 Project: Flink

Re: Either left() vs left(value)

2015-11-24 Thread Gyula Fóra
I opened a PR: https://github.com/apache/flink/pull/1402 with my suggestions, let me know what you think and we can either merge this or leave it as it is :) I would also like to hear the opinions of others about this. Vasiliki Kalavri ezt írta (időpont: 2015. nov. 23., H, 22:03): > Either is

Re: Null Pointer Exception in tests but only in COLLECTION mode

2015-11-24 Thread Martin Junghanns
Hi Max, fixed in https://github.com/apache/flink/pull/1396 Best, Martin On 24.11.2015 13:46, Maximilian Michels wrote: Hi André, hi Martin, This looks very much like a bug. Martin, I would be happy if you opened a JIRA issue. Thanks, Max On Sun, Nov 22, 2015 at 12:27 PM, Martin Junghanns w

[jira] [Created] (FLINK-3076) Display channel exchange mode in the plan visualizer

2015-11-24 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-3076: --- Summary: Display channel exchange mode in the plan visualizer Key: FLINK-3076 URL: https://issues.apache.org/jira/browse/FLINK-3076 Project: Flink Issu

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

2015-11-24 Thread Robert Metzger
Thank you Nick. I'll look into the check_compatiblilty.sh script to see which tools are used. I think finding breaking API changes immediately is a better process then reworking the APIs before a release. As you can see from my email response times (2 days since your email), I'm probably too overl

could you please add me Contributor list?

2015-11-24 Thread jun aoki
So that I can assign myself to Flink jiras? -- -jun

Re: could you please add me Contributor list?

2015-11-24 Thread Henry Saputra
Hi Jun, What is your JIRA username? But for now, you can always work on JIRA issue without assigning to yourself. Just add to comment that you are planning to work on it. - Henry On Tue, Nov 24, 2015 at 5:18 PM, jun aoki wrote: > So that I can assign myself to Flink jiras? > > -- > -jun

Re: could you please add me Contributor list?

2015-11-24 Thread jun aoki
Hi Henry, thank you for helping. My id is jaoki On Tue, Nov 24, 2015 at 9:47 PM, Henry Saputra wrote: > Hi Jun, > > What is your JIRA username? > > But for now, you can always work on JIRA issue without assigning to > yourself. Just add to comment that you are planning to work on it. > > - Henry