[jira] [Created] (FLINK-6067) ProjectNode and FilterNode cannot merge in Batch TableAPI

jingzhang created FLINK-6067: Summary: ProjectNode and FilterNode cannot merge in Batch TableAPI Key: FLINK-6067 URL: https://issues.apache.org/jira/browse/FLINK-6067 Project: Flink Issue Type: B

[jira] [Created] (FLINK-6066) Separate logical and physical RelNode layer in Flink

Kurt Young created FLINK-6066: - Summary: Separate logical and physical RelNode layer in Flink Key: FLINK-6066 URL: https://issues.apache.org/jira/browse/FLINK-6066 Project: Flink Issue Type: Sub-

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

Thanks for the update! Just merged to 1.2.1 also: [FLINK-5962] [checkpoints] Remove scheduled cancel-task from timer queue to prevent memory leaks The remaining issue list looks good, but I would say that (5) is optional. It is not a critical production bug. On Wed, Mar 15, 2017 at 5:38 PM, Tz

Re: Flink as a Service (FaaS)

Hi jinkui, I haven't go down to that deep yet. Sounds like you have better idea what needs to be in place. Can you try to come up with a doc and may be draw some diagram so we can discuss from there? My original intention is to discuss general function gap of running lots of micro services(like t

[jira] [Created] (FLINK-6065) Make TransportClient for ES5 pluggable

Robert Metzger created FLINK-6065: - Summary: Make TransportClient for ES5 pluggable Key: FLINK-6065 URL: https://issues.apache.org/jira/browse/FLINK-6065 Project: Flink Issue Type: Improvemen

Re: [DISCUSS] Project build time and possible restructuring

@Robert - I think once we know that a separate git repo works well, and that it actually solves problems, I see no reason to not create a connectors repository later. The infrastructure changes should be identical for two or more repositories. On Wed, Mar 15, 2017 at 5:22 PM, Till Rohrmann wrote:

Re: [Discuss] Retraction for Flink Streaming

Hi Shaoxuan, thanks a lot for this proposal! Support for retractions is a super nice and important feature and will enable many more use cases for the Table API / SQL. I'm really excited to see this happening. I made a first pass over your proposal and added a few comments. I'll do another pass so

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

Thanks a lot for the updates so far everyone! From the discussion so far, the below is the still unfixed pending issues for 1.1.5 / 1.2.1 release. Since there’s only one backport for 1.1.5 left, I think having an RC for 1.1.5 near the end of this week / early next week is very promising, as bas

Re: [DISCUSS] Project build time and possible restructuring

I think it should not be at least the flink-dist but exactly the remaining flink-dist module. Otherwise we do redundant work. On Wed, Mar 15, 2017 at 5:03 PM, Robert Metzger wrote: > "flink-core" means the main repository, not the "flink-core" module. > > When doing a release, we need to build t

Re: [DISCUSS] Project build time and possible restructuring

"flink-core" means the main repository, not the "flink-core" module. When doing a release, we need to build the flink main code first, because the flink-libraries depend on that. Once the "flink-libraries" are build, we need to run the main build again (at least the flink-dist module), so that it

Re: [DISCUSS] Project build time and possible restructuring

I'm ok with point 3. Concerning point 8: Why do we have to build flink-core twice after having it built as a dependency for flink-libraries? This seems wrong to me. Cheers, Till On Wed, Mar 15, 2017 at 4:23 PM, Robert Metzger wrote: > Thank you. Running on AWS is a good idea! > Let me know if

Re: [DISCUSS] Project build time and possible restructuring

Thank you. Running on AWS is a good idea! Let me know if you (or anybody else) wants to help me with the infrastructure work! Any help is much appreciated (as I've said before, I don't really have time for doing this, but it has to be done :) ) I'm against creating two new repositories. I fear tha

[jira] [Created] (FLINK-6064) wrong BlobServer hostname in TaskExecutor

Nico Kruber created FLINK-6064: -- Summary: wrong BlobServer hostname in TaskExecutor Key: FLINK-6064 URL: https://issues.apache.org/jira/browse/FLINK-6064 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-17 Side Inputs

Hi, thanks for you input! :-) Regarding 1) I don't see the benefit of integrating windowing into the side-input logic. Windowing can happen upstream and whenever that emits new data then operator will notice because there is new input. Having windowing inside the side-input of an operator as well

Re: [DISCUSS] Project build time and possible restructuring

Robert, appreciate your kickstarting this task. We should compare the verification time with and without the listed modules. I’ll try to run this by tomorrow on AWS and on Travis. Should we maintain separate repos for flink-contrib and flink-libraries? Are you intending that we move flink-table

[jira] [Created] (FLINK-6063) HA Configuration doesn't work with Flink 1.2

Razvan created FLINK-6063: - Summary: HA Configuration doesn't work with Flink 1.2 Key: FLINK-6063 URL: https://issues.apache.org/jira/browse/FLINK-6063 Project: Flink Issue Type: Bug Compon

Re: [POLL] Who still uses Java 7 with Flink ?

I've put it also on our Twitter account: https://twitter.com/ApacheFlink/status/842015062667755521 On Wed, Mar 15, 2017 at 2:19 PM, Martin Neumann wrote: > I think this easier done in a straw poll than in an email conversation. > I created one at: http://www.strawpoll.me/12535073 > (Note that yo

Re: [DISCUSS] Project build time and possible restructuring

Thank you for looking into this Till. I think we then have to split the repositories. My main motivation for doing this is that it seems to be the only feasible way of scaling the community to allow more committers working on the libraries. I'll take care of getting things started. As the next s

Re: [POLL] Who still uses Java 7 with Flink ?

I think this easier done in a straw poll than in an email conversation. I created one at: http://www.strawpoll.me/12535073 (Note that you have multiple choices.) Though I prefer Java 8 most of the time I have to work on Java 7. A lot of the infrastructure I work on still runs Java 7, one of the

Re: [POLL] Who still uses Java 7 with Flink ?

I think this easier done in a straw poll than in an email conversation. I created one at: http://www.strawpoll.me/12535073 Though I prefer Java 8 most of the time I have to work on Java 7. A lot of the infrastructure I work on still runs Java 7, one of the companies I build a prototype for a

Re: [DISCUSS] Project build time and possible restructuring

In theory we could have a merging bot which solves the problem of the "commit window". Once the PR passes all tests and has enough +1s, the bot could do the merging and, thus, it effectively linearizes the merge process. I think the second point is actually a disadvantage because there is not such

[jira] [Created] (FLINK-6062) Re-read a Kafka topic at the reception of an event

Julien Preisner created FLINK-6062: -- Summary: Re-read a Kafka topic at the reception of an event Key: FLINK-6062 URL: https://issues.apache.org/jira/browse/FLINK-6062 Project: Flink Issue Ty

[POLL] Who still uses Java 7 with Flink ?

Hi all! I would like to get a feeling how much Java 7 is still being used among Flink users. At some point, it would be great to drop Java 7 support and make use of Java 8's new features, but first we would need to get a feeling how much Java 7 is still used. Would be happy if users on Java 7 re

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

Can we fix this issue in the 1.2.1: Flink-python tests cost too long time https://issues.apache.org/jira/browse/FLINK-5650 > 在 2017年3月15日，下午6:29，Vladislav Pernin 写道： > > I just tested in in my reproducer. It works. > > 2017-03-15 11:22 GMT+01

Re: [jira] [Created] (FLINK-6061) NPE on TypeSerializer.serialize with a RocksDBStateBackend calling state.entries in the open() function

What would be the better / less dirty way of expiring a MapState or more generally being able to access entries of a keyed state at some interval ? The currentKey has to be set in the AbstractRocksDBState. Incrementing a counter in the rich function and at some modulo, doing the job does not reall

[jira] [Created] (FLINK-6061) NPE on TypeSerializer.serialize with a RocksDBStateBackend calling state.entries in the open() function

Vladislav Pernin created FLINK-6061: --- Summary: NPE on TypeSerializer.serialize with a RocksDBStateBackend calling state.entries in the open() function Key: FLINK-6061 URL: https://issues.apache.org/jira/browse/F

[jira] [Created] (FLINK-6060) not exist class referance in the scala function annotation

shijinkui created FLINK-6060: Summary: not exist class referance in the scala function annotation Key: FLINK-6060 URL: https://issues.apache.org/jira/browse/FLINK-6060 Project: Flink Issue Type:

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

I just tested in in my reproducer. It works. 2017-03-15 11:22 GMT+01:00 Aljoscha Krettek : > I did in fact just open a PR for > > https://issues.apache.org/jira/browse/FLINK-6001 > > NPE on TumblingEventTimeWindows with ContinuousEventTimeTrigger and > > allowedLateness > > > On Tue, Mar 14, 2017

Re: Question: RemoteStreamEnvironment submit in detach mode.

I would also really like to have this feature in. I'll try and see if I can find some time to get the PR into mergeable shape. On Wed, Mar 15, 2017, at 09:11, Liangfei Su wrote: > Thanks for the information, Evgeny. > > It looks no fixed version for these feature which i think it quite common > r

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

I did in fact just open a PR for > https://issues.apache.org/jira/browse/FLINK-6001 > NPE on TumblingEventTimeWindows with ContinuousEventTimeTrigger and > allowedLateness On Tue, Mar 14, 2017, at 18:20, Vladislav Pernin wrote: > Hi, > > I would also include the following (not yet resolved) iss

[jira] [Created] (FLINK-6059) Reject DataSet and DataStream without RowTypeInformation

Fabian Hueske created FLINK-6059: Summary: Reject DataSet and DataStream without RowTypeInformation Key: FLINK-6059 URL: https://issues.apache.org/jira/browse/FLINK-6059 Project: Flink Issue

[jira] [Created] (FLINK-6058) Don't read DEFAULT_PARALLELISM from GlobalConfiguration

Aljoscha Krettek created FLINK-6058: --- Summary: Don't read DEFAULT_PARALLELISM from GlobalConfiguration Key: FLINK-6058 URL: https://issues.apache.org/jira/browse/FLINK-6058 Project: Flink I

回复：[jira] [Created] (FLINK-6057) Better default needed for num network buffers

Hi Luke,        Yes, it is really not very friendly for users to set the number of network buffers. And it may cause OOM exception if not set the proper amount for large scale job. The proper amount should be calculated automatically by framework based on the number of input and output channels

[jira] [Created] (FLINK-6057) Better default needed for num network buffers

Luke Hutchison created FLINK-6057: - Summary: Better default needed for num network buffers Key: FLINK-6057 URL: https://issues.apache.org/jira/browse/FLINK-6057 Project: Flink Issue Type: Bug

Re: Question: RemoteStreamEnvironment submit in detach mode.

Thanks for the information, Evgeny. It looks no fixed version for these feature which i think it quite common requirement. Do we have any planned release to include this fix? Thanks Ralph On Wed, Mar 15, 2017 at 3:49 PM, Evgeny Kincharov wrote: > Hi, Ralph. > > You are right, there is only o

Re: [DISCUSS] FLIP-17 Side Inputs

Hi, Aljoscha, I just go through your prototype. I like the design of the SideInputReader which can make it flexible to determine when we can get the side input. I agree that side inputs are API sugar on the top of the three components(n-ary inputs, broadcast state and input buffering), following i

[jira] [Created] (FLINK-6056) apache-rat exclude flink directory in tools

jinkuishi created FLINK-6056: Summary: apache-rat exclude flink directory in tools Key: FLINK-6056 URL: https://issues.apache.org/jira/browse/FLINK-6056 Project: Flink Issue Type: Wish

RE: Question: RemoteStreamEnvironment submit in detach mode.

Hi, Ralph. You are right, there is only one way to run streaming job at the current version 1.2 and it is blocking the main thread. We have some open issues connected with this [1], [2], and the open PR related this topic [3]. You may look for details there. Thanks for your interest in this top

Question: RemoteStreamEnvironment submit in detach mode.

Hi, I get a question when try to submit a graph to a remote stream RemoteStreamEnvironment in detach mode. >From the client API, it looks when use call execute(). User doesn't have a way to control whether run in detach mode or not. Since it's the RemoteStreamEnvironment create a StandaloneClust