[jira] [Created] (FLINK-11146) Get rid of legacy codes from ClusterClient

2018-12-12 Thread TisonKun (JIRA)
TisonKun created FLINK-11146: Summary: Get rid of legacy codes from ClusterClient Key: FLINK-11146 URL: https://issues.apache.org/jira/browse/FLINK-11146 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Flink client api enhancement for downstream project

2018-12-12 Thread Jeff Zhang
Hi Till, Thanks for the feedback. You are right that I expect better programmatic job submission/control api which could be used by downstream project. And it would benefit for the flink ecosystem. When I look at the code of flink scala-shell and sql-client (I believe they are not the core of flink

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2018-12-12 Thread Becket Qin
Hi Piotrek, 1. Regarding optimization. Sure there are many cases that the decision is hard to make. But that does not make it any easier for the users to make those decisions. I imagine 99% of the users would just naively use cache. I am not saying we can optimize in all the cases. But as long as

Re: [DISCUSS] Releasing Flink 1.7.1

2018-12-12 Thread bupt_ljy
Hi Chesnay, Thanks for these useful fixes. +1 for the release. Best, Jiayi Liao Original Message Sender:Hequn chengchenghe...@gmail.com Recipient:dev...@flink.apache.org Date:Thursday, Dec 13, 2018 09:37 Subject:Re: [DISCUSS] Releasing Flink 1.7.1 Hi Chesnay, Thanks for the efforts. +1 fo

Re: [DISCUSS] Releasing Flink 1.7.1

2018-12-12 Thread Hequn Cheng
Hi Chesnay, Thanks for the efforts. +1 for the release. It's nice to have these fixes. Best, Hequn On Wed, Dec 12, 2018 at 11:09 PM Till Rohrmann wrote: > Thanks for starting this discussion Chesnay. +1 for creating the 1.7.1 > release since it already contains very useful fixes. > > Cheers, >

Re: [DISCUSS] Releasing Flink 1.6.3

2018-12-12 Thread Jeff Zhang
+1 for a more stable flink 1.6 release. Hequn Cheng 于2018年12月13日周四 上午9:32写道: > Hi Gordon, > > Thanks for the discussion! > +1 for creating the 1.6.3 release. It would be nice if we have a new fix > for the previous release. > > Best, Hequn > > On Wed, Dec 12, 2018 at 11:05 PM Till Rohrmann > wr

Re: [DISCUSS] Releasing Flink 1.6.3

2018-12-12 Thread Hequn Cheng
Hi Gordon, Thanks for the discussion! +1 for creating the 1.6.3 release. It would be nice if we have a new fix for the previous release. Best, Hequn On Wed, Dec 12, 2018 at 11:05 PM Till Rohrmann wrote: > Thanks for starting this discussion Gordon. +1 for creating the 1.6.3 > release since we

[jira] [Created] (FLINK-11145) Fix Hadoop version handling in binary release script

2018-12-12 Thread Thomas Weise (JIRA)
Thomas Weise created FLINK-11145: Summary: Fix Hadoop version handling in binary release script Key: FLINK-11145 URL: https://issues.apache.org/jira/browse/FLINK-11145 Project: Flink Issue Ty

Re: [DISCUSS] Flink SQL DDL Design

2018-12-12 Thread Rong Rong
Thanks for the summary effort @shuyi. Sorry for jumping in the discussion so late. As of the scope of MVP, I think we might want to consider adding "table update mode" problem to it. I agree with @timo that might not be easily changed in the future if the flags has to be part of the schema/column

Re: [DISCUSS] Releasing Flink 1.7.1

2018-12-12 Thread Till Rohrmann
Thanks for starting this discussion Chesnay. +1 for creating the 1.7.1 release since it already contains very useful fixes. Cheers, Till On Wed, Dec 12, 2018 at 1:11 PM vino yang wrote: > Hi Chesnay, > > +1 to release Flink 1.7.1 > > Best, > Vino > > Chesnay Schepler 于2018年12月12日周三 下午8:04写道: >

Re: [DISCUSS] Releasing Flink 1.6.3

2018-12-12 Thread Till Rohrmann
Thanks for starting this discussion Gordon. +1 for creating the 1.6.3 release since we have already quite some good fixes in the release-1.6 branch. Cheers, Till On Wed, Dec 12, 2018 at 12:58 PM vino yang wrote: > Hi Gordon, > > +1 to release Flink 1.6.3. > > Best, > Vino > > Tzu-Li (Gordon) Ta

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2018-12-12 Thread Piotr Nowojski
Hi, Thanks for the quick answer :) Re 1. I generally agree with you, however couple of points: a) the problem with using automatic caching is bigger, because you will have to decide, how do you compare IO vs CPU costs and if you pick wrong, additional IO costs might be enormous or even can cr

[jira] [Created] (FLINK-11144) Run Tests on Java 9

2018-12-12 Thread Gary Yao (JIRA)
Gary Yao created FLINK-11144: Summary: Run Tests on Java 9 Key: FLINK-11144 URL: https://issues.apache.org/jira/browse/FLINK-11144 Project: Flink Issue Type: Sub-task Components: Tests

[jira] [Created] (FLINK-11143) AskTimeoutException is thrown during job submission and completion

2018-12-12 Thread Alex Vinnik (JIRA)
Alex Vinnik created FLINK-11143: --- Summary: AskTimeoutException is thrown during job submission and completion Key: FLINK-11143 URL: https://issues.apache.org/jira/browse/FLINK-11143 Project: Flink

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2018-12-12 Thread Becket Qin
Hi Piotrek, Not sure if you noticed, in my last email, I was proposing `CacheHandle cache()` to avoid the potential side effect due to function calls. Let's look at the disagreement in your reply one by one. 1. Optimization chances Optimization is never a trivial work. This is exactly why we s

[jira] [Created] (FLINK-11142) Undefined behavior in the conversion from DataStream/DataSet to Table

2018-12-12 Thread Yangze Guo (JIRA)
Yangze Guo created FLINK-11142: -- Summary: Undefined behavior in the conversion from DataStream/DataSet to Table Key: FLINK-11142 URL: https://issues.apache.org/jira/browse/FLINK-11142 Project: Flink

Connection leak with flink elastic Sink

2018-12-12 Thread Vijay Bhaskar
Hi We are using flink elastic sink which streams at the rate of 1000 events/sec, as described in https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html . We are observing connection leak of elastic connections. After few minutes all the open connections are exceedi

Re: [DISCUSS] Releasing Flink 1.7.1

2018-12-12 Thread vino yang
Hi Chesnay, +1 to release Flink 1.7.1 Best, Vino Chesnay Schepler 于2018年12月12日周三 下午8:04写道: > Hello, > > I propose releasing Flink 1.7.1 before the end of next week. > > Some critical issue have been identified since 1.7.0, including a state > migration issue when migrating from 1.5.3 (FLINK-11

[DISCUSS] Releasing Flink 1.7.1

2018-12-12 Thread Chesnay Schepler
Hello, I propose releasing Flink 1.7.1 before the end of next week. Some critical issue have been identified since 1.7.0, including a state migration issue when migrating from 1.5.3 (FLINK-11087) and a packaging issue in the presto-s3-filesystem (FLINK-11085, to be merged later today). Given

Re: [DISCUSS] Releasing Flink 1.6.3

2018-12-12 Thread vino yang
Hi Gordon, +1 to release Flink 1.6.3. Best, Vino Tzu-Li (Gordon) Tai 于2018年12月12日周三 下午7:35写道: > Hi Flink community, > > I would like to ask what you think about releasing 1.6.3. > > There's a few critical fixes in the 1.6 branch that users would benefit > from with another bugfix release for t

[DISCUSS] Releasing Flink 1.6.3

2018-12-12 Thread Tzu-Li (Gordon) Tai
Hi Flink community, I would like to ask what you think about releasing 1.6.3. There's a few critical fixes in the 1.6 branch that users would benefit from with another bugfix release for the series. Some are already mentioned by Till in the recent 1.5.6 release discussion thread, and their fixes

[jira] [Created] (FLINK-11141) Key generation for RocksDBMapState can theoretically be ambiguous

2018-12-12 Thread Stefan Richter (JIRA)
Stefan Richter created FLINK-11141: -- Summary: Key generation for RocksDBMapState can theoretically be ambiguous Key: FLINK-11141 URL: https://issues.apache.org/jira/browse/FLINK-11141 Project: Flink

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2018-12-12 Thread Piotr Nowojski
Hi Becket, > Regarding the chance of optimization, it might not be that rare. Some very > simple statistics could already help in many cases. For example, simply > maintaining max and min of each fields can already eliminate some > unnecessary table scan (potentially scanning the cached table) if

Re: [DISCUSS] Flink SQL DDL Design

2018-12-12 Thread Teja MVSR
Hi all, I have been following this thread and it looks interesting. Can I please be of any help, please let me know. Thanks, Teja On Wed, Dec 12, 2018, 4:31 AM Kurt Young Sounds great, thanks for the effort, Shuyi. > > Best, > Kurt > > > On Wed, Dec 12, 2018 at 5:14 PM Shuyi Chen wrote: > > >

Re: [DISCUSS] Flink SQL DDL Design

2018-12-12 Thread Kurt Young
Sounds great, thanks for the effort, Shuyi. Best, Kurt On Wed, Dec 12, 2018 at 5:14 PM Shuyi Chen wrote: > Hi all, > > I summarize the MVP based on the features that we agreed upon. For table > update mode and custom watermark strategy and ts extractor, I found there > are some discussions, so

Re: [DISCUSS] Flink SQL DDL Design

2018-12-12 Thread Shuyi Chen
Hi all, I summarize the MVP based on the features that we agreed upon. For table update mode and custom watermark strategy and ts extractor, I found there are some discussions, so I decided to leave them out for the MVP. For row/map/array data type, I think we can add it as well if everyone agrees

[jira] [Created] (FLINK-11140) Can not create a Path from an empty string while use BasePathBucketAssigner

2018-12-12 Thread Matrix42 (JIRA)
Matrix42 created FLINK-11140: Summary: Can not create a Path from an empty string while use BasePathBucketAssigner Key: FLINK-11140 URL: https://issues.apache.org/jira/browse/FLINK-11140 Project: Flink

Re: [DISCUSS] Support Interactive Programming in Flink Table API

2018-12-12 Thread Jeff Zhang
Hi Becket, Introducing CacheHandle seems too complicated. That means users have to maintain Handler properly. And since cache is just a hint for optimizer, why not just return Table itself for cache method. This hint info should be kept in Table I believe. So how about adding method cache and un

[jira] [Created] (FLINK-11139) stream non window join support state ttl

2018-12-12 Thread zhaoshijie (JIRA)
zhaoshijie created FLINK-11139: -- Summary: stream non window join support state ttl Key: FLINK-11139 URL: https://issues.apache.org/jira/browse/FLINK-11139 Project: Flink Issue Type: Improvement