Re: Use of loading flink-conf.yaml in Flink-sql client

2019-09-15 Thread Dipanjan Mazumder
Hi,    i was reading through the Flink docs, and i have got to an understanding that each application will have its own instance of Jobamanager and TaskManagers and so every application will have to have a initial configuration for defining the application topology to be drawn in the flink clus

[ANNOUNCE] Weekly Community Update 2019/37

2019-09-15 Thread Konstantin Knauf
Dear community, happy to share this week's community update with the release of Flink 1.8.2, more work in the area of dynamic resource management, three proposals in the SQL space and a bit more. Flink Development == * [releases] *Flink 1.8.2* has been released. [1] * [resource mana

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-15 Thread wenlong.lwl
Hi, Xintong, thanks for the great proposal. big +1 for the feature! It is something like mapreduce-1.0 to mapreduce-2.0. I like the design on the whole. One point may need to be included in the proposal:How we deal with slot share group and dynamic slot allocation? It can be quite different with d

Re: [DISCUSS] FLIP-56: Dynamic Slot Allocation

2019-09-15 Thread Xintong Song
Thanks for the comments, Till and Wenlong. @Wenlong Regarding slot sharing, the general idea is to request a slot with resources for tasks of the entire slot sharing group. Details can be found in FLIP-53 [1], regarding how to decide the slot sharing groups and how to manage task resources within

[jira] [Created] (FLINK-14078) Introduce more JDBCDialect implementations

2019-09-15 Thread Canbin Zheng (Jira)
Canbin Zheng created FLINK-14078: Summary: Introduce more JDBCDialect implementations Key: FLINK-14078 URL: https://issues.apache.org/jira/browse/FLINK-14078 Project: Flink Issue Type: New Fe

[jira] [Created] (FLINK-14079) Full data type support in planner

2019-09-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-14079: Summary: Full data type support in planner Key: FLINK-14079 URL: https://issues.apache.org/jira/browse/FLINK-14079 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-14081) Support precision of TimeType

2019-09-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-14081: Summary: Support precision of TimeType Key: FLINK-14081 URL: https://issues.apache.org/jira/browse/FLINK-14081 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14080) Support precision of TimestampType

2019-09-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-14080: Summary: Support precision of TimestampType Key: FLINK-14080 URL: https://issues.apache.org/jira/browse/FLINK-14080 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14082) Support precision of DayTimeIntervalType and YearMonthIntervalType

2019-09-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-14082: Summary: Support precision of DayTimeIntervalType and YearMonthIntervalType Key: FLINK-14082 URL: https://issues.apache.org/jira/browse/FLINK-14082 Project: Flink

[jira] [Created] (FLINK-14083) Support StructuredType

2019-09-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-14083: Summary: Support StructuredType Key: FLINK-14083 URL: https://issues.apache.org/jira/browse/FLINK-14083 Project: Flink Issue Type: Sub-task Compone

[jira] [Created] (FLINK-14084) Support ZonedTimestampType

2019-09-15 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-14084: Summary: Support ZonedTimestampType Key: FLINK-14084 URL: https://issues.apache.org/jira/browse/FLINK-14084 Project: Flink Issue Type: Sub-task Com

Re: How stable is FlinkSQL.

2019-09-15 Thread srikanth flink
Hi Stephan, - I'm using streaming SQL - 1.9 version as recommended by Flink to have a updated stable version. Thanks Srikanth On Fri, Sep 13, 2019 at 8:30 PM Stephan Ewen wrote: > Can you share some more details? > > - are you running batch SQL or streaming SQL > - are you running the orig

Does Flink has Retention policy?

2019-09-15 Thread srikanth flink
Hi there, I came across Flink and FlinkSQL and using FlinkSQL for stream processing. Flink runs as 3 node cluster with embedded Zookeeper, given heap 80GB on each. I came across few issues and would like to get some clarification. - Job1: Using Flink(java) to read and flatten my JSON and writ