[jira] [Created] (FLINK-14967) Add a utility for creating data types via reflection

2019-11-26 Thread Timo Walther (Jira)
Timo Walther created FLINK-14967: Summary: Add a utility for creating data types via reflection Key: FLINK-14967 URL: https://issues.apache.org/jira/browse/FLINK-14967 Project: Flink Issue Ty

[jira] [Created] (FLINK-14966) one error in document

2019-11-26 Thread yuzhemin (Jira)
yuzhemin created FLINK-14966: Summary: one error in document Key: FLINK-14966 URL: https://issues.apache.org/jira/browse/FLINK-14966 Project: Flink Issue Type: Improvement Components:

[jira] [Created] (FLINK-14965) CatalogTableStatistics UNKNOWN should be consistent with TableStats UNKNOWN

2019-11-26 Thread Terry Wang (Jira)
Terry Wang created FLINK-14965: -- Summary: CatalogTableStatistics UNKNOWN should be consistent with TableStats UNKNOWN Key: FLINK-14965 URL: https://issues.apache.org/jira/browse/FLINK-14965 Project: Flin

Re: [DISCUSS] Multi-topics consuming from KafkaTableSource

2019-11-26 Thread Gyula Fóra
Hi all, I think the same reasoning applies here as with the regular Kafka consumer sources. Those support multiple topics, even topic patterns assuming the same schema. Some times you have so many topics that you can't create a source for each or as with the pattern the topics might be changing

[jira] [Created] (FLINK-14964) Support configure remote flink jar

2019-11-26 Thread Zili Chen (Jira)
Zili Chen created FLINK-14964: - Summary: Support configure remote flink jar Key: FLINK-14964 URL: https://issues.apache.org/jira/browse/FLINK-14964 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Multi-topics consuming from KafkaTableSource

2019-11-26 Thread Jingsong Li
Hi leo: Can you share why KafkaSource supports multiple topics? Why can't we use multiple sources and then union them? Best, Jingsong Lee On Wed, Nov 27, 2019 at 8:19 AM leo wrote: > Hi, > > As Flink does not own the data, so it can not forbid this. It's the > responsibility of user to deseria

[jira] [Created] (FLINK-14963) Massive Duplicated Useless Oss Message in jobmanager.log

2019-11-26 Thread Eric Lee (Jira)
Eric Lee created FLINK-14963: Summary: Massive Duplicated Useless Oss Message in jobmanager.log Key: FLINK-14963 URL: https://issues.apache.org/jira/browse/FLINK-14963 Project: Flink Issue Type:

[jira] [Created] (FLINK-14962) FindBugs warnings: Inefficient use of keySet iterator instead of entrySet iterator

2019-11-26 Thread Dezhi Cai (Jira)
Dezhi Cai created FLINK-14962: - Summary: FindBugs warnings: Inefficient use of keySet iterator instead of entrySet iterator Key: FLINK-14962 URL: https://issues.apache.org/jira/browse/FLINK-14962 Project:

Re: [DISCUSS] Make Managed Memory always off-heap (Adjustment to FLIP-49)

2019-11-26 Thread Jingsong Li
Hi Stephan, +1 to default have off-heap managed memory. >From the perspective of batch, In our long-term performance test and online practice: - There is no significant difference in performance between heap and off-heap memory. If it is a heap object, the JVM has many opportunities to optimize i

[jira] [Created] (FLINK-14961) Join with a LookupableTableSource: Expected results are inconsistent when using SQL expressions or UDF

2019-11-26 Thread Rockey Cui (Jira)
Rockey Cui created FLINK-14961: -- Summary: Join with a LookupableTableSource: Expected results are inconsistent when using SQL expressions or UDF Key: FLINK-14961 URL: https://issues.apache.org/jira/browse/FLINK-14961

Re: [DISCUSS] Releasing Flink 1.8.3

2019-11-26 Thread Hequn Cheng
Hi Jincheng, Thanks a lot for your timely help. I'm on my way to the release. Best, Hequn On Wed, Nov 27, 2019 at 7:36 AM jincheng sun wrote: > Hi Hequn, > > Thank you for your great job! Looking forward the first RC of 1.8.3 ! > BTW: The version of 1.8.4 already created here: > https://issues

Re: [PROPOSAL] Add a JIRA component "Deployment / Kubernetes"

2019-11-26 Thread tison
Oops I just notice that there is already such a component...Sorry for the noise. tison 于2019年11月27日周三 上午10:33写道: > Hi devs, > > Recently Flink on k8s effort[1] has been revived. When working through the > commit process I > notice that these tickets are tracked as "Runtime / Coordination". > Al

[PROPOSAL] Add a JIRA component "Deployment / Kubernetes"

2019-11-26 Thread tison
Hi devs, Recently Flink on k8s effort[1] has been revived. When working through the commit process I notice that these tickets are tracked as "Runtime / Coordination". Although it makes sense in some directions, I'd prefer to introduce a dedicate "Deployment / Kubernetes" component for tracking th

[jira] [Created] (FLINK-14960) Dependency shading of table modules test fails on Travis

2019-11-26 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-14960: --- Summary: Dependency shading of table modules test fails on Travis Key: FLINK-14960 URL: https://issues.apache.org/jira/browse/FLINK-14960 Project: Flink Issue

[DISCUSS] Make Managed Memory always off-heap (Adjustment to FLIP-49)

2019-11-26 Thread Stephan Ewen
Hi all! Yesterday, some of the people involved in FLIP-49 had a long discussion about managed memory in Flink. Particularly, the fact that we have managed memory either on heap or off heap and that FLIP-49 introduced having both of these types of memory at the same time. ==> What we want to sugge

[jira] [Created] (FLINK-14959) Support precision of LocalZonedTimestampType in blink planner

2019-11-26 Thread Zhenghua Gao (Jira)
Zhenghua Gao created FLINK-14959: Summary: Support precision of LocalZonedTimestampType in blink planner Key: FLINK-14959 URL: https://issues.apache.org/jira/browse/FLINK-14959 Project: Flink

[jira] [Created] (FLINK-14958) ProgramTargetDescriptor#jobID possibly can be of type JobID instead of String

2019-11-26 Thread Zili Chen (Jira)
Zili Chen created FLINK-14958: - Summary: ProgramTargetDescriptor#jobID possibly can be of type JobID instead of String Key: FLINK-14958 URL: https://issues.apache.org/jira/browse/FLINK-14958 Project: Flin

[jira] [Created] (FLINK-14957) Remove no power option -yst

2019-11-26 Thread Zili Chen (Jira)
Zili Chen created FLINK-14957: - Summary: Remove no power option -yst Key: FLINK-14957 URL: https://issues.apache.org/jira/browse/FLINK-14957 Project: Flink Issue Type: Improvement Compo

[jira] [Created] (FLINK-14956) MemoryMappedBoundedData Compressed Buffer Slicer

2019-11-26 Thread Nicholas Jiang (Jira)
Nicholas Jiang created FLINK-14956: -- Summary: MemoryMappedBoundedData Compressed Buffer Slicer Key: FLINK-14956 URL: https://issues.apache.org/jira/browse/FLINK-14956 Project: Flink Issue Ty

Re: [DISCUSS] Multi-topics consuming from KafkaTableSource

2019-11-26 Thread leo
Hi, As Flink does not own the data, so it can not forbid this. It's the responsibility of user to deserialize messages from these topics with custom schema deser. On 11/26/19 10:55 AM, Jun Zhang wrote: If the data formats of the Multi-topics are different, what do you plan to do? On 11/

Re: [DISCUSS] Releasing Flink 1.8.3

2019-11-26 Thread jincheng sun
Hi Hequn, Thank you for your great job! Looking forward the first RC of 1.8.3 ! BTW: The version of 1.8.4 already created here: https://issues.apache.org/jira/projects/FLINK/versions/12346552 Best, Jincheng Hequn Cheng 于2019年11月26日周二 下午8:18写道: > Hi all, > > I would like to share with you that

[jira] [Created] (FLINK-14955) Not able to write to swift via StreamingFileSink.forBulkFormat

2019-11-26 Thread Simya Jose (Jira)
Simya Jose created FLINK-14955: -- Summary: Not able to write to swift via StreamingFileSink.forBulkFormat Key: FLINK-14955 URL: https://issues.apache.org/jira/browse/FLINK-14955 Project: Flink I

[jira] [Created] (FLINK-14954) Consider providing an official OpenAPI specification of REST Monitoring API

2019-11-26 Thread Jira
Michaël Melchiore created FLINK-14954: - Summary: Consider providing an official OpenAPI specification of REST Monitoring API Key: FLINK-14954 URL: https://issues.apache.org/jira/browse/FLINK-14954

Re: SQL for Avro GenericRecords on Parquet

2019-11-26 Thread Peter Huang
Hi Hanan, After investigating the issue by using the test case you provided, I think there is a big in it. Currently, the parquet predicts push down use the predicate literal type to construct the FilterPredicate. The issue happens when the data type of value in predicate inferred from SQL doesn't

[jira] [Created] (FLINK-14953) Parquet table source should use schema type to build FilterPredicate

2019-11-26 Thread Zhenqiu Huang (Jira)
Zhenqiu Huang created FLINK-14953: - Summary: Parquet table source should use schema type to build FilterPredicate Key: FLINK-14953 URL: https://issues.apache.org/jira/browse/FLINK-14953 Project: Flink

Re: set job level TTL

2019-11-26 Thread Andrey Zagrebin
Hi Chen, if I understand correctly, your question is not about the state with TTL, correct me if I am wrong. We could consider adding this feature to Flink: run x time => save point => shutdown job but it seems to me that it is something quite application specific, not sure how high priority it w

[jira] [Created] (FLINK-14952) Yarn containers can exceed physical memory limits when using BoundedBlockingSubpartition.

2019-11-26 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-14952: -- Summary: Yarn containers can exceed physical memory limits when using BoundedBlockingSubpartition. Key: FLINK-14952 URL: https://issues.apache.org/jira/browse/FLINK-14952

[jira] [Created] (FLINK-14951) State TTL backend end-to-end test fail when taskManager has multiple slot

2019-11-26 Thread Yangze Guo (Jira)
Yangze Guo created FLINK-14951: -- Summary: State TTL backend end-to-end test fail when taskManager has multiple slot Key: FLINK-14951 URL: https://issues.apache.org/jira/browse/FLINK-14951 Project: Flink

Re: [DISCUSS] Releasing Flink 1.8.3

2019-11-26 Thread Hequn Cheng
Hi all, I would like to share with you that all blockers are resolved now. If there are no more critical issues, I will create the first RC tomorrow and vote on it directly. Hope everything goes well! Thank you all for the help of fixing, reviewing, driving and discussions! Best, Hequn On Tue,

[jira] [Created] (FLINK-14950) Support getKey in WindowOperator.Context

2019-11-26 Thread Jiayi Liao (Jira)
Jiayi Liao created FLINK-14950: -- Summary: Support getKey in WindowOperator.Context Key: FLINK-14950 URL: https://issues.apache.org/jira/browse/FLINK-14950 Project: Flink Issue Type: Improvement