Re: recover from svaepoint

2021-06-02 Thread Till Rohrmann
Forwarding 周瑞's message to a duplicate thread: After our analysis, we found a bug in the org.apache.flink.streaming.connectors.kafka.internals.FlinkKafkaInternalProducer.resumeTransaction method The analysis process is as follows: org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkF

[jira] [Created] (FLINK-22862) Support profiling in PyFlink

2021-06-02 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-22862: Summary: Support profiling in PyFlink Key: FLINK-22862 URL: https://issues.apache.org/jira/browse/FLINK-22862 Project: Flink Issue Type: Improvement

Re: [Discuss] Planning Flink 1.14

2021-06-02 Thread Kurt Young
Thanks for bringing this up. I have one thought about the release period. In a short word: shall we try to extend the release period for 1 month? There are a couple of reasons why I want to bring up this proposal. 1) I observed that lots of users are actually far behind the current Flink version

[jira] [Created] (FLINK-22861) TIMESTAMPADD + timestamp_ltz type throws CodeGenException when comparing with timestamp type

2021-06-02 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-22861: --- Summary: TIMESTAMPADD + timestamp_ltz type throws CodeGenException when comparing with timestamp type Key: FLINK-22861 URL: https://issues.apache.org/jira/browse/FLINK-22861

Re: Integrating new connector with Flink SQL.

2021-06-02 Thread Jingsong Li
Hi santhosh, 1.I recommend you use the new source with ScanTablesource. 2.You can use `org.apache.flink.table.connector.source.SourceProvider` to integrate to ScanTablesource. (Introduced in 1.12) 3.You can just implement a new source, this one can be used by both Flink DataStream and Flink SQL.

[jira] [Created] (FLINK-22860) Supplementary 'HELP' command prompt message for SQL-Cli.

2021-06-02 Thread Roc Marshal (Jira)
Roc Marshal created FLINK-22860: --- Summary: Supplementary 'HELP' command prompt message for SQL-Cli. Key: FLINK-22860 URL: https://issues.apache.org/jira/browse/FLINK-22860 Project: Flink Issue

[jira] [Created] (FLINK-22859) Wordcount on Docker test (custom fs plugin) fails due to output hash mismatch

2021-06-02 Thread Xintong Song (Jira)
Xintong Song created FLINK-22859: Summary: Wordcount on Docker test (custom fs plugin) fails due to output hash mismatch Key: FLINK-22859 URL: https://issues.apache.org/jira/browse/FLINK-22859 Project

RE: [VOTE] Watermark propagation with Sink API

2021-06-02 Thread Zhou, Brian
+1 (non-binding) Thanks Eron, looking forward to seeing this feature soon. Thanks, Brian -Original Message- From: Arvid Heise Sent: Wednesday, June 2, 2021 15:44 To: dev Subject: Re: [VOTE] Watermark propagation with Sink API [EXTERNAL EMAIL] +1 (binding) Thanks Eron for driving t

Integrating new connector with Flink SQL.

2021-06-02 Thread santhosh venkat
Hi, Please correct me if I'm wrong anywhere. I'm just new to Flink and trying to navigate the landscape. Within my company, currently we're trying to develop a connector for our internal change data capture system(brooklin) for flink. We are planning to use Flink SQL as a primary API to build str

[jira] [Created] (FLINK-22858) avro-confluent doesn't support confluent schema registry that has security enabled

2021-06-02 Thread TAO XIAO (Jira)
TAO XIAO created FLINK-22858: Summary: avro-confluent doesn't support confluent schema registry that has security enabled Key: FLINK-22858 URL: https://issues.apache.org/jira/browse/FLINK-22858 Project: F

[jira] [Created] (FLINK-22857) Add possibility to call built-in functions in SpecializedFunction

2021-06-02 Thread Timo Walther (Jira)
Timo Walther created FLINK-22857: Summary: Add possibility to call built-in functions in SpecializedFunction Key: FLINK-22857 URL: https://issues.apache.org/jira/browse/FLINK-22857 Project: Flink

Re: recover from svaepoint

2021-06-02 Thread Piotr Nowojski
Hi, I think there is no generic way. If this error has happened indeed after starting a second job from the same savepoint, or something like that, user can change Sink's operator UID. If this is an issue of intentional recovery from an earlier checkpoint/savepoint, maybe `FlinkKafkaProducer#setL

[jira] [Created] (FLINK-22856) Move our Azure pipelines away from Ubuntu 16.04 by September

2021-06-02 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-22856: -- Summary: Move our Azure pipelines away from Ubuntu 16.04 by September Key: FLINK-22856 URL: https://issues.apache.org/jira/browse/FLINK-22856 Project: Flink

Re: [Discuss] Planning Flink 1.14

2021-06-02 Thread JING ZHANG
Hi Dawid, Joe & Xintong, Thanks for starting the discussion. I would like to polish Window TVFs[1][2] which is a popular feature in SQL introduced in 1.13. The detailed items are as follows. 1. Add more computations based on Window TVF * Window Join (which is already merged in master branch)

[jira] [Created] (FLINK-22855) Translate the 'Overview of Python API' page into Chinese.

2021-06-02 Thread Roc Marshal (Jira)
Roc Marshal created FLINK-22855: --- Summary: Translate the 'Overview of Python API' page into Chinese. Key: FLINK-22855 URL: https://issues.apache.org/jira/browse/FLINK-22855 Project: Flink Issue

[jira] [Created] (FLINK-22854) Translate 'Apache Flink Documentation' index page to Chinese

2021-06-02 Thread Roc Marshal (Jira)
Roc Marshal created FLINK-22854: --- Summary: Translate 'Apache Flink Documentation' index page to Chinese Key: FLINK-22854 URL: https://issues.apache.org/jira/browse/FLINK-22854 Project: Flink I

[jira] [Created] (FLINK-22853) FLinkSql聚合函数max/min/sum返回结果重复

2021-06-02 Thread Raypon Wang (Jira)
Raypon Wang created FLINK-22853: --- Summary: FLinkSql聚合函数max/min/sum返回结果重复 Key: FLINK-22853 URL: https://issues.apache.org/jira/browse/FLINK-22853 Project: Flink Issue Type: Bug Compone

[Discuss] Planning Flink 1.14

2021-06-02 Thread Xintong Song
Hi all, As 1.13 has been released for a while, I think it is a good time to start planning for the 1.14 release cycle. - Release managers: This time we'd like to have a team of 3 release managers. Dawid, Joe and I would like to volunteer for it. What do you think about it? - Timeline: According

[jira] [Created] (FLINK-22852) Add SPNEGO authentication support

2021-06-02 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-22852: - Summary: Add SPNEGO authentication support Key: FLINK-22852 URL: https://issues.apache.org/jira/browse/FLINK-22852 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-22851) Add basic authentication support

2021-06-02 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-22851: - Summary: Add basic authentication support Key: FLINK-22851 URL: https://issues.apache.org/jira/browse/FLINK-22851 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Dashboard/HistoryServer authentication

2021-06-02 Thread Gabor Somogyi
Hi team, Happy to be here and hope I can provide quality additions in the future. Thank you all for helpful the suggestions! Considering them the FLIP has been modified and the work continues on the already existing Jira. BR, G On Wed, Jun 2, 2021 at 11:23 AM Márton Balassi wrote: > Thanks,

[jira] [Created] (FLINK-22850) org.apache.flink.configuration.ConfigOptions.defaultValue() and noDefaultValue() is Deprecated.

2021-06-02 Thread lixiaobao (Jira)
lixiaobao created FLINK-22850: - Summary: org.apache.flink.configuration.ConfigOptions.defaultValue() and noDefaultValue() is Deprecated. Key: FLINK-22850 URL: https://issues.apache.org/jira/browse/FLINK-22850

Re: [DISCUSS] Dashboard/HistoryServer authentication

2021-06-02 Thread Márton Balassi
Thanks, Chesney - I totally missed that. Answered on the ticket too, let us continue there then. Till, I agree that we should keep this codepath as slim as possible. It is an important design decision that we aim to keep the list of authentication protocols to a minimum. We believe that this shoul

Re: [VOTE] Watermark propagation with Sink API

2021-06-02 Thread Arvid Heise
+1 (binding) Thanks Eron for driving this effort; it will open up new exciting use cases. On Tue, Jun 1, 2021 at 7:17 PM Eron Wright wrote: > After some good discussion about how to enhance the Sink API to process > watermarks, I believe we're ready to proceed with a vote. Voting will be > ope

[jira] [Created] (FLINK-22849) Drop remaining usages of legacy planner in E2E tests and Python

2021-06-02 Thread Timo Walther (Jira)
Timo Walther created FLINK-22849: Summary: Drop remaining usages of legacy planner in E2E tests and Python Key: FLINK-22849 URL: https://issues.apache.org/jira/browse/FLINK-22849 Project: Flink

[jira] [Created] (FLINK-22848) Deprecate unquoted options for SET / RESET

2021-06-02 Thread Jira
Ingo Bürk created FLINK-22848: - Summary: Deprecate unquoted options for SET / RESET Key: FLINK-22848 URL: https://issues.apache.org/jira/browse/FLINK-22848 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-22847) SET should print options quoted

2021-06-02 Thread Jira
Ingo Bürk created FLINK-22847: - Summary: SET should print options quoted Key: FLINK-22847 URL: https://issues.apache.org/jira/browse/FLINK-22847 Project: Flink Issue Type: Improvement C

[jira] [Created] (FLINK-22846) Add SPNEGO authentication support

2021-06-02 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-22846: - Summary: Add SPNEGO authentication support Key: FLINK-22846 URL: https://issues.apache.org/jira/browse/FLINK-22846 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-22845) Add Basic authentication support

2021-06-02 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-22845: - Summary: Add Basic authentication support Key: FLINK-22845 URL: https://issues.apache.org/jira/browse/FLINK-22845 Project: Flink Issue Type: Sub-task