[jira] [Created] (FLINK-30604) Remove the limitation of that parallelism decided by adaptive batch scheduler must be power-of-two

Lijie Wang created FLINK-30604: -- Summary: Remove the limitation of that parallelism decided by adaptive batch scheduler must be power-of-two Key: FLINK-30604 URL: https://issues.apache.org/jira/browse/FLINK-30604

Re: Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

Hi Dong, I think the key part is indeed your sentence that "FLIP-33 intends to standard the convention of metric name/semantic, but not the implementation." When I read FLIP-33, that's not how I interpreted it. So I've been reading back in the FLIP-33 discussion thread, there were some more clarif

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

Hi Adam, Just a side note, since your code is licensed under GPL, I believe it can't be included with Flink since GPL licenses are considered category X and can't be included with Apache-licensed projects [1]. Best regards, Martijn [1] https://www.apache.org/legal/resolved.html#category-x On M

Re: [DISCUSS] Check and add sql-connector uber jar e2e tests

Hi Weijie I personally agree with the proposal, it guarantees keeping high standards across our repos. Speaking from AWS connectors side, we have maintained e2e tests for Kinesis and Firehose sql connectors, DDB to follow. Happy to coordinate regarding any new conventions or suggestions. On 0

Re: [DISCUSS] Check and add sql-connector uber jar e2e tests

Hi Weijie, Thanks for bringing this up. Fabian, Arvid and I also discussed these things like a year ago when we thought about the externalization of connectors. I believe that in the end, you will need to have a 'Smoke Test' [1] for a connector for a DataStream API program and one for a Table/SQL

Re:Re: Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

Thanks the discussion of Dong and Martijn! Since the constructor is only marked with @deprecated now, it's compatible with the existing connectors. This proposal mainly provides support for the connectors that need override numRecordsIn report logic, which can be updated according to their r

[VOTE] FLIP-282: Introduce Delete & Update API

Hi, all. I'd like to start a vote on FLIP-282: Introduce Delete & Update API[1]. You can find the discussion on it in here[2]. The vote will last for at least 72 hours (Jan 12th at 10:00 AM GMT ) unless there is objection or insufficient votes. [1] [ https://cwiki.apache.org/confluence/pag

Re: [DISCUSS] Check and add sql-connector uber jar e2e tests

Hi Hamdy and Martijn, Thanks for quickly reply. SmokeKafkaITCase is very enlightening to me, and I totally agree with Martijn about introducing smoke test and testing Datastream/SQL at the same time. Maybe both SmokeXXXITCase or XXXSmokeTest is ok. I think we can create a umbrella ticket to revie

Re: [VOTE] FLIP-282: Introduce Delete & Update API

+1 (non binding ) thank you for driving On Mon, 9 Jan 2023 at 3:36 PM, yuxia wrote: > Hi, all. > > I'd like to start a vote on FLIP-282: Introduce Delete & Update API[1]. > You can find the discussion on it in here[2]. > > The vote will last for at least 72 hours (Jan 12th at 10:00 AM GMT ) >

[VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

Hi all, Thanks for all the feedback so far. Based on the discussion[1], we have come to a consensus, so I would like to start a vote on FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice[2]. The vote will last for at least 72 hours (Jan 12th at 10:00 GMT) unless there is an objection o

Re: [DISCUSS] FLIP-282: Introduce Delete & Update API

Hi, all. I have started voting[1] for this FLIP. If you have question, you can reply in this discuss thread. [1]: https://lists.apache.org/thread/3m91zys8dgnzyypoq88gkb2t55lqvgh1 Best regards, Yuxia - 原始邮件 - 发件人: "yuxia" 收件人: "dev" 发送时间: 星期五, 2023年 1 月 06日 下午 8:16:21 主题: Re: [DISCUSS]

[jira] [Created] (FLINK-30605) 'Streaming File Sink end-to-end test' fails with UnsupportedOperationException

Matthias Pohl created FLINK-30605: - Summary: 'Streaming File Sink end-to-end test' fails with UnsupportedOperationException Key: FLINK-30605 URL: https://issues.apache.org/jira/browse/FLINK-30605 Pro

[jira] [Created] (FLINK-30606) Bump Pulsar to 2.11.0

Yufan Sheng created FLINK-30606: --- Summary: Bump Pulsar to 2.11.0 Key: FLINK-30606 URL: https://issues.apache.org/jira/browse/FLINK-30606 Project: Flink Issue Type: Improvement Compone

[jira] [Created] (FLINK-30607) Table.to_pandas doesn't support Map type

Xuannan Su created FLINK-30607: -- Summary: Table.to_pandas doesn't support Map type Key: FLINK-30607 URL: https://issues.apache.org/jira/browse/FLINK-30607 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-280: Introduce a new explain mode to provide SQL advice

Hi, devs, I have started the vote[1] for this FLIP. If you have any questions, please don't hesitate to reply to this thread. [1] https://lists.apache.org/thread/bsgqvvs9wx1dkv7p3m9ctockh84rl11j Best, Jane On Fri, Jan 6, 2023 at 6:48 PM Jane Chan wrote: > Hi, devs, > > Thanks for all the feed

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

+0 1. Downloaded the source archive release staged at https://dist.apache.org/repos/dist/dev/flink/flink-kubernetes-operator-1.3.1-rc1/ 2. Verified the signature 3. Inspected the extracted source code for binaries 4. Compiled the source code 5. Verified license files / headers 6. Deployed to test

[Discussion] externalize Hive connector

Hi there, Following community guidance Externalized+Connector+development , We would like to initiate discussion on moving connector/hive to apache/flink-connector-hive

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

@Maximilian Michels this is a completely intentional improvement and it is required to ensure consistency for some operations within the operator logic. On Mon, Jan 9, 2023 at 4:06 PM Maximilian Michels wrote: > +0 > > 1. Downloaded the source archive release staged at > > https://dist.apache.o

Re: [Discussion] externalize Hive connector

Hi Chen, Thanks for bringing this up! I think it would be great if the Hive connector is externalized. We've already previously established [1] that it should be externalized. I believe the only reason why this hasn't been done yet is because it's blocked by https://issues.apache.org/jira/browse/F

[jira] [Created] (FLINK-30608) support rename table

Jun Zhang created FLINK-30608: - Summary: support rename table Key: FLINK-30608 URL: https://issues.apache.org/jira/browse/FLINK-30608 Project: Flink Issue Type: Improvement Reporter:

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

+1 (binding) Thanks for clarifying. I wanted to make sure this is not an unintended regression. On Mon, Jan 9, 2023 at 4:26 PM Gyula Fóra wrote: > @Maximilian Michels this is a completely intentional > improvement and it is required to ensure consistency for some operations > within the operat

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

Hi Martijn, This is not reasonable to merge Flink and OpenLogReplicator code. A simple network connection is enough and leave the projects separate. They can remain distinct. You can have client/SDK code for OpenLogReplicator (like StreamClient.cpp in the code) which would be licensed with Apac

[jira] [Created] (FLINK-30609) Add ephemeral storage to CRD

Matyas Orhidi created FLINK-30609: - Summary: Add ephemeral storage to CRD Key: FLINK-30609 URL: https://issues.apache.org/jira/browse/FLINK-30609 Project: Flink Issue Type: New Feature

Re: Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

Hi Martijn, No worries. Thanks for the discussion! Suppose we agree to deprecate the existing SourceReaderBase public constructor, it means we plan to remove this construct (which could be backward incompatible) sooner or later. I think it makes sense to at least update all the externalized conne

Re: [VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

+1 (binding) On Mon, Jan 9, 2023 at 6:19 PM Jane Chan wrote: > > Hi all, > > Thanks for all the feedback so far. > Based on the discussion[1], we have come to a consensus, so I would like to > start a vote on FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL > advice[2]. > > The vote will la

Re: [VOTE] FLIP-282: Introduce Delete & Update API

+1 binding On Mon, Jan 9, 2023 at 6:14 PM Samrat Deb wrote: > > +1 (non binding ) > > thank you for driving > > > > On Mon, 9 Jan 2023 at 3:36 PM, yuxia wrote: > > > Hi, all. > > > > I'd like to start a vote on FLIP-282: Introduce Delete & Update API[1]. > > You can find the discussion on it in

[jira] [Created] (FLINK-30610) Flink-table-runtime free for disk io in flink-core

Jingsong Lee created FLINK-30610: Summary: Flink-table-runtime free for disk io in flink-core Key: FLINK-30610 URL: https://issues.apache.org/jira/browse/FLINK-30610 Project: Flink Issue Type

Re: [DISCUSS] Check and add sql-connector uber jar e2e tests

Hi All, Thank you for all your comments. If there is no further concern, I will create the umbrella JIRA and promote the smoke test of connectors. Best regards, Weijie weijie guo 于2023年1月9日周一 18:06写道： > Hi Hamdy and Martijn, > > Thanks for quickly reply. > > SmokeKafkaITCase is very enlighte

[jira] [Created] (FLINK-30611) Expire snapshot should be reentrant

Jingsong Lee created FLINK-30611: Summary: Expire snapshot should be reentrant Key: FLINK-30611 URL: https://issues.apache.org/jira/browse/FLINK-30611 Project: Flink Issue Type: Bug

[VOTE] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

Hi all, Thanks for all the feedback so far. Based on the discussion[1], we have come to a consensus, so I would like to start a vote on FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs[2]. The vote will last for at least 72 hours (Jan 12th at 12:00 GMT) unless there is a

[jira] [Created] (FLINK-30612) ObjectArrayConverter is not thread-safe，raised by reuseArray、reuseWriter

guoxin created FLINK-30612: -- Summary: ObjectArrayConverter is not thread-safe，raised by reuseArray、reuseWriter Key: FLINK-30612 URL: https://issues.apache.org/jira/browse/FLINK-30612 Project: Flink

Re: [Discussion] externalize Hive connector

Hi Martijin, Thank you for sharing your thoughts. In my opinion, FLINK-26603 is no longer blocking FLINK-30064. That being said, we could consider keeping FLINK-26603 as follow-up work and breaking it down into smaller tasks as we proceed. [PR] https://github.com/apache/flink-connector-hive/pull/

[ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Hi everyone, On behalf of the PMC, I'm very happy to announce Lincoln Lee as a new Flink committer. Lincoln Lee has been a long-term Flink contributor since 2017. He mainly works on Flink SQL parts and drives several important FLIPs, e.g., FLIP-232 (Retry Async I/O), FLIP-234 ( Retryable Lookup J

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Impressive work in streaming semantics, well deserved! Best, Leonard > On Jan 10, 2023, at 11:52 AM, Jark Wu wrote: > > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Lincoln Lee as a new Flink > committer. > > Lincoln Lee has been a long-term F

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

Thanks for the FLIP! +1 (binding) Best, Qingsheng On Wed, Jan 4, 2023 at 10:08 AM Hang Ruan wrote: > Hi all, > > Thanks for all the feedback so far. > Based on the discussion[1], we have come to a consensus, so I would like to > start a vote on FLIP-274: Introduce metric group for > OperatorCo

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Jingsong On Tue, Jan 10, 2023 at 11:56 AM Leonard Xu wrote: > > Congratulations, Lincoln! > > Impressive work in streaming semantics, well deserved! > > > Best, > Leonard > > > > On Jan 10, 2023, at 11:52 AM, Jark Wu wrote: > > > > Hi everyone, > > > > On behalf

[jira] [Created] (FLINK-30613) Improve resolving schema compatibility -- Milestone one

Hangxiang Yu created FLINK-30613: Summary: Improve resolving schema compatibility -- Milestone one Key: FLINK-30613 URL: https://issues.apache.org/jira/browse/FLINK-30613 Project: Flink Issue

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Lijie Jingsong Li 于2023年1月10日周二 12:07写道： > Congratulations, Lincoln! > > Best, > Jingsong > > On Tue, Jan 10, 2023 at 11:56 AM Leonard Xu wrote: > > > > Congratulations, Lincoln! > > > > Impressive work in streaming semantics, well deserved! > > > > > > Best, >

[jira] [Created] (FLINK-30614) Improve resolving schema compatibility -- Milestone two

Hangxiang Yu created FLINK-30614: Summary: Improve resolving schema compatibility -- Milestone two Key: FLINK-30614 URL: https://issues.apache.org/jira/browse/FLINK-30614 Project: Flink Issue

[jira] [Created] (FLINK-30615) Refactor ReadWriteTableITCase to get rid of Managed Table

yuzelin created FLINK-30615: --- Summary: Refactor ReadWriteTableITCase to get rid of Managed Table Key: FLINK-30615 URL: https://issues.apache.org/jira/browse/FLINK-30615 Project: Flink Issue Type: S

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Yuan On Tue, Jan 10, 2023 at 12:23 PM Lijie Wang wrote: > Congratulations, Lincoln! > > Best, > Lijie > > Jingsong Li 于2023年1月10日周二 12:07写道： > > > Congratulations, Lincoln! > > > > Best, > > Jingsong > > > > On Tue, Jan 10, 2023 at 11:56 AM Leonard Xu wrote: >

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, well deserved! Best, Yanfei Yuan Mei 于2023年1月10日周二 13:16写道： > Congratulations, Lincoln! > > Best, > Yuan > > On Tue, Jan 10, 2023 at 12:23 PM Lijie Wang > wrote: > > > Congratulations, Lincoln! > > > > Best, > > Lijie > > > > Jingsong Li 于2023年1月10日周二 12:07写道： > > > > > Congr

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best regards, Weijie Lijie Wang 于2023年1月10日周二 12:24写道： > Congratulations, Lincoln! > > Best, > Lijie > > Jingsong Li 于2023年1月10日周二 12:07写道： > > > Congratulations, Lincoln! > > > > Best, > > Jingsong > > > > On Tue, Jan 10, 2023 at 11:56 AM Leonard Xu wrote: > > > >

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Regards, Dian On Tue, Jan 10, 2023 at 1:31 PM weijie guo wrote: > Congratulations, Lincoln! > > Best regards, > > Weijie > > > Lijie Wang 于2023年1月10日周二 12:24写道： > > > Congratulations, Lincoln! > > > > Best, > > Lijie > > > > Jingsong Li 于2023年1月10日周二 12:07写道： > > > >

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

Thank you all for testing this, closing the vote now. Gyula On Mon, Jan 9, 2023 at 5:27 PM Maximilian Michels wrote: > +1 (binding) > > Thanks for clarifying. I wanted to make sure this is not an unintended > regression. > > On Mon, Jan 9, 2023 at 4:26 PM Gyula Fóra wrote: > > > @Maximilian Mi

[RESULT] [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

I'm happy to announce that we have unanimously approved this release. There are 6 approving votes, 3 of which are binding: * Marton Balassi (binding) * Gyula Fora (binding) * Maximilian Mixhels (binding) * Jim Busche * Geng Biao * Hao t Chang There are no disapproving votes. Thanks everyone! Gy

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! On Tue, Jan 10, 2023 at 1:34 PM Dian Fu wrote: > Congratulations, Lincoln! > > Regards, > Dian > > On Tue, Jan 10, 2023 at 1:31 PM weijie guo > wrote: > > > Congratulations, Lincoln! > > > > Best regards, > > > > Weijie > > > > > > Lijie Wang 于2023年1月10日周二 12:24写道： >

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Hangxiang Yu 于2023年1月10日周二 13:46写道： > Congratulations, Lincoln! > > On Tue, Jan 10, 2023 at 1:34 PM Dian Fu wrote: > > > Congratulations, Lincoln! > > > > Regards, > > Dian > > > > On Tue, Jan 10, 2023 at 1:31 PM weijie guo > > wrote: > > > > > Congratulations, Lincol

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Shuiqiang Benchao Li 于2023年1月10日 周二13:48写道： > Congratulations, Lincoln! > > Hangxiang Yu 于2023年1月10日周二 13:46写道： > > > Congratulations, Lincoln! > > > > On Tue, Jan 10, 2023 at 1:34 PM Dian Fu wrote: > > > > > Congratulations, Lincoln! > > > > > > Regards, > > >

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best regards, Yuxia - 原始邮件 - 发件人: "Shuiqiang Chen" 收件人: "dev" 发送时间: 星期二, 2023年 1 月 10日 下午 1:50:50 主题: Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee Congratulations, Lincoln! Best, Shuiqiang Benchao Li 于2023年1月10日 周二13:48写道： > Congratulations, Linc

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Qingsheng On Tue, Jan 10, 2023 at 2:06 PM yuxia wrote: > Congratulations, Lincoln! > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Shuiqiang Chen" > 收件人: "dev" > 发送时间: 星期二, 2023年 1 月 10日 下午 1:50:50 > 主题: Re: [ANNOUNCE] New Apache Flink Committer - Lincol

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Big congrats, Lincoln! Best, Jane On Tue, Jan 10, 2023 at 2:10 PM Qingsheng Ren wrote: > Congratulations, Lincoln! > > Best, > Qingsheng > > On Tue, Jan 10, 2023 at 2:06 PM yuxia wrote: > > > Congratulations, Lincoln! > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Shuiqiang

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Yuxin Qingsheng Ren 于2023年1月10日周二 14:11写道： > Congratulations, Lincoln! > > Best, > Qingsheng > > On Tue, Jan 10, 2023 at 2:06 PM yuxia wrote: > > > Congratulations, Lincoln! > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Shuiqiang Chen" > >

Re:Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best regards, Wencong 在 2023-01-10 13:25:09，"Yanfei Lei" 写道： >Congratulations, well deserved! > >Best, >Yanfei > >Yuan Mei 于2023年1月10日周二 13:16写道： > >> Congratulations, Lincoln! >> >> Best, >> Yuan >> >> On Tue, Jan 10, 2023 at 12:23 PM Lijie Wang >> wrote

Re: Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best regards, Junrui Wencong Liu 于2023年1月10日周二 14:40写道： > Congratulations, Lincoln! > > Best regards, > Wencong > > > > > > > > > > > > > > > 在 2023-01-10 13:25:09，"Yanfei Lei" 写道： > >Congratulations, well deserved! > > > >Best, > >Yanfei > > > >Yuan Mei 于2023年1月10日周

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congrats, Lincoln! Best, Biao Geng 获取 Outlook for iOS 发件人: Wencong Liu 发送时间: Tuesday, January 10, 2023 2:39:47 PM 收件人: dev@flink.apache.org 主题: Re:Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee Congratulations, Lincoln! Best rega

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Guowei On Tue, Jan 10, 2023 at 2:57 PM Biao Geng wrote: > Congrats, Lincoln! > Best, > Biao Geng > > 获取 Outlook for iOS > > 发件人: Wencong Liu > 发送时间: Tuesday, January 10, 2023 2:39:47 PM > 收件人: dev@flink.ap

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Best, Hang Biao Geng 于2023年1月10日周二 14:57写道： > Congrats, Lincoln! > Best, > Biao Geng > > 获取 Outlook for iOS > > 发件人: Wencong Liu > 发送时间: Tuesday, January 10, 2023 2:39:47 PM > 收件人: dev@flink.apache.org > 主题: Re:

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

Congratulations, Lincoln! Thanks, Biao /'bɪ.aʊ/ On Tue, 10 Jan 2023 at 14:59, Hang Ruan wrote: > Congratulations, Lincoln! > > Best, > Hang > > Biao Geng 于2023年1月10日周二 14:57写道： > > > Congrats, Lincoln! > > Best, > > Biao Geng > > > > 获取 Outlook for iOS > >

Re: [VOTE] FLIP-280: Introduce EXPLAIN PLAN_ADVICE to provide SQL advice

+1 (binding) Best, Godfrey Jingsong Li 于2023年1月10日周二 09:56写道： > > +1 (binding) > > On Mon, Jan 9, 2023 at 6:19 PM Jane Chan wrote: > > > > Hi all, > > > > Thanks for all the feedback so far. > > Based on the discussion[1], we have come to a consensus, so I would like to > > start a vote on FLIP

[jira] [Created] (FLINK-30616) Don't support batchMessageId when restore from checkpoint

songv created FLINK-30616: - Summary: Don't support batchMessageId when restore from checkpoint Key: FLINK-30616 URL: https://issues.apache.org/jira/browse/FLINK-30616 Project: Flink Issue Type: Bug