[jira] [Created] (FLINK-37569) Add userKeySerializer in RegisteredKeyValueStateBackendMetaInfo V2

2025-03-26 Thread Yue Ma (Jira)
Yue Ma created FLINK-37569: -- Summary: Add userKeySerializer in RegisteredKeyValueStateBackendMetaInfo V2 Key: FLINK-37569 URL: https://issues.apache.org/jira/browse/FLINK-37569 Project: Flink Issue

[jira] [Created] (FLINK-37568) Improve the method flatten for class GlobalConfiguration

2025-03-26 Thread Jiaan Geng (Jira)
Jiaan Geng created FLINK-37568: -- Summary: Improve the method flatten for class GlobalConfiguration Key: FLINK-37568 URL: https://issues.apache.org/jira/browse/FLINK-37568 Project: Flink Issue Ty

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-26 Thread Shengkai Fang
Many thanks for your reminder, Leonard. Here's the link I mentioned[1]. Best, Shengkai [1] https://github.com/apache/flink/pull/26358 Leonard Xu 于2025年3月27日周四 10:05写道: > Your link is broken, Shengkai > > Best, > Leonard > > > 2025年3月27日 10:01,Shengkai Fang 写道: > > > > Hi, All. > > > > I write

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-26 Thread Leonard Xu
Your link is broken, Shengkai Best, Leonard > 2025年3月27日 10:01,Shengkai Fang 写道: > > Hi, All. > > I write a simple demo to illustrate my idea. Hope this helps. > > Best, > Shengkai > > https://github.com/apache/flink/compare/master...fsk119:flink:example?expand=1 > > Gabor Somogyi 于2025年3

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-26 Thread Shengkai Fang
Hi, All. I write a simple demo to illustrate my idea. Hope this helps. Best, Shengkai https://github.com/apache/flink/compare/master...fsk119:flink:example?expand=1 Gabor Somogyi 于2025年3月26日周三 15:54写道: > > I'm fine with a seperate SQL connector for metadata, so maybe we could > update the FLI

Re: [VOTE] FLIP-511: Support transaction id pooling in Kafka connector

2025-03-26 Thread Efrat Levitan
+1 (non-binding) Thanks for addressing this issue! Efrat On Wed, 26 Mar 2025 at 17:35 Arvid Heise wrote: > Dear devs, > > I'd like to start the voting on FLIP-511 [1]. > Don't hesitate to ask for more details and clarifications on the > respective discussion thread [2]. > > The vote will be op

[jira] [Created] (FLINK-37567) Flink clusters not be clean up when using job cancellation as suspend mechanism

2025-03-26 Thread Alan Zhang (Jira)
Alan Zhang created FLINK-37567: -- Summary: Flink clusters not be clean up when using job cancellation as suspend mechanism Key: FLINK-37567 URL: https://issues.apache.org/jira/browse/FLINK-37567 Project:

[VOTE] FLIP-511: Support transaction id pooling in Kafka connector

2025-03-26 Thread Arvid Heise
Dear devs, I'd like to start the voting on FLIP-511 [1]. Don't hesitate to ask for more details and clarifications on the respective discussion thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes are casted. Best, Arvid [1] https://cwiki.ap

[jira] [Created] (FLINK-37566) table.exclude was invalid in binlog reading phase when scan.binlog.newly-added-table.enabled

2025-03-26 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37566: -- Summary: table.exclude was invalid in binlog reading phase when scan.binlog.newly-added-table.enabled Key: FLINK-37566 URL: https://issues.apache.org/jira/browse/FLINK-37566

RE: [DISCUSS] FLIP-511: Support transaction id pooling in Kafka connector

2025-03-26 Thread David Radley
Hi Arvid, This sounds like a good idea. I wondered: * Can we check that we have the appropriate permissions and the appropriate Kafka level(at least API key 66) to validate this option. * I am wondering what we do for the 2 errors that the Kafka API can give namely * COORDINATOR_

[VOTE] FLIP-487: Show history of rescales in Web UI for AdaptiveScheduler

2025-03-26 Thread Yuepeng Pan
Hi devs, Thanks for all the feedback on FLIP-487[1][2]. I'd like to start a vote for FLIP-487. The vote will last for at least 72 hours unless there is an objection or insufficient votes. Thanks, Yuepeng Pan [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-487%3A+Show+history

[jira] [Created] (FLINK-37565) Connector python CI failing for JDK 8

2025-03-26 Thread Hong Liang Teoh (Jira)
Hong Liang Teoh created FLINK-37565: --- Summary: Connector python CI failing for JDK 8 Key: FLINK-37565 URL: https://issues.apache.org/jira/browse/FLINK-37565 Project: Flink Issue Type: Bug

[RESULT] [VOTE] FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

2025-03-26 Thread Gyula Fóra
Hi all, The vote for FLIP-503 [1], has been concluded. The FLIP was accepted by the community with 9 votes: - Gyula Fora (binding) - Andrea Sella (non-binding) - Poorvank Bhatia (non-binding) - Marton Balassi (binding) - Matyas Orhidi (binding) - Ferenc Csaky (binding) - Thomas Weise (binding) -

Re: [VOTE] FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

2025-03-26 Thread Gyula Fóra
Thank you all! Closing this vote now On Mon, Mar 24, 2025 at 4:48 PM Mate Czagany wrote: > +1 (non-binding) > > Alexander Fedulov ezt írta (időpont: 2025. > márc. 24., H, 10:28): > > > +1 (binding) > > > > On Mon, 24 Mar 2025 at 01:34, Thomas Weise wrote: > > > > > > +1 (binding) > > > > > > >

[DISCUSS] FLIP-XXX: Checkpoint supports the Operator to customize asynchronous operation

2025-03-26 Thread jufang he
Hi devs, I would like to start a discussion about FLIP-XXX: Checkpoint supports the Operator to customize asynchronous operation [1]. In some Flink task operators, slow operations such as file uploads or data flushing may be performed during the synchronous phase of Checkpoint. Due to performan

[jira] [Created] (FLINK-37564) Rework job error fetching to use Exception history instead of JobResults

2025-03-26 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-37564: -- Summary: Rework job error fetching to use Exception history instead of JobResults Key: FLINK-37564 URL: https://issues.apache.org/jira/browse/FLINK-37564 Project: Flink

[jira] [Created] (FLINK-37561) Report system events

2025-03-26 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-37561: -- Summary: Report system events Key: FLINK-37561 URL: https://issues.apache.org/jira/browse/FLINK-37561 Project: Flink Issue Type: Sub-task Compo

[jira] [Created] (FLINK-37562) Do not check all JM replicas in observer

2025-03-26 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-37562: -- Summary: Do not check all JM replicas in observer Key: FLINK-37562 URL: https://issues.apache.org/jira/browse/FLINK-37562 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-37563) Add additional metrics for DDB connector

2025-03-26 Thread Abhi Gupta (Jira)
Abhi Gupta created FLINK-37563: -- Summary: Add additional metrics for DDB connector Key: FLINK-37563 URL: https://issues.apache.org/jira/browse/FLINK-37563 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-37560) Document events reporting

2025-03-26 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-37560: -- Summary: Document events reporting Key: FLINK-37560 URL: https://issues.apache.org/jira/browse/FLINK-37560 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-37559) Produce warn log whenever CommitterOperator#snapshotState called

2025-03-26 Thread Chanhyeong Cho (Jira)
Chanhyeong Cho created FLINK-37559: -- Summary: Produce warn log whenever CommitterOperator#snapshotState called Key: FLINK-37559 URL: https://issues.apache.org/jira/browse/FLINK-37559 Project: Flink

Re: [DISCUSS] FLIP-432: Faster Checkpoint & Recovery for Disaggregated State

2025-03-26 Thread Han Yin
Hi Yanfei, Thanks for your response! 1. Yeah. The feature has been evaluated on a Flink job with roughly 200 GB state size. The feature reduces the average checkpoint duration from dozens of seconds to within seconds. And the recovery duration is reduced from several minutes to about ten seconds

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-26 Thread Gabor Somogyi
> I'm fine with a seperate SQL connector for metadata, so maybe we could update the FLIP about our discussion? Sorry, I've forgotten this part. Yeah, no matter we choose I'm going to update the FLIP. G On Wed, Mar 26, 2025 at 8:51 AM Gabor Somogyi wrote: > Hi All, > > I've also lack of the kn

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-26 Thread Gabor Somogyi
Hi All, I've also lack of the knowledge of PTF so I've read just the motivation part: "The SQL 2016 standard introduced a way of defining custom SQL operators defined by ISO/IEC 19075-7:2021 (Part 7: Polymorphic table functions). ~200 pages define how this new kind of function can consume and pro

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-26 Thread Gyula Fóra
Hi Zakelly , Shengkai! I don't know too much about PTFs, it would be interesting to see how the usage would look in practice. Do you have some mockup/example in mind how the PTF would look for example when want to: - Simply display/aggregate whats in the metadata - Join keyed state with some me