[jira] [Created] (FLINK-34947) Do not scale down JM in Orphan deletion propagation and reduce timeout

2024-03-26 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-34947: -- Summary: Do not scale down JM in Orphan deletion propagation and reduce timeout Key: FLINK-34947 URL: https://issues.apache.org/jira/browse/FLINK-34947 Project: Flink

[DISCUSS] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-03-26 Thread Rui Fan
Hi flink developers, I'd like to start a discussion to discuss FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI[1]. Currently, the jobType has 2 types in Flink: STREAMING and BATCH. They work on completely different principles, such as: scheduler, shuffle, join, etc. These diff

[jira] [Created] (FLINK-34946) Flink Website Menu Adjustment

2024-03-26 Thread Zhongqiang Gong (Jira)
Zhongqiang Gong created FLINK-34946: --- Summary: Flink Website Menu Adjustment Key: FLINK-34946 URL: https://issues.apache.org/jira/browse/FLINK-34946 Project: Flink Issue Type: Improvement

Re: Re:Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-26 Thread Yun Tang
Congratulations, cool works! Best Yun Tang From: Feifan Wang Sent: Wednesday, March 27, 2024 10:27 To: dev@flink.apache.org Subject: Re:Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed Congratulations ! 在 2024-03-22 12:04:39,"Hangxiang Yu"

Re: [DISCUSS] FLIP-423 ~FLIP-428: Introduce Disaggregated State Storage and Management in Flink 2.0

2024-03-26 Thread Zakelly Lan
Hi devs, It seems there is no more concern or suggestion for a while. We'd like to start voting recently. Best, Zakelly On Wed, Mar 27, 2024 at 11:46 AM Zakelly Lan wrote: > Hi everyone, > > Piotr and I had a long discussion about the checkpoint duration issue. We > think that the lambda seri

[jira] [Created] (FLINK-34945) Support recover shuffle descriptor and partition metrics from tiered storage

2024-03-26 Thread Junrui Li (Jira)
Junrui Li created FLINK-34945: - Summary: Support recover shuffle descriptor and partition metrics from tiered storage Key: FLINK-34945 URL: https://issues.apache.org/jira/browse/FLINK-34945 Project: Flink

Re: [DISCUSS] FLIP-423 ~FLIP-428: Introduce Disaggregated State Storage and Management in Flink 2.0

2024-03-26 Thread Zakelly Lan
Hi everyone, Piotr and I had a long discussion about the checkpoint duration issue. We think that the lambda serialization approach I proposed in last mail may bring in more issues, the most serious one is that users may not be able to modify their code in serialized lambda to perform a bug fix.

[jira] [Created] (FLINK-34944) Use Incremental Source Framework in Flink CDC OceanBase Source Connector

2024-03-26 Thread He Wang (Jira)
He Wang created FLINK-34944: --- Summary: Use Incremental Source Framework in Flink CDC OceanBase Source Connector Key: FLINK-34944 URL: https://issues.apache.org/jira/browse/FLINK-34944 Project: Flink

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-26 Thread Jark Wu
Thanks for the PoC and updating, The final syntax looks good to me, at least it is a nice and concise first step. SELECT f1, f2, label FROM ML_PREDICT( input => `my_data`, model => `my_cat`.`my_db`.`classifier_model`, args => DESCRIPTOR(f1, f2)); Besides, what built-in models w

Re: Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-26 Thread Jinzhong Li
Congratulations! Thanks for the great work! Best, Jinzhong On Wed, Mar 27, 2024 at 10:28 AM Feifan Wang wrote: > Congratulations ! > > > 在 2024-03-22 12:04:39,"Hangxiang Yu" 写道: > >Congratulations! > >Thanks for the efforts. > > > >On Fri, Mar 22, 2024 at 10:00 AM Yanfei Lei wrote: > > > >> C

Re:Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-26 Thread Feifan Wang
Congratulations ! 在 2024-03-22 12:04:39,"Hangxiang Yu" 写道: >Congratulations! >Thanks for the efforts. > >On Fri, Mar 22, 2024 at 10:00 AM Yanfei Lei wrote: > >> Congratulations! >> >> Best regards, >> Yanfei >> >> Xuannan Su 于2024年3月22日周五 09:21写道: >> > >> > Congratulations! >> > >> > Best rega

[jira] [Created] (FLINK-34943) Support Flink 1.19, 1.20-SNAPSHOT for OpenSearch connector

2024-03-26 Thread Zhongqiang Gong (Jira)
Zhongqiang Gong created FLINK-34943: --- Summary: Support Flink 1.19, 1.20-SNAPSHOT for OpenSearch connector Key: FLINK-34943 URL: https://issues.apache.org/jira/browse/FLINK-34943 Project: Flink

[jira] [Created] (FLINK-34942) Support Flink 1.19, 1.20-SNAPSHOT for OpenSearch connector

2024-03-26 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-34942: --- Summary: Support Flink 1.19, 1.20-SNAPSHOT for OpenSearch connector Key: FLINK-34942 URL: https://issues.apache.org/jira/browse/FLINK-34942 Project: Flink

[jira] [Created] (FLINK-34941) Cannot convert org.apache.flink.streaming.api.CheckpointingMode to org.apache.flink.core.execution.CheckpointingMode

2024-03-26 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-34941: --- Summary: Cannot convert org.apache.flink.streaming.api.CheckpointingMode to org.apache.flink.core.execution.CheckpointingMode Key: FLINK-34941 URL: https://issues.apache.o

Re: [DISCUSS] FLINK-34440 Support Debezium Protobuf Confluent Format

2024-03-26 Thread Kevin Lam
Thanks Anupam! Looking forward to it. On Thu, Mar 14, 2024 at 1:50 AM Anupam Aggarwal wrote: > Hi Kevin, > > Thanks, these are some great points. > Just to clarify, I do agree that the subject should be an option (like in > the case of RegistryAvroFormatFactory). > We could fallback to subject a

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-26 Thread Hao Li
Hi Timo, Yeah. For `primary key` and `from table(...)` those are explicitly matched in parser: [1]. > SELECT f1, f2, label FROM ML_PREDICT( input => `my_data`, model => `my_cat`.`my_db`.`classifier_model`, args => DESCRIPTOR(f1, f2)); This named argument syntax looks good to me

[jira] [Created] (FLINK-34940) LeaderContender implementations handle invalid state

2024-03-26 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34940: - Summary: LeaderContender implementations handle invalid state Key: FLINK-34940 URL: https://issues.apache.org/jira/browse/FLINK-34940 Project: Flink Issue

Re: [DISCUSS] FLIP-437: Support ML Models in Flink SQL

2024-03-26 Thread Timo Walther
Hi Hao, > `TABLE(my_data)` and `MODEL(my_cat.my_db.classifier_model)` doesn't > work since `TABLE` and `MODEL` are already key words This argument doesn't count. The parser supports introducing keywords that are still non-reserved. For example, this enables using "key" for both primary key and

[DISCUSS] FLIP-XXX: Introduce Flink SQL variables

2024-03-26 Thread Ferenc Csaky
Hello devs, I would like to start a discussion about FLIP-XXX: Introduce Flink SQL variables [1]. The main motivation behing this change is to be able to abstract Flink SQL from environment-specific configuration and provide a way to carry jobs between environments (e.g. dev-stage-prod) without

[jira] [Created] (FLINK-34939) Harden TestingLeaderElection

2024-03-26 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34939: - Summary: Harden TestingLeaderElection Key: FLINK-34939 URL: https://issues.apache.org/jira/browse/FLINK-34939 Project: Flink Issue Type: Bug Comp

[jira] [Created] (FLINK-34938) Incorrect behaviour for comparison functions

2024-03-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-34938: Summary: Incorrect behaviour for comparison functions Key: FLINK-34938 URL: https://issues.apache.org/jira/browse/FLINK-34938 Project: Flink Issue Ty

[jira] [Created] (FLINK-34937) Apache Infra GHA policy update

2024-03-26 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34937: - Summary: Apache Infra GHA policy update Key: FLINK-34937 URL: https://issues.apache.org/jira/browse/FLINK-34937 Project: Flink Issue Type: Bug Co

Bug report for reading Hive table as streaming source.

2024-03-26 Thread Xiaolong Wang
Hi, I found a weird bug when reading a Hive table as a streaming source. In summary, if the first partition is not time related, then the Hive table cannot be read as a streaming source. e.g. I've a Hive table in the definition of ``` CREATE TABLE article ( id BIGINT, edition STRING, dt STRING