[jira] [Created] (FLINK-37799) Flink native model predict runtime implementation

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37799: -- Summary: Flink native model predict runtime implementation Key: FLINK-37799 URL: https://issues.apache.org/jira/browse/FLINK-37799 Project: Flink Issue Type: Sub-task

Re: [VOTE] Apache Flink CDC Release 3.4.0, release candidate #3

2025-05-13 Thread Yanquan Lv
Thanks to all developers for your verification, We have collected enough valid votes. I will announce the voting results and release this version later. wang junbo 于2025年5月13日周二 18:12写道: > +1(non-binding) > > - Build success from source with JDK8 & maven3.9.6 > - Reviewed release note at > https

Schema Evolution in Sql-Client

2025-05-13 Thread Ashish Khatkar
Hi, Why does flink sql-client allows altering column types from nullable to non nullable?Here is an example - CREATE CATALOG paimon WITH ('type' = 'paimon', 'warehouse' = 'file:/Users/ashish/Documents/flink-1.20.1/paimon_warehouse'); - USE CATALOG paimon; - CREATE TABLE evolution_test(field1 IN

[jira] [Created] (FLINK-37785) Cannot set advanced RocksDB options in state processor job

2025-05-13 Thread Alexis Sarda-Espinosa (Jira)
Alexis Sarda-Espinosa created FLINK-37785: - Summary: Cannot set advanced RocksDB options in state processor job Key: FLINK-37785 URL: https://issues.apache.org/jira/browse/FLINK-37785 Project:

[jira] [Created] (FLINK-37786) Infer schema information from the state of Source and emit it to avoid maintaining state in PreTransform

2025-05-13 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37786: -- Summary: Infer schema information from the state of Source and emit it to avoid maintaining state in PreTransform Key: FLINK-37786 URL: https://issues.apache.org/jira/browse/FLINK-377

Re: [VOTE] Apache Flink CDC Release 3.4.0, release candidate #3

2025-05-13 Thread wang junbo
+1(non-binding) - Build success from source with JDK8 & maven3.9.6 - Reviewed release note at https://github.com/apache/flink-web/pull/790 Find duplicate description for `scan.incremental.snapshot.unbounded-chunk-first.enabled` parameter in MySQL documentation. https://github.com/apache/flink-cdc

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-13 Thread Andrei Kaigorodov
Hi Roman, Thank you for the proposal. This is a much-needed feature. One question: For the PUT request, does it make sense to use a distinct HTTP status code in the response when the request fails due to a conflicting update? Since the new expected version field is included in the body, 409 Conf

[jira] [Created] (FLINK-37784) [Doc] Remove Duplicate Description for `scan.incremental.snapshot.unbounded-chunk-first.enabled` Parameter in Documentation

2025-05-13 Thread JunboWang (Jira)
JunboWang created FLINK-37784: - Summary: [Doc] Remove Duplicate Description for `scan.incremental.snapshot.unbounded-chunk-first.enabled` Parameter in Documentation Key: FLINK-37784 URL: https://issues.apache.org/jir

Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread Sharath
Hello, Have you tried enabling the buffer debloating feature to improve checkpoint times? Refer taskmanager.network.memory.buffer-debloat.enabled in https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/ Regards, Sharath On Tue, May 13, 2025 at 1:59 AM 张河川 wrote: > Hi Fli

Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread 张河川
Hi Flink community, I’m encountering an issue with PyFlink where a FlatMap operator invokes an external service (using a PyTorch model to generate embedding vectors). The operator processes data very slowly, leading to an extremely long initial checkpoint start delay, which eventually causes ch

[jira] [Created] (FLINK-37783) TieredStorage doesn't work when Buffer Debloating is enabled

2025-05-13 Thread Jira
David Morávek created FLINK-37783: - Summary: TieredStorage doesn't work when Buffer Debloating is enabled Key: FLINK-37783 URL: https://issues.apache.org/jira/browse/FLINK-37783 Project: Flink

Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread 张河川
Hi Flink community, I’m encountering an issue with PyFlink where a FlatMap operator invokes an external service (using a PyTorch model to generate embedding vectors). The operator processes data very slowly, leading to an extremely long initial checkpoint start delay, which eventually causes ch

Re:Re: Re: Re: [DISCUSS] FLIP-486: Introduce a new DeltaJoin

2025-05-13 Thread Xuyang
Hi, Feng. Let me address your questions: 1. As you mentioned, these filters should be recalculated. We can apply the filters after lookup the source, and then perform the join. Moreover, we can consider pushing the filters down to the lookup source to achieve a more efficient lookup. 2. You

[jira] [Created] (FLINK-37797) Documentation

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37797: -- Summary: Documentation Key: FLINK-37797 URL: https://issues.apache.org/jira/browse/FLINK-37797 Project: Flink Issue Type: Sub-task Reporter: Hao Li --

[jira] [Created] (FLINK-37789) Integrate with sqlvalidator

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37789: -- Summary: Integrate with sqlvalidator Key: FLINK-37789 URL: https://issues.apache.org/jira/browse/FLINK-37789 Project: Flink Issue Type: Sub-task Reporter: Ha

[jira] [Created] (FLINK-37788) Refractor equalsTo() method in classes to use Java 17 instanceof operator

2025-05-13 Thread Pranav Mahajan (Jira)
Pranav Mahajan created FLINK-37788: -- Summary: Refractor equalsTo() method in classes to use Java 17 instanceof operator Key: FLINK-37788 URL: https://issues.apache.org/jira/browse/FLINK-37788 Project

Re: [DISCUSS] FLIP-505: Flink History Server Scability Improvements, Remote Data Store Fetch and Per Job Fetch

2025-05-13 Thread Becket Qin
Thanks for the FLIP, Allison. The proposal makes a lot of sense in general. The history server is critical to the Flink batch. A few suggestions: 1. It might make sense to keep the existing config *historyserver.archive.retained-jobs*. This will only be used to determine the total number of jobs t

[jira] [Created] (FLINK-37787) Refractor Apache Flink project to use Java 21 features

2025-05-13 Thread Pranav Mahajan (Jira)
Pranav Mahajan created FLINK-37787: -- Summary: Refractor Apache Flink project to use Java 21 features Key: FLINK-37787 URL: https://issues.apache.org/jira/browse/FLINK-37787 Project: Flink Is

Re: Re: [VOTE] FLIP-497: Early Fire Support for Flink SQL Interval Join

2025-05-13 Thread Becket Qin
Thanks for the FLIP, Weqiing. +1 (binding) Cheers, Jiangjie (Becket) Qin On Tue, Feb 11, 2025 at 6:21 PM Xuyang wrote: > +1 (non-binding)-- > > Best! > Xuyang > > > > > > 在 2025-01-30 00:18:47,"Xingcan Cui" 写道: > >+1 (binding) > > > >Best, > >Xingcan > > > >On Mon, Jan 27, 2025 at 8:

Upgrade Flink's log4j to to 2.24.3

2025-05-13 Thread David Radley
Hi, I notice that Flink log4j is at level 2.24.1. There are critical bugs [2] that are fixed if we move to 2.24.3. I am happy to raise a Jira, make the update and backport to v2 and v1.20 (which is currently at an even lower level), if someone is willing to merge, Kind regards, David. [1]

[jira] [Created] (FLINK-37791) Integrate with sql to rel converter

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37791: -- Summary: Integrate with sql to rel converter Key: FLINK-37791 URL: https://issues.apache.org/jira/browse/FLINK-37791 Project: Flink Issue Type: Sub-task Repo

[jira] [Created] (FLINK-37790) Add model factory related interfaces/classes

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37790: -- Summary: Add model factory related interfaces/classes Key: FLINK-37790 URL: https://issues.apache.org/jira/browse/FLINK-37790 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-37792) Physical rewrite for ml_predict

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37792: -- Summary: Physical rewrite for ml_predict Key: FLINK-37792 URL: https://issues.apache.org/jira/browse/FLINK-37792 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-37794) ml_evaluate sql function skeleton

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37794: -- Summary: ml_evaluate sql function skeleton Key: FLINK-37794 URL: https://issues.apache.org/jira/browse/FLINK-37794 Project: Flink Issue Type: Sub-task Report

[jira] [Created] (FLINK-37795) Physical rewrite for ml_evaluate

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37795: -- Summary: Physical rewrite for ml_evaluate Key: FLINK-37795 URL: https://issues.apache.org/jira/browse/FLINK-37795 Project: Flink Issue Type: Sub-task Reporte

[jira] [Created] (FLINK-37793) Codegen for ml_predict

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37793: -- Summary: Codegen for ml_predict Key: FLINK-37793 URL: https://issues.apache.org/jira/browse/FLINK-37793 Project: Flink Issue Type: Sub-task Reporter: Hao Li

[jira] [Created] (FLINK-37796) Codegen for ml_evaluate

2025-05-13 Thread Hao Li (Jira)
Hao Li created FLINK-37796: -- Summary: Codegen for ml_evaluate Key: FLINK-37796 URL: https://issues.apache.org/jira/browse/FLINK-37796 Project: Flink Issue Type: Sub-task Reporter: Hao Li

Re: [RESULT][VOTE] FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design

2025-05-13 Thread Hao Li
Hi Dev, I'm happy to announce that FLIP-525: Model ML_PREDICT, ML_EVALUATE Implementation Design [1] has been accepted with 8 approving votes (5 binding) [2]. There are no disapproving votes. Yash Anand (non-binding) Mayank Juneja (non-binding) Shengkai Fang (binding) Ron Liu (binding) Yunfeng Zh

[jira] [Created] (FLINK-37798) Persist DelayedScaleDown info in state store

2025-05-13 Thread Sai Sharath Dandi (Jira)
Sai Sharath Dandi created FLINK-37798: - Summary: Persist DelayedScaleDown info in state store Key: FLINK-37798 URL: https://issues.apache.org/jira/browse/FLINK-37798 Project: Flink Issue

Re:Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread 张河川
Hello, I tried the configuration you mentioned, but it doesn't seem to work. Still, thank you for your response! At 2025-05-13 17:54:03, "Sharath" wrote: >Hello, > >Have you tried enabling the buffer debloating feature to improve checkpoint >times? Refer taskmanager.network.memor

Re:Re: Long Initial Checkpoint Start Delay in PyFlink FlatMap Operator

2025-05-13 Thread Hirson Zhang
Hello, I tried the configuration you mentioned, but it doesn't seem to work. Still, thank you for your response! At 2025-05-13 17:54:03, "Sharath" wrote: >Hello, > >Have you tried enabling the buffer debloating feature to improve checkpoint >times? Refer taskmanager.network.memor

[jira] [Created] (FLINK-37800) When using jobmanager-pod-template to deploy a Native Kubernetes Application Job, an error message appears in the log: 'Invalid value: "/opt/flink/artifacts": must be un

2025-05-13 Thread Yang Zhou (Jira)
Yang Zhou created FLINK-37800: - Summary: When using jobmanager-pod-template to deploy a Native Kubernetes Application Job, an error message appears in the log: 'Invalid value: "/opt/flink/artifacts": must be unique,' Key: FLINK-37800