Re: [VOTE] FLIP-507: Add Model DDL methods in TABLE API

2025-03-11 Thread Yash Anand
Hi Timo, Thanks for pointing that out. I have added the full API of the ModelDescriptor in the FLIP. Thanks, Yash Anand On Mon, Mar 10, 2025 at 11:27 AM Timo Walther wrote: > Hi Yash, > > could you provide the full API of the ModelDescriptor in the FLIP? > > Thanks, > Timo > > > On 10.03.25 1

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Leonard Xu
+1 for this proposal after went through Dawid and Xuyang’s discussion, and I think we can start a vote if there’re no objections. For the naming, shorter is better in this case, +1 for ChangelogMode.keyOnlyDeletes() after discussed with ChatGPT. Best, Leonard

Re: [VOTE] Release flink-connector-mongodb v2.0.0, release candidate #2

2025-03-11 Thread Robert Metzger
+1 (binding) - Source archive looks ok (no binaries, maven builds, versions in pom seem right) - Staging repo looks ok On Thu, Feb 27, 2025 at 9:07 AM Jiabao Sun wrote: > Hi everyone, > > We need more binding votes to move forward with this release. > > Thanks, > Jiabao > > On 2025/02/08 08:20

Re: [VOTE] FLIP-507: Add Model DDL methods in TABLE API

2025-03-11 Thread Leonard Xu
Thanks @Anand for the quick response and update, the updated FLIP looks clear enough to me. +1(binding) Best, Leonard > 2025年3月11日 12:46,Yash Anand 写道: > > Hi Leonard, > > Thank you for your input. I will update the FLIP accordingly to make it > more clear and standardized enough. > > Than

[jira] [Created] (FLINK-37460) Using State Processor API and Kafka Sink with Exactly once delivery leads to org.apache.kafka.common.errors.InvalidPidMappingException

2025-03-11 Thread Grzegorz Liter (Jira)
Grzegorz Liter created FLINK-37460: -- Summary: Using State Processor API and Kafka Sink with Exactly once delivery leads to org.apache.kafka.common.errors.InvalidPidMappingException Key: FLINK-37460 URL: https://i

Re: [DISCUSS] FLIP-XXX: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

2025-03-11 Thread Sergio Chong Loo
@Gyula, Thanks for the input, I also second the “blue/green” naming convention; and yes none of the colors is meant to have any meaning or purpose other than distinction. @Alexis, Indeed, so far the proposal/doc suggests a FlinkBlueGreenDeployment CRD Sergio > On Mar 6, 2025, at 9:12 AM, Ale

Re: [VOTE] FLIP-507: Add Model DDL methods in TABLE API

2025-03-11 Thread Timo Walther
Hi Yash, thanks for updating the FLIP. Here is more feedback from my side: Add `option(ConfigOption configOption, T value)` similar to `TableDescriptor`. Use `comment()` instead of `setComment` to be in sync with `TableDescriptor`. In `TableDescriptor`, the schema is not part of the `newB

[jira] [Created] (FLINK-37457) Support asArgument in Python Table API

2025-03-11 Thread Timo Walther (Jira)
Timo Walther created FLINK-37457: Summary: Support asArgument in Python Table API Key: FLINK-37457 URL: https://issues.apache.org/jira/browse/FLINK-37457 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-37456) Stream data for SQL state connector in case of List and Map state

2025-03-11 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-37456: - Summary: Stream data for SQL state connector in case of List and Map state Key: FLINK-37456 URL: https://issues.apache.org/jira/browse/FLINK-37456 Project: Flink

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-11 Thread Gabor Somogyi
Hi Shengkai, > 1. State TTL for Value Columns >From directional perspective I agree your idea how it can be implemented. Previously I've mentioned that TTL information is not exposed on the state processor API (which the SQL state connector uses to read data) and unless somebody show me the oppos

[jira] [Created] (FLINK-37453) Migration Tests / data_stream_migration_test (8, 1.20.1) failed

2025-03-11 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-37453: -- Summary: Migration Tests / data_stream_migration_test (8, 1.20.1) failed Key: FLINK-37453 URL: https://issues.apache.org/jira/browse/FLINK-37453 Project: Flink

[jira] [Created] (FLINK-37454) Unit Tests (mongodb6) / compile_and_test (8, generic) is not stable

2025-03-11 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-37454: -- Summary: Unit Tests (mongodb6) / compile_and_test (8, generic) is not stable Key: FLINK-37454 URL: https://issues.apache.org/jira/browse/FLINK-37454 Project: Flink

Re: Re: [DISCUSS] FLIP-495: Support AdaptiveScheduler record and query the rescale history

2025-03-11 Thread Yuepeng Pan
Hi, Matthias, So sorry for the long delay in completing and following up. I'm still making the necessary research to address the issues mentioned previously in this FLIP. FYI, the new round of adjustments has been completed, and I've attached an overview of a few issues. > Also keep in mind th

[jira] [Created] (FLINK-37459) Document correction - 1.9.0

2025-03-11 Thread Shalini M (Jira)
Shalini M created FLINK-37459: - Summary: Document correction - 1.9.0 Key: FLINK-37459 URL: https://issues.apache.org/jira/browse/FLINK-37459 Project: Flink Issue Type: Improvement Rep

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Leonard Xu
+1(binding) Best, Leonard > 2025年3月11日 18:42,Timo Walther 写道: > > +1 (binding) > > Thanks for working on this! > > Timo > > On 11.03.25 11:05, Dawid Wysakowicz wrote: >> Hi everyone, >> I'd like to start a vote on FLIP-510: Drop ChangelogNormalize for >> operations which don't need it[1] >>

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Dawid Wysakowicz
Thank you for the feedback. I updated the name in the Flip and I'll start the vote in a separate thread. On Tue, 11 Mar 2025 at 09:21, Leonard Xu wrote: > +1 for this proposal after went through Dawid and Xuyang’s discussion, and > I think we can start a vote if there’re no objections. > > For t

Re: [VOTE] FLIP-509 Pluggable Batching For Async Sink

2025-03-11 Thread Doğuşcan Namal
Thanks for the FLIP Poorvank. We already had a discussion on the Google doc and thanks for addressing the comments. It looks good as is to me, +1(non-binding) On Tue, 11 Mar 2025 at 03:39, Poorvank Bhatia wrote: > Hi all, > > I would like to start a vote on FLIP-509 > < > https://cwiki.apache.o

Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Timo Walther
+1 (binding) Thanks for working on this! Timo On 11.03.25 11:05, Dawid Wysakowicz wrote: Hi everyone, I'd like to start a vote on FLIP-510: Drop ChangelogNormalize for operations which don't need it[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours u

Re:Re: [VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Xuyang
+1(binding) -- Best! Xuyang 在 2025-03-11 19:12:05,"Leonard Xu" 写道: >+1(binding) > >Best, >Leonard > >> 2025年3月11日 18:42,Timo Walther 写道: >> >> +1 (binding) >> >> Thanks for working on this! >> >> Timo >> >> On 11.03.25 11:05, Dawid Wysakowicz wrote: >>> Hi everyone, >>> I'd li

[jira] [Created] (FLINK-37455) Create error Event when job goes into FAILED state

2025-03-11 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-37455: -- Summary: Create error Event when job goes into FAILED state Key: FLINK-37455 URL: https://issues.apache.org/jira/browse/FLINK-37455 Project: Flink Issue Type: Im

Re: [DISCUSS] FLIP-XXX: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

2025-03-11 Thread Alexis Sarda-Espinosa
Hi everyone, I had also thought about this kind of functionality in the past and I'm very interested to see how it works out. I had imagined something like a FlinkContinuousDeployment as CRD, just putting it out there. Regards, Alexis. On Thu, 6 Mar 2025, 17:31 Gyula Fóra, wrote: > Hi! > > I t

[jira] [Created] (FLINK-37446) Support asArgument in Table API

2025-03-11 Thread Timo Walther (Jira)
Timo Walther created FLINK-37446: Summary: Support asArgument in Table API Key: FLINK-37446 URL: https://issues.apache.org/jira/browse/FLINK-37446 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-37428) Add JDK21 to Docker build matrix

2025-03-11 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-37428: -- Summary: Add JDK21 to Docker build matrix Key: FLINK-37428 URL: https://issues.apache.org/jira/browse/FLINK-37428 Project: Flink Issue Type: Sub-task Affects

Re: [DISCUSS] FLIP-XXX: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

2025-03-11 Thread Gyula Fóra
I think we should proceed with the vote :) Let me start the voting thread. On Tue, Mar 11, 2025 at 2:56 PM Sergio Chong Loo wrote: > @Gyula, > > Thanks for the input, I also second the “blue/green” naming convention; > and yes none of the colors is meant to have any meaning or purpose other >

[VOTE] FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

2025-03-11 Thread Gyula Fóra
Hi all, I would like to start the vote for FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic), on behalf of Sergio Chong Loo. This FLIP was discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1] ht

Re: [VOTE] Release flink-connector-mongodb v2.0.0, release candidate #2

2025-03-11 Thread Ferenc Csaky
+1 (non-binding) - Verified checksums and signatures - Built source with JDK8 and Maven 3.8.6 - Reviewed web PR Thanks, Ferenc On Tuesday, March 11th, 2025 at 09:17, Robert Metzger wrote: > > > +1 (binding) > > - Source archive looks ok (no binaries, maven builds, versions in pom seem >

[VOTE] FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Dawid Wysakowicz
Hi everyone, I'd like to start a vote on FLIP-510: Drop ChangelogNormalize for operations which don't need it[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/x/7o

[jira] [Created] (FLINK-37447) [Flink CDC] Supports Variant type in YAML API

2025-03-11 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-37447: -- Summary: [Flink CDC] Supports Variant type in YAML API Key: FLINK-37447 URL: https://issues.apache.org/jira/browse/FLINK-37447 Project: Flink Issue Type: New Fea

Re: [VOTE] FLIP-507: Add Model DDL methods in TABLE API

2025-03-11 Thread Mingge Deng
Thanks Yash! +1 (binding) Best, Mingge On Mon, Mar 10, 2025 at 8:51 AM Dawid Wysakowicz wrote: > +1 (binding) > Best, > Dawid > > On Wed, 19 Feb 2025 at 18:37, Hao Li wrote: > > > +1 (non-binding) > > > > Thanks Yash, > > Hao > > > > On Tue, Feb 18, 2025 at 10:46 AM Yash Anand > > > wrote:

Re: FLIP-510: Drop ChangelogNormalize for operations which don't need it

2025-03-11 Thread Timo Walther
Hi Dawid, thanks for this proposal. This is a very nice improvement to the SQL engine. Changelog normalize is a very state-intensive operation. Any possibility to avoid it should be implemented. Updating ChangelogMode is an elegant solution and avoid updating the RowKind. When we introduced

Re: Flink CDC to Paimon

2025-03-11 Thread Taher Koitawala
Hi All, I am facing issues finding a class Org.apache.flink.cdc.connectors.shaded.org.apache.kafka.connect.json.JsonConverter I have added 1. Flink-connector-base 1.18.1 Flink-connector-debezium 3.1.0 Flink-cdc-pipeline-connectors-values 3.1.0 Flink-cdc-base 3.1.0 Flink-cdc-pipeline-conn

[jira] [Created] (FLINK-37434) Incremental checkpoint not configurable for ForSt's sync mode

2025-03-11 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-37434: --- Summary: Incremental checkpoint not configurable for ForSt's sync mode Key: FLINK-37434 URL: https://issues.apache.org/jira/browse/FLINK-37434 Project: Flink

[jira] [Created] (FLINK-37448) status.taskManager.replicas to show actual TaskManager count

2025-03-11 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-37448: -- Summary: status.taskManager.replicas to show actual TaskManager count Key: FLINK-37448 URL: https://issues.apache.org/jira/browse/FLINK-37448 Project: Flink

[jira] [Created] (FLINK-37429) Map each original column name to a new name and support column name containing '-'

2025-03-11 Thread Xiao Huang (Jira)
Xiao Huang created FLINK-37429: -- Summary: Map each original column name to a new name and support column name containing '-' Key: FLINK-37429 URL: https://issues.apache.org/jira/browse/FLINK-37429 Projec

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-11 Thread Shengkai Fang
Hi Gabor, > 2. Adding a new connector with `savepoint-metadata` I would argue against introducing a new connector type named savepoint-metadata, as the existing Catalog mechanism can inherently provide the necessary connector factory capabilities. I’ve detailed this proposal in branch[1]. Please

Re: Question on usage of SQL Gateway with a remote Flink cluster

2025-03-11 Thread Shengkai Fang
Sql client uses the REST API to submit flink jobs. So you should use the option `rest.address` and `rest.port` to expose jobmanager address. Best, Shengkai Ammu P 于2025年3月5日周三 15:53写道: > Hi everyone, > > I am trying to run a SQL script through SQL Client in gateway mode. The > gateway is runnin

[jira] [Created] (FLINK-37461) Unstable test case in MySqlToPaimonE2eITCase#testSyncWholeDatabase

2025-03-11 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37461: -- Summary: Unstable test case in MySqlToPaimonE2eITCase#testSyncWholeDatabase Key: FLINK-37461 URL: https://issues.apache.org/jira/browse/FLINK-37461 Project: Flink

Re: Integrating LLMs into apache flink pipelines

2025-03-11 Thread Shengkai Fang
Thanks for your sharing! Could you confirm whether the pipeline is capable of generating structured summaries for product-specific comments or customer feedback? Best, Shengkai Ganesh 于2025年2月23日周日 13:38写道: > Hi Everyone, I've been experimenting with integrating LLMs into ETL and > data pipeli

[jira] [Created] (FLINK-37462) Unstable test case in FlinkParallelizedPipelineITCase.testRegularTablesSourceInMultipleParallelism

2025-03-11 Thread Yanquan Lv (Jira)
Yanquan Lv created FLINK-37462: -- Summary: Unstable test case in FlinkParallelizedPipelineITCase.testRegularTablesSourceInMultipleParallelism Key: FLINK-37462 URL: https://issues.apache.org/jira/browse/FLINK-37462

[jira] [Created] (FLINK-37444) Migrate ArrowSourceFunction to SourceV2

2025-03-11 Thread Poorvank Bhatia (Jira)
Poorvank Bhatia created FLINK-37444: --- Summary: Migrate ArrowSourceFunction to SourceV2 Key: FLINK-37444 URL: https://issues.apache.org/jira/browse/FLINK-37444 Project: Flink Issue Type: Tec

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

2025-03-11 Thread Zakelly Lan
Hi Gabor and Shengkai, Thanks for sharing your thoughts! This is a long discussion and sorry for the late reply (I'm busy catching up with release 2.0 these days). 1. State TTL for Value Columns Let me first clarify your thoughts to ensure I understand correctly. IIUC, there is no persistent co

Re: [VOTE] Release 2.0.0, release candidate #2

2025-03-11 Thread Zakelly Lan
+1 (non-binding) - verified signatures & checksums - built from source using jdk11 - reviewed the website PR - run several Nexmark queries with disaggregated state on oss, and tried rescaling, everything looks fine. On Mon, Mar 10, 2025 at 5:06 PM Xintong Song wrote: > Hi everyone, > > There we

Re: [VOTE] Release 2.0.0, release candidate #2

2025-03-11 Thread Yuepeng Pan
+1 non-binding.- Donwload the rc2 source and binary package.- Check signatures & checksums- Build from source- Run wordcount demo on standalone cluster. Best, Yuepeng Pan At 2025-03-12 12:50:41, "Zakelly Lan" wrote: >+1 (non-binding) > >- verified signatures & checksums >- built from source u

[jira] [Created] (FLINK-37463) Make ForStMapState Serializer ThreadLocal

2025-03-11 Thread Yue Ma (Jira)
Yue Ma created FLINK-37463: -- Summary: Make ForStMapState Serializer ThreadLocal Key: FLINK-37463 URL: https://issues.apache.org/jira/browse/FLINK-37463 Project: Flink Issue Type: Bug Compo

Re: [VOTE] Release 2.0.0, release candidate #2

2025-03-11 Thread weijie guo
+1(binding) - verified signature and checksum - built from source using jdk11 and jdk17 - checked maven staging jars - ran a simple test job in local cluster Best regards, Weijie Zakelly Lan 于2025年3月12日周三 12:51写道: > +1 (non-binding) > > - verified signatures & checksums > - built from source

Re: [VOTE] FLIP-507: Add Model DDL methods in TABLE API

2025-03-11 Thread Dawid Wysakowicz
+1 (binding) Best, Dawid On Wed, 19 Feb 2025 at 18:37, Hao Li wrote: > +1 (non-binding) > > Thanks Yash, > Hao > > On Tue, Feb 18, 2025 at 10:46 AM Yash Anand > wrote: > > > Hi Everyone, > > > > I'd like to start a vote on FLIP-507: Add Model DDL methods in TABLE API > > [1] which has been disc

Re: [DISCUSS] Split-level Watermark Metrics

2025-03-11 Thread Efrat Levitan
Thanks for the feedback Piotr. If there's no other feedback, voting will start in a couple of days. Efrat On Mon, 3 Mar 2025 at 11:15, Piotr Nowojski wrote: > Hi Efrat, > > Thanks for the proposal and +1 from my side for this FLIP. > > Flink currently has a huge observability gap when it comes t