Re: [DISCUSS] FLIP-514: Custom Evaluator plugin for Flink Autoscaler

Hi Gyula, Good morning! Thanks so much for your review and feedback! I absolutely agree about the problem of identifying a job vertex well enough to make more informative decisions using the custom evaluator. Would be more than happy to explore an approach for associating operators with vertex

[jira] [Created] (FLINK-37522) Add description for the config about file system connection limit

Jiaan Geng created FLINK-37522: -- Summary: Add description for the config about file system connection limit Key: FLINK-37522 URL: https://issues.apache.org/jira/browse/FLINK-37522 Project: Flink

[jira] [Created] (FLINK-37525) Introduce overload findModuleFactory to improve the performance of installModules

Jiaan Geng created FLINK-37525: -- Summary: Introduce overload findModuleFactory to improve the performance of installModules Key: FLINK-37525 URL: https://issues.apache.org/jira/browse/FLINK-37525 Project

Re: [VOTE] FLIP-513: Split-level watermark metrics

Hi everyone, Thank you all for your votes. I am now closing the vote and will announce the results in a separate email. On Tue, 18 Mar 2025 at 03:32, Yuepeng Pan wrote: > +1 (non-binding). Thanks for driving it! > Best Regards,Yuepeng Pan > > At 2025-03-18 08:12:22, "Xiqian YU" wrote: > >+1 (n

Re: [VOTE] FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

+1 (non-binding) On Thu, 20 Mar, 2025, 5:28 pm Andrea Sella, wrote: > +1 (non-binding) > > but eager to trying it out as we have custom code similar to the proposal > during deployment. > > Cheers, > > On Thu, 20 Mar 2025 at 08:23, Gyula Fóra wrote: > > > +1 (binding) > > > > And a gentle remin

[RESULT] [VOTE] FLIP-509: Pluggable Batching For Async Sink

Hi all, The voting for FLIP-509: Pluggable Batching For Async Sink has concluded. There were 3 binding + 2 (non-binding) votes & no disapprovals. - Danny Cranmer (binding) - Hong Liang (bindin

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

Hi All, @Zakelly: Gyula summarised it correctly what I meant so please treat the content as mine. As an addition I'm not against to add CLI at all, I'm just stating that in some cases like this, users would like to have a self-serving solution where they can provide SQL statements which can trigge

[jira] [Created] (FLINK-37520) One input datastream operator with async state

Yanfei Lei created FLINK-37520: -- Summary: One input datastream operator with async state Key: FLINK-37520 URL: https://issues.apache.org/jira/browse/FLINK-37520 Project: Flink Issue Type: Sub-ta

Re: [DISCUSS] FLIP-512: Add meta information to SQL state connector

Hi! @Zakelly Lan I think what Gabor means is that users want to have predefined SQL scripts to perform state analysis tasks to debug/identify problems. Such as write a SQL script that joins the metadata table with the state and do some analytics on it. If we have a meta table then the SQL script

Re: [VOTE] FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

+1 (binding) And a gentle reminder to please vote :) Cheers, Gyula On Tue, Mar 11, 2025 at 3:02 PM Gyula Fóra wrote: > Hi all, > > I would like to start the vote for FLIP-503: Blue/Green Deployments for > Flink on Kubernetes: Phase 1 (basic), on behalf of Sergio Chong Loo. > > This FLIP was di

[jira] [Created] (FLINK-37521) Two input datastream operators with async state

Yanfei Lei created FLINK-37521: -- Summary: Two input datastream operators with async state Key: FLINK-37521 URL: https://issues.apache.org/jira/browse/FLINK-37521 Project: Flink Issue Type: Sub-t

[jira] [Created] (FLINK-37523) e2e: Test stream python udf sql job fails

Fabian Paul created FLINK-37523: --- Summary: e2e: Test stream python udf sql job fails Key: FLINK-37523 URL: https://issues.apache.org/jira/browse/FLINK-37523 Project: Flink Issue Type: Bug

[RESULT] [VOTE] FLIP-513: Split-level watermark metrics

Hi everyone, The voting for FLIP-513: Split-level watermark metrics [1] has concluded. There were 3 binding, 4 non-binding votes & no disapprovals. - Piotr Nowojski (binding) - Roman Khachatryan (binding) - Stefan Richter (binding) - Ahmed Hamdy (non-binding) - Aleksandr Savonin (non-binding) - Y

[jira] [Created] (FLINK-37526) Promote SinkV2 API classes from @Experimental to @PublicEvolving

Maximilian Michels created FLINK-37526: -- Summary: Promote SinkV2 API classes from @Experimental to @PublicEvolving Key: FLINK-37526 URL: https://issues.apache.org/jira/browse/FLINK-37526 Project:

[jira] [Created] (FLINK-37527) Add `KuduSource` documentation

Ferenc Csaky created FLINK-37527: Summary: Add `KuduSource` documentation Key: FLINK-37527 URL: https://issues.apache.org/jira/browse/FLINK-37527 Project: Flink Issue Type: New Feature

Re: [DISCUSS] FLIP-514: Custom Evaluator plugin for Flink Autoscaler

Hi Pradeepta, Generally, the proposal looks good to me. The only issue I see is that it is fairly generic. On the one hand, it allows plugging in additional evaluation logic into the autoscaler, which allows for many new ways to extend the current autoscaling logic. On the other hand, normal users

Re: [DISCUSS] FLIP-514: Custom Evaluator plugin for Flink Autoscaler

Hi! I think this is a very nice proposal, +1 from me! The only practical challenge is to actually be able to identify what job vertex is what in the job graph to make good predictions / feed in external information but this is definitely out of scope here. The autoscaler in general needs to have

Re: [VOTE] FLIP-503: Blue/Green Deployments for Flink on Kubernetes: Phase 1 (basic)

One question. When we say > >- Records/data will be duplicated for as long as both pipelines are >running concurrently > > Do we mean the source record will be forwarded to both pipelines? I'm asking this just as a concern because, depending on the implementation, a source like Kafka coul