[DISCUSS] FLIP-467: Introduce Generalized Watermarks

Hi Devs, Jeyhun Karimov, Weijie Guo and I would like to initiate a discussion about FLIP-467: Introduce Generalized Watermarks [1]. Based on our findings, we recognize the need for specific events that require propagation and alignment across streams, functioning similarly to watermarks. An examp

Re: Plans for JDBC connector for 1.20?

Hi Yanquan and Ruan, Echo to David's request. To support lineage integration in JDBC connector, we need to drop the support of Flink 1.18 for back compatibility check as the recent change in Kafka Connector https://issues.apache.org/jira/browse/FLINK-35109. Best Regards Peter Huang On Mon,

[jira] [Created] (FLINK-36875) Complete timer before emiting watermark

Yanfei Lei created FLINK-36875: -- Summary: Complete timer before emiting watermark Key: FLINK-36875 URL: https://issues.apache.org/jira/browse/FLINK-36875 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-36874) KafkaSourceBuilder in pyflink did not implement setDeserializer() method

Shuyi Chen created FLINK-36874: -- Summary: KafkaSourceBuilder in pyflink did not implement setDeserializer() method Key: FLINK-36874 URL: https://issues.apache.org/jira/browse/FLINK-36874 Project: Flink

[jira] [Created] (FLINK-36873) Adapting batch job progress recovery to Apache Celeborn

xuhuang created FLINK-36873: --- Summary: Adapting batch job progress recovery to Apache Celeborn Key: FLINK-36873 URL: https://issues.apache.org/jira/browse/FLINK-36873 Project: Flink Issue Type: Imp

Re: Plans for JDBC connector for 1.20?

Hi, Yanquan & David. I would like to help to release the version jdbc-3.3.0. Thanks~ Best, Hang On Tue, Dec 3, 2024 at 10:42 PM David Radley wrote: > Hi Yanquan, > Thanks for your support. Yes and support for the new dialects. > > It would be great to get open PRs merged to support new dialect

Re: [DISCUSS] FLIP-487: Show history of rescales in Web UI for AdaptiveScheduler

Hi, community, It has been a few days since this FLIP/Email-threads split proposal was shared, and we haven’t received any additional relevant suggestions or comments so far. We would like to wait another 1-2 days for more feedback. If no significantly different opinions are expressed by then,

Re: [Discussion] Session Clusters Support Heterogeneous Task Manager Images

Hello, Sorry for the delay. I agree I think that works for most workflows. The only caveat would be CUDA based ML workflows. You can't bundle CUDA into a dependency bundle. Overall, it works in application mode. It would just be awesome to use Session clusters for Batch / ephemeral test streaming

Re: [DISCUSS] FLIP-487: Show history of rescales in Web UI for AdaptiveScheduler

Thanks Peter for the comments. > Beside this, should we create new metrics for internal rescale to > distinguish the metrics for internal restart? > Created a jira for this https://issues.apache.org/jira/browse/FLINK-36871. This will make sense to me. I'm just wondering, how should the relations

[jira] [Created] (FLINK-36872) The standby jobmanager webui cannot view the taskmanager logs

yuanfenghu created FLINK-36872: -- Summary: The standby jobmanager webui cannot view the taskmanager logs Key: FLINK-36872 URL: https://issues.apache.org/jira/browse/FLINK-36872 Project: Flink Is

Re: [DISCUSS] FLIP-489: Add missing dropTable/dropView methods to TableEnvironment

> While reading it was surprised that the > create functions like `createTable` return void. >The suggestion to return boolean like the other drop methods seems good. Thanks for the feedback. I agree there is a number of things which could be improved in this interface including the one you've me

Re: [DISCUSS] FLIP-487: Show history of rescales in Web UI for AdaptiveScheduler

Thanks for the proposal. It is a very useful feature for the observability of autoscaling. . Echo to Eric. We need to add some content about where to store the rescale events. We may consider using the same mechanism of how exception history is stored. Beside this, should we create new metrics for

[jira] [Created] (FLINK-36871) Add internalRescale metrics to distinguish internalRestart Metrics

Zhenqiu Huang created FLINK-36871: - Summary: Add internalRescale metrics to distinguish internalRestart Metrics Key: FLINK-36871 URL: https://issues.apache.org/jira/browse/FLINK-36871 Project: Flink

[jira] [Created] (FLINK-36870) Start scale down timer after initial scale up

Maximilian Michels created FLINK-36870: -- Summary: Start scale down timer after initial scale up Key: FLINK-36870 URL: https://issues.apache.org/jira/browse/FLINK-36870 Project: Flink Iss

Re: Blue/Green Deployments support for Flink

Hello Sergio and Jose, Apologies for jumping into the conversation. I’m very interested in what you’ve learned about blue/green Flink deployments. I’m currently working on a project aimed at achieving no-downtime Flink deployments, so your insights would be highly valuable. Thank you, Yang

[jira] [Created] (FLINK-36869) Introduce TemporalRowTimeJoinOperator in TemporalJoin with Async State API

Wang Qilong created FLINK-36869: --- Summary: Introduce TemporalRowTimeJoinOperator in TemporalJoin with Async State API Key: FLINK-36869 URL: https://issues.apache.org/jira/browse/FLINK-36869 Project: Fli

[jira] [Created] (FLINK-36868) Provide file system methods with string parameters for ForSt JNI

Zakelly Lan created FLINK-36868: --- Summary: Provide file system methods with string parameters for ForSt JNI Key: FLINK-36868 URL: https://issues.apache.org/jira/browse/FLINK-36868 Project: Flink

[jira] [Created] (FLINK-36867) flink on k8s operator could not create TM

周龙华 created FLINK-36867: --- Summary: flink on k8s operator could not create TM Key: FLINK-36867 URL: https://issues.apache.org/jira/browse/FLINK-36867 Project: Flink Issue Type: Bug Components:

RE: [VOTE] FLIP-484: Add custom metric variables to operators

Hi Piotr, Thanks your you quick response and for the extra details. I am thinking that if this were use case driven then * there would be some use cases like the table name that could be derived from existing metadata, either names, tags or

[jira] [Created] (FLINK-36866) Support more CDC YAML numeric casting paths

yux created FLINK-36866: --- Summary: Support more CDC YAML numeric casting paths Key: FLINK-36866 URL: https://issues.apache.org/jira/browse/FLINK-36866 Project: Flink Issue Type: Bug Component

[jira] [Created] (FLINK-36865) Provide UNIX_TIMESTAMP series functions in YAML pipeline

yux created FLINK-36865: --- Summary: Provide UNIX_TIMESTAMP series functions in YAML pipeline Key: FLINK-36865 URL: https://issues.apache.org/jira/browse/FLINK-36865 Project: Flink Issue Type: New Featur