Re: [VOTE] FLIP-398: Improve Serialization Configuration And Usage In Flink

+1 (non-binding) Best, Zhanghao Chen From: Yong Fang Sent: Wednesday, December 27, 2023 14:54 To: dev Subject: [VOTE] FLIP-398: Improve Serialization Configuration And Usage In Flink Hi devs, Thanks for all feedback about the FLIP-398: Improve Serialization Con

Re: [VOTE] FLIP-398: Improve Serialization Configuration And Usage In Flink

+1 (binding) Best, Xintong On Wed, Dec 27, 2023 at 2:54 PM Yong Fang wrote: > Hi devs, > > Thanks for all feedback about the FLIP-398: Improve Serialization > Configuration And Usage In Flink [1] which has been discussed in [2]. > > I'd like to start a vote for it. The vote will be open for

Re: [DISCUSS] FLIP-397: Add config options for administrator JVM options

+1 for this, we have met jobs that need to set GC policies different from the default ones to improve performance. Separating the default and user-set ones can help us better manage them. Best, Fang Yong On Fri, Dec 22, 2023 at 9:18 PM Benchao Li wrote: > +1 from my side, > > I also met some sc

[VOTE] FLIP-398: Improve Serialization Configuration And Usage In Flink

Hi devs, Thanks for all feedback about the FLIP-398: Improve Serialization Configuration And Usage In Flink [1] which has been discussed in [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1] https://cwiki.apac

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

Hi Lijie, Thanks for the reminder! I missed this. Considering the name occupation, how about naming it as `execution.checkpointing.type`? Actually I think the current `execution.checkpointing.mode` is confusing in some ways, maybe `execution.checkpointing.data-consistency` is better. Best, Zak

Re: [VOTE] Release flink-connector-pulsar 4.1.0, release candidate #1

+1 (binding) - Verified checksum and signature - Checked that no binary exists in source code release - Checked Maven staging repository contains expected JARs - Checked release note - Built from source locally - Reviewed web PR Thanks for the effort, Leonard! Best, Qingsheng On Thu, Dec 21,

[jira] [Created] (FLINK-33949) METHOD_ABSTRACT_NOW_DEFAULT should be both source compatible and binary compatible

Wencong Liu created FLINK-33949: --- Summary: METHOD_ABSTRACT_NOW_DEFAULT should be both source compatible and binary compatible Key: FLINK-33949 URL: https://issues.apache.org/jira/browse/FLINK-33949 Proj

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

Hi, Rui Fan. Thanks for this FLIP. I think the key of LOCAL_NUMBER_TASK_MANAGER is better as 'minicluster.number-of-taskmanagers' or 'minicluster.taskmanager-number' instead of 'minicluster.number-taskmanager'. Best, Hang Xuannan Su 于2023年12月27日周三 12:40写道： > Hi Xintong and Rui, > > Thanks for

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

Hi Zakelly, >> I'm wondering if `execution.checkpointing.savepoint-dir` would be better. `execution.checkpointing.dir` and `execution.checkpointing.savepoint-dir` are also fine for me. >> So I think an enumeration option `execution.checkpointing.mode` which can be 'full' (default) or 'incrementa

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

Hi Xintong and Rui, Thanks for the quick feedback and the suggestions. > 1. I think the default value for `TASK_MANAGER_LOG_PATH_KEY` should be "no > default". I have considered both ways of describing the default value. However, I found out that some of the configurations, such as `web.tmpdir`,

Re: [DISCUSS] FLIP-397: Add config options for administrator JVM options

Thanks everyone. I'll start voting after the New Year's holiday if there's no further comment. Best, Zhanghao Chen From: Benchao Li Sent: Friday, December 22, 2023 21:18 To: dev@flink.apache.org Subject: Re: [DISCUSS] FLIP-397: Add config options for administrat

[jira] [Created] (FLINK-33948) Cleanup usage of org.apache.flink.table.api.dataview.MapView#MapView

Jacky Lau created FLINK-33948: - Summary: Cleanup usage of org.apache.flink.table.api.dataview.MapView#MapView Key: FLINK-33948 URL: https://issues.apache.org/jira/browse/FLINK-33948 Project: Flink

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

Hi everyone, Thanks all for your comments! @Yanfei > 1. For some state backends that do not support incremental checkpoint, > how does the execution.checkpointing.incrementaloption take effect? Or > is it better to put incremental under state.backend.xxx.incremental? > I'd rather not put the opt

[jira] [Created] (FLINK-33947) Fix bugs in DelegatingConfiguration missed the prefix mapping

RocMarshal created FLINK-33947: -- Summary: Fix bugs in DelegatingConfiguration missed the prefix mapping Key: FLINK-33947 URL: https://issues.apache.org/jira/browse/FLINK-33947 Project: Flink I

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

Thanks for driving this effort, Yangze! The proposal overall LGTM. Other from the throughput enhancement in the OLAP scenario, the separation of leader election/discovery services and the metadata persistence services will also make the HA impl clearer and easier to maintain. Just a minor commen

[jira] [Created] (FLINK-33946) RocksDb sets setAvoidFlushDuringShutdown to true to speed up Task Cancel

Yue Ma created FLINK-33946: -- Summary: RocksDb sets setAvoidFlushDuringShutdown to true to speed up Task Cancel Key: FLINK-33946 URL: https://issues.apache.org/jira/browse/FLINK-33946 Project: Flink

[jira] [Created] (FLINK-33945) Cleanup usage of deprecated org.apache.flink.table.api.dataview.ListView#ListView

Jacky Lau created FLINK-33945: - Summary: Cleanup usage of deprecated org.apache.flink.table.api.dataview.ListView#ListView Key: FLINK-33945 URL: https://issues.apache.org/jira/browse/FLINK-33945 Project:

[jira] [Created] (FLINK-33944) Apache Flink: Process to restore more than one job on job manager startup from the respective savepoints

Vijay created FLINK-33944: - Summary: Apache Flink: Process to restore more than one job on job manager startup from the respective savepoints Key: FLINK-33944 URL: https://issues.apache.org/jira/browse/FLINK-33944

[jira] [Created] (FLINK-33943) Apache flink: Issues after configuring HA (using zookeeper setting)

Vijay created FLINK-33943: - Summary: Apache flink: Issues after configuring HA (using zookeeper setting) Key: FLINK-33943 URL: https://issues.apache.org/jira/browse/FLINK-33943 Project: Flink Issue

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

Awesome! Big +1. Regards, Nicholas Jiang On 2023/12/07 03:24:59 Leonard Xu wrote: > Dear Flink devs, > > As you may have heard, we at Alibaba (Ververica) are planning to donate CDC > Connectors for the Apache Flink project[1] to the Apache Flink community. > > CDC Connectors for Apache Flink

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

> > Configuration has a `public T get(ConfigOption option)` method. > Could we remove all `Xxx getXxx(ConfigOption configOption)` methods? Note: all `public void setXxx(ConfigOption key, Xxx value)` methods > can be replaced with `public Configuration set(ConfigOption option, > T value)` as we

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

> > These features don't have a public option, but they work. I'm not sure > whether these features are used by some advanced users. > Actually, I think some of them are valuable! For example: > > - ConfigConstants.YARN_CONTAINER_START_COMMAND_TEMPLATE > allows users to define the start command o

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

In addition to what is written in FLIP, I found some methods of Configuration are not necessary. And I wanna hear more thoughts from all of you. Configuration has a `public T get(ConfigOption option)` method. Could we remove all `Xxx getXxx(ConfigOption configOption)` methods? Such as: - public i

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

Thanks Xintong for the quick feedback and the good suggestions! > 1. I think the default value for `TASK_MANAGER_LOG_PATH_KEY` should be "no > default". We can explain in the description that if not configured, > `System.getProperty("log.file")` will be used, but that is not a default > value. Ex

Re: [DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

Thanks for the efforts, Rui and Xuannan. I think it's a good idea to migrate string-key configuration accesses to ConfigOption-s in general. I have a few suggestions / questions regarding the FLIP. 1. I think the default value for `TASK_MANAGER_LOG_PATH_KEY` should be "no default". We can explai

Re: [SUMMARY] Flink 1.19 Release Sync 12/12/2023

Hi devs, Happy holidays to you all! Since most RMs are not available for today‘s meeting, and there were no new attendees after waiting 10min at google meet, today's sync was skipped. Here's the brief summary for 1.19 release: - *Features & issues tracking* There're 5 weeks until the feature f

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

Hi Zakelly, Thanks for driving the discussion. 1. >> But I'm not so sure since there is only one savepoint-related option. Maybe someone else could share some thoughts here. How about state.savepoints.dir -> execution.checkpointing.savepoint.dir state.checkpoints.dir -> execution.checkpointing.c

[jira] [Created] (FLINK-33942) DelegatingConfiguration misses the perfix for some methods

Rui Fan created FLINK-33942: --- Summary: DelegatingConfiguration misses the perfix for some methods Key: FLINK-33942 URL: https://issues.apache.org/jira/browse/FLINK-33942 Project: Flink Issue Type:

[Discuss] FLIP-407: Improve Flink Client performance in interactive scenarios

Hi devs, I'm opening this thread to discuss FLIP-407: Improve Flink Client performance in interactive scenarios. The POC test results and design doc can be found at: FLIP-407

[DISCUSS] FLIP-405: Migrate string configuration key to ConfigOption

Hi all, Xuannan(cced) and I would like to start a discussion on FLIP-405: FLIP-405: Migrate string configuration key to ConfigOption[1]. As Flink progresses to 2.0, we want to enhance the user experience with the existing configuration. In FLIP-77, Flink introduced ConfigOption with DataType and