[jira] [Created] (FLINK-33750) Remove deprecated config options.

2023-12-04 Thread Junrui Li (Jira)
Junrui Li created FLINK-33750: - Summary: Remove deprecated config options. Key: FLINK-33750 URL: https://issues.apache.org/jira/browse/FLINK-33750 Project: Flink Issue Type: Sub-task Co

[jira] [Created] (FLINK-33749) Remove deprecated getter method in Configuration.

2023-12-04 Thread Junrui Li (Jira)
Junrui Li created FLINK-33749: - Summary: Remove deprecated getter method in Configuration. Key: FLINK-33749 URL: https://issues.apache.org/jira/browse/FLINK-33749 Project: Flink Issue Type: Sub-t

Re: [DISCUSS] Resolve diamond inheritance of Sink.createWriter

2023-12-04 Thread weijie guo
Thanks Martijn for driving this! I'm +1 to reverting the breaking change. > For new functionality or changes we can make easily, we should switch to the decorative/mixin interface approach used successfully in the source and table interfaces. I like the way of switching to mixin interface. Bes

Re: [DISCUSS] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-04 Thread Paul Lam
Hi Lijie, Recovery for batch jobs is no doubt a long-awaited feature. Thanks for the proposal! I’m concerned about the multi-job scenario. In session mode, users could use web submission to upload and run jars which may produce multiple Flink jobs. However, these jobs may not be submitted at onc

Re: [DISCUSS] FLIP-335: Removing Flink's Time classes as part of Flink 2.0

2023-12-04 Thread weijie guo
Thanks for driving this FLIP, Matthias. I'm +1 for this. Best regards, Weijie Matthias Pohl 于2023年7月19日周三 22:32写道: > The overall Scala-related plan for this FLIP is to ignore the Scala API > because of FLIP-265. The CEP Java/Scala version > parity (through the PatternScalaAPICompletenessTest

Re: [DISCUSS] Resolve diamond inheritance of Sink.createWriter

2023-12-04 Thread Becket Qin
I am with Gyula about fixing the current SinkV2 API. A SinkV3 seems not necessary because we are not changing the fundamental design of the API. Hopefully we can modify the interface structure a little bit to make it similar to the Source while still keep the backwards compatibility. For example,

Subscribe Apache Flink development email.

2023-12-04 Thread aaron ai
Subscribe Apache Flink development email.

[jira] [Created] (FLINK-33748) Remove legacy TableSource/TableSink API in 2.0

2023-12-04 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-33748: -- Summary: Remove legacy TableSource/TableSink API in 2.0 Key: FLINK-33748 URL: https://issues.apache.org/jira/browse/FLINK-33748 Project: Flink Issue Type: Techni

Re: [DISCUSS] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-04 Thread Lijie Wang
Thanks for raising this valueable point, Xintong Supporting external shuffle service makes sense to me. In order to recover the internal states of ShuffleMaster after JM restarts, we will add the following 3 methods to ShuffleMaster: boolean supportsBatchSnapshot(); void snapshotState(Completable

[jira] [Created] (FLINK-33747) Remove Sink V1 API in 2.0

2023-12-04 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-33747: -- Summary: Remove Sink V1 API in 2.0 Key: FLINK-33747 URL: https://issues.apache.org/jira/browse/FLINK-33747 Project: Flink Issue Type: Technical Debt Co

[jira] [Created] (FLINK-33746) More precise dynamic selection of Hybrid Shuffle or AQE

2023-12-04 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33746: - Summary: More precise dynamic selection of Hybrid Shuffle or AQE Key: FLINK-33746 URL: https://issues.apache.org/jira/browse/FLINK-33746 Project: Flink Issue Type:

[jira] [Created] (FLINK-33745) Dynamically choose hybrid shuffle or AQE in a job level

2023-12-04 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33745: - Summary: Dynamically choose hybrid shuffle or AQE in a job level Key: FLINK-33745 URL: https://issues.apache.org/jira/browse/FLINK-33745 Project: Flink Issue Type:

[jira] [Created] (FLINK-33744) Hybrid shuffle avoids restarting the whole job when failover

2023-12-04 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33744: - Summary: Hybrid shuffle avoids restarting the whole job when failover Key: FLINK-33744 URL: https://issues.apache.org/jira/browse/FLINK-33744 Project: Flink Issue

[jira] [Created] (FLINK-33743) Support consuming multiple subpartitions on a single channel

2023-12-04 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33743: - Summary: Support consuming multiple subpartitions on a single channel Key: FLINK-33743 URL: https://issues.apache.org/jira/browse/FLINK-33743 Project: Flink Issue

Re: [DISCUSS] Release flink-connector-parent v1.01

2023-12-04 Thread Péter Váry
Hi Etienne, Which branch would you cut the release from? I find the flink-connector-parent branches confusing. If I merge a PR to the ci_utils branch, would it immediately change the CI workflow of all of the connectors? If I merge something to the release_utils branch, would it immediately cha

[jira] [Created] (FLINK-33742) Hybrid Shuffle should work well with Adaptive Query Execution

2023-12-04 Thread Yuxin Tan (Jira)
Yuxin Tan created FLINK-33742: - Summary: Hybrid Shuffle should work well with Adaptive Query Execution Key: FLINK-33742 URL: https://issues.apache.org/jira/browse/FLINK-33742 Project: Flink Issu

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread liu ron
Hi, xuyang Thanks for starting this FLIP discussion, currently there are two types of window aggregation in Flink SQL, namely legacy group window aggregation and window tvf aggregation, these two types of window aggregation are not fully aligned in behavior, which will bring a lot of confusion to

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread Shengkai Fang
Hi, Xuyang. Thanks for your great work. Big +1! After reading the FLIP, I have some questions. Please read the content below. 1. Support SESSION Window TVF Aggregation a. Could you give an example about the pass-through column? A session window may contain multiple rows, which value is select

[jira] [Created] (FLINK-33741) introduce stat dump period and statsLevel configuration

2023-12-04 Thread xiaogang zhou (Jira)
xiaogang zhou created FLINK-33741: - Summary: introduce stat dump period and statsLevel configuration Key: FLINK-33741 URL: https://issues.apache.org/jira/browse/FLINK-33741 Project: Flink Iss

Re:Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread Xuyang
Hi, Feng and David. Thank you very much to share your thoughts. This flip does not include the official exposure of these experimental conf to users. Thus there is not adetailed description of this part. However, in view that some technical users may have added these experimental conf in actu

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread David Anderson
The current situation (where we have both the legacy windows and the TVF-based windows) is confusing for users, and I'd like to see us move forward as rapidly as possible. Since the early fire, late fire, and allowed lateness features were never documented or exposed to users, I don't feel that we

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

2023-12-04 Thread Alexander Fedulov
Hi everyone, As we progress with the 1.19 release, which might potentially (although not likely) be the last in the 1.x line, I'd like to revive our discussion on the LTS support matter. There is a general consensus that due to breaking API changes in 2.0, extending bug fixes support by designatin

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-04 Thread Feng Jin
Hi xuyang, Thank you for initiating this proposal. I'm glad to see that TVF's functionality can be fully supported. Regarding the early fire, late fire, and allow lateness features, how will they be provided to users? The documentation doesn't seem to provide a detailed description of this part.

Re: Doc about cleaning savePoints and checkPoints

2023-12-04 Thread Rodrigo Meneses
Thanks for providing that information! On Fri, Dec 1, 2023 at 12:58 AM Zakelly Lan wrote: > Hi, Rodrigo > > It appears that the configurations you mentioned in your first question are > related to the flink kubernetes operator. Are you using the flink > kubernetes operator? > > In regards to th

Re: [DISCUSS] Contribute Flink Doris Connector to the Flink community

2023-12-04 Thread Leonard Xu
Hey, wudi I’ve added permission for you, hope to see your FLIP. Best, Leonard > 2023年12月1日 上午10:09,wudi <676366...@qq.com.INVALID> 写道: > > Thank you everyone. But I encountered a problem when creating FLIP. There is > no permission to create files in the Flink Improvement Proposals [1] space

Re: [VOTE] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-12-04 Thread Jingsong Li
+1 binding On Mon, Dec 4, 2023 at 10:33 PM Etienne Chauchot wrote: > > Correct, > > I forgot that in the bylaws, committer vote is binding for FLIPs thanks > for the reminder. > > Best > > Etienne > > Le 30/11/2023 à 10:43, Leonard Xu a écrit : > > +1(binding) > > > > Btw, @Etienne, IIRC, your vo

Re: [VOTE] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-12-04 Thread Etienne Chauchot
Correct, I forgot that in the bylaws, committer vote is binding for FLIPs thanks for the reminder. Best Etienne Le 30/11/2023 à 10:43, Leonard Xu a écrit : +1(binding) Btw, @Etienne, IIRC, your vote should be a binding one. Best, Leonard 2023年11月30日 下午5:03,Etienne Chauchot 写道: +1 (no

Re: [DISCUSS] FLIP-395: Migration to GitHub Actions

2023-12-04 Thread Chesnay Schepler
We could limit the (first) trial run to branches. PRs wouldn't be affected (avoiding a bunch of concerns about maybe blocking PRs and misleading people into thinking that CI is green), we'd have a better handle on how much capacity we are consuming, but contributors would still get the new set

[jira] [Created] (FLINK-33740) Introduce a flip to list the supported sql patterns

2023-12-04 Thread xuyang (Jira)
xuyang created FLINK-33740: -- Summary: Introduce a flip to list the supported sql patterns Key: FLINK-33740 URL: https://issues.apache.org/jira/browse/FLINK-33740 Project: Flink Issue Type: Improveme

Re: [DISCUSS] Resolve diamond inheritance of Sink.createWriter

2023-12-04 Thread Gyula Fóra
Hi All! Based on the discussion above, I feel that the most reasonable approach from both developers and users perspective at this point is what Becket lists as Option 1: Revert the naming change to the backward compatible version and accept that the names are not perfect (treat it as legacy). O

Re: [DISCUSS] FLIP-395: Deprecate Global Aggregator Manager

2023-12-04 Thread Zhanghao Chen
Hi Benchao, I think part of the reason is that a general global coordination mechanism is complex and hence subject to some internals changes in the future. Instead of directly exposing the full mechanism to users, it might be better to expose some well-defined subset of the feature set to user

[jira] [Created] (FLINK-33739) Document FLIP-364: Improve the exponential-delay restart-strategy

2023-12-04 Thread Rui Fan (Jira)
Rui Fan created FLINK-33739: --- Summary: Document FLIP-364: Improve the exponential-delay restart-strategy Key: FLINK-33739 URL: https://issues.apache.org/jira/browse/FLINK-33739 Project: Flink Issu

[jira] [Created] (FLINK-33738) Make exponential-delay restart-strategy the default restart strategy

2023-12-04 Thread Rui Fan (Jira)
Rui Fan created FLINK-33738: --- Summary: Make exponential-delay restart-strategy the default restart strategy Key: FLINK-33738 URL: https://issues.apache.org/jira/browse/FLINK-33738 Project: Flink I

[jira] [Created] (FLINK-33737) Merge multiple Exceptions into one attempt for exponential-delay restart-strategy

2023-12-04 Thread Rui Fan (Jira)
Rui Fan created FLINK-33737: --- Summary: Merge multiple Exceptions into one attempt for exponential-delay restart-strategy Key: FLINK-33737 URL: https://issues.apache.org/jira/browse/FLINK-33737 Project: Flin

[jira] [Created] (FLINK-33736) Update default value of exponential-delay.max-backoff and exponential-delay.backoff-multiplier

2023-12-04 Thread Rui Fan (Jira)
Rui Fan created FLINK-33736: --- Summary: Update default value of exponential-delay.max-backoff and exponential-delay.backoff-multiplier Key: FLINK-33736 URL: https://issues.apache.org/jira/browse/FLINK-33736

[jira] [Created] (FLINK-33735) FLIP-364: Improve the exponential-delay restart-strategy

2023-12-04 Thread Rui Fan (Jira)
Rui Fan created FLINK-33735: --- Summary: FLIP-364: Improve the exponential-delay restart-strategy Key: FLINK-33735 URL: https://issues.apache.org/jira/browse/FLINK-33735 Project: Flink Issue Type: Ne

[RESULT][VOTE] FLIP-364: Improve the exponential-delay restart-strategy

2023-12-04 Thread Rui Fan
Dear developers, FLIP-364 Improve the exponential-delay restart-strategy[1] has been accepted and voted through this thread [2]. The proposal received 6 approving votes, 5 of which are binding, and there is no disapproval. - Maximilian Michels (binding) - Zhanghao Chen - Zhu Zhu (binding) - Matt

Re: [VOTE] FLIP-364: Improve the restart-strategy

2023-12-04 Thread Rui Fan
Thanks to everyone who participated in this discussion and vote. And closing this vote thread, results will be announced in a separate email. Best, Rui On Sun, Dec 3, 2023 at 1:44 PM Jing Ge wrote: > +1(binding) > Thanks! > > Best regards, > Jing > > On Fri, Dec 1, 2023 at 9:29 AM Matthias Poh

Re: [ANNOUNCE] Apache Flink Kafka Connectors 3.0.2 released

2023-12-04 Thread Martijn Visser
Hi Gordon, Thanks for the release! I've pushed one hotfix [1], to make sure that the Flink documentation shows the correct version number for the Flink version it's compatible with. Best regards, Martijn [1] https://github.com/apache/flink-connector-kafka/commit/6c3d3d06689336f2fd37bfa5a3b17a5