Re: [ANNOUNCE] New Apache Flink PMC Member - Qingsheng Ren

2023-04-22 Thread Rui Fan
Congratulations, Qingsheng! Best, Rui Fan On Sun, Apr 23, 2023 at 10:23 AM Benchao Li wrote: > Congratulations, Qingsheng! > > yuxia 于2023年4月23日周日 09:24写道: > > > Congratulations, Qingsheng! > > > > Best regards, > > Yuxia > > > > - 原始邮件

Re: [ANNOUNCE] New Apache Flink PMC Member - Leonard Xu

2023-04-22 Thread Rui Fan
Congratulations, Leonard! Best, Rui Fan On Sun, Apr 23, 2023 at 10:24 AM Benchao Li wrote: > Congratulations, Leonard! > > Lincoln Lee 于2023年4月23日周日 09:12写道: > > > Congratulations, Leonard! > > > > Best, > > Lincoln Lee > > > > > > Yuxin

Re: [VOTE] FLIP-288: Enable Dynamic Partition Discovery by Default in Kafka Source

2023-04-24 Thread Rui Fan
+1 (binding) Best, Rui Fan On Tue, Apr 25, 2023 at 10:06 AM Biao Geng wrote: > +1 (non-binding) > Best, > Biao Geng > > Martijn Visser 于2023年4月24日周一 20:20写道: > > > +1 (binding) > > > > On Mon, Apr 24, 2023 at 4:10 AM Feng Jin wrote: > > >

Re: [DISCUSS] FLIP-306: Unified File Merging Mechanism for Checkpoints

2023-05-04 Thread Rui Fan
hard to give a default value. For jobs with many tasks in a TM, it may be useful for file merging. However, it doesn't work well for jobs with a small number of tasks in a TM. I prefer just adding the `max-file-pool-size`, and the `pool size = number of tasks / max-file-pool-size`. WDYT? Maybe I

Re: [DISCUSS] FLIP-306: Unified File Merging Mechanism for Checkpoints

2023-05-05 Thread Rui Fan
Hi Zakelly, Thanks for the clarification! Currently, I understand what you mean, and LGTM. Best, Rui Fan On Fri, May 5, 2023 at 12:27 PM Zakelly Lan wrote: > Hi all, > > @Yun Tang and I have an offline discussion, and we agreed that: > > 1. The design of this FLIP is prett

Re: [VOTE] FLIP-306: Unified File Merging Mechanism for Checkpoints

2023-05-09 Thread Rui Fan
Thanks for driving this proposal, Zakelly. +1(binding) Best, Rui Fan On Wed, May 10, 2023 at 11:04 AM Hangxiang Yu wrote: > Hi Zakelly. > Thanks for driving this. > +1 (no-binding) > > On Wed, May 10, 2023 at 10:52 AM Yuan Mei wrote: > > > Thanks for driving t

Re: [VOTE] FLIP-305: Support atomic for CREATE TABLE AS SELECT(CTAS) statement

2023-06-12 Thread Rui Fan
+1 (binding) Best, Rui Fan On Mon, Jun 12, 2023 at 19:58 liu ron wrote: > +1 (no-binding) > > Best, > Ron > > Jing Ge 于2023年6月12日周一 19:33写道: > > > +1(binding) Thanks! > > > > Best regards, > > Jing > > > > On Mon, Jun 12, 2023 at 12:01

Re: [VOTE] FLIP-311: Support Call Stored Procedure

2023-06-12 Thread Rui Fan
+1 (binding) Best, Rui Fan On Mon, Jun 12, 2023 at 22:20 Benchao Li wrote: > +1 (binding) > > yuxia 于2023年6月12日周一 17:58写道: > > > Hi everyone, > > Thanks for all the feedback about FLIP-311: Support Call Stored > > Procedure[1]. Based on the discussion [2], we

Re: [VOTE] FLIP-295: Support lazy initialization of catalogs and persistence of catalog configurations

2023-06-14 Thread Rui Fan
+1(binding) Best, Rui Fan On Wed, Jun 14, 2023 at 16:24 Hang Ruan wrote: > +1 (non-binding) > > Thanks for Feng driving it. > > Best, > Hang > > Feng Jin 于2023年6月14日周三 10:36写道: > > > Hi everyone > > > > Thanks for all the feedback about the F

Re: [VOTE] FLIP-246: Dynamic Kafka Source (originally Multi Cluster Kafka Source)

2023-06-22 Thread Rui Fan
+1 (binding) +1 for DynamicKafkaSource Best, Rui Fan On Wed, Jun 21, 2023 at 6:57 PM Thomas Weise wrote: > +1 (binding) > > > On Mon, Jun 19, 2023 at 8:09 AM Ryan van Huuksloot > wrote: > > > +1 (non-binding) > > > > +1 for DynamicKafkaSource > >

Re: [VOTE] FLIP-324: Introduce Runtime Filter for Flink Batch Jobs

2023-06-23 Thread Rui Fan
+1(binding), thanks for driving this improvement. Best, Rui Fan On Sat, Jun 24, 2023 at 4:55 AM Jing Ge wrote: > +1(binding) > > Best Regards, > Jing > > On Fri, Jun 23, 2023 at 5:50 PM Lijie Wang > wrote: > > > Hi all, > > > > Thanks for all the feed

Re: Re: [ANNOUNCE] Apache Flink has won the 2023 SIGMOD Systems Award

2023-07-04 Thread Rui Fan
Congratulations! Best, Rui Fan On Tue, Jul 4, 2023 at 2:08 PM Zhu Zhu wrote: > Congratulations everyone! > > Thanks, > Zhu > > Hang Ruan 于2023年7月4日周二 14:06写道: > > > > Congratulations! > > > > Best, > > Hang > > > >

Re: [ANNOUNCE] Flink 1.18 Feature Freeze Extended until July 24th, 2023

2023-07-06 Thread Rui Fan
Thanks for the update, and thank you for your efforts for the 1.18 release! Best, Rui Fan On Thu, Jul 6, 2023 at 2:40 PM Qingsheng Ren wrote: > Hi devs, > > Recently we collected some feedback from developers, and in order to give > more time for polishing some important features

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

2023-07-17 Thread Rui Fan
+1(binding) Best, Rui Fan On Tue, Jul 18, 2023 at 12:04 PM Dong Lin wrote: > Hi all, > > We would like to start the vote for FLIP-309: Support using larger > checkpointing interval when source is processing backlog [1]. This FLIP was > discussed in this thread [2]. > >

Re: Kubernetes Operator 1.6.0 release planning

2023-07-19 Thread Rui Fan
Thanks Gyula for driving this release. +1 for the timeline Best, Rui Fan On Wed, Jul 19, 2023 at 11:03 PM Gyula Fóra wrote: > Hi Devs! > > Based on our release schedule, it is about time for the next Flink K8s > Operator minor release. > > There are still some minor work ite

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-24 Thread Rui Fan
Congratulations, Yong Fang Best, Rui Fan On Mon, Jul 24, 2023 at 3:42 PM Matt Wang wrote: > Congratulations, Yong Fang > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Feng Jin | > | Date | 07/24/2023 15:27 | > | To | |

Re: [VOTE] FLIP-322 Cooldown period for adaptive scheduler. Second vote.

2023-07-29 Thread Rui Fan
/viewpage.action?pageId=263424711 Best Rui Fan On Fri, 28 Jul 2023 at 19:21, Martijn Visser wrote: > +1 (binding) > > On Fri, Jul 14, 2023 at 11:59 AM Prabhu Joseph > > wrote: > > > *+1 (non-binding)* > > > > Thanks for working on this. We have seen good improvement

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.0, release candidate #1

2023-07-29 Thread Rui Fan
+1 (non-binding) - Compiled and tested the source code via mvn verify - Verified the signatures - Downloaded the image : docker pull ghcr.io/apache/flink-kubernetes-operator:e7045a6 - Deployed helm chart to test cluster - Ran example job Best, Rui Fan On Thu, Jul 27, 2023 at 10:53 PM Gyula Fóra

[DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-01 Thread Rui Fan
Hi all, I and Samrat(cc'ed) created the FLIP-334[1] to decoupling the autoscaler and kubernetes. Currently, the flink-autoscaler is tightly integrated with Kubernetes. There are compelling reasons to extend the use of flink-autoscaler to more types of Flink jobs: 1. With the recent merge of the E

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-01 Thread Rui Fan
t will take more time if I develop all the tests during POC. So I removed these tests in my POC. These tests will be completed in the final PR, and the test is very useful for less bugs. Best, Rui Fan On Tue, Aug 1, 2023 at 10:10 PM Maximilian Michels wrote: > Hi Rui, > > Thanks for the

Re: [VOTE] FLIP-333: Redesign Apache Flink website

2023-08-03 Thread Rui Fan
+1(binding), thanks for driving this proposal, it's cool ! Best, Rui Fan On Thu, Aug 3, 2023 at 6:06 PM Jing Ge wrote: > +1, thanks for driving it! > > Best regards, > Jing > > On Thu, Aug 3, 2023 at 4:49 AM Mohan, Deepthi > wrote: > > > Hi, > > > &

Re: [ANNOUNCE] New Apache Flink Committer - Weihua Hu

2023-08-03 Thread Rui Fan
Congratulations Weihua, well deserved! Best, Rui Fan On Fri, Aug 4, 2023 at 11:19 AM Xintong Song wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce Weihua Hu as a new Flink > Committer! > > Weihua has been consistently contributing to the pro

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

2023-08-03 Thread Rui Fan
Congratulations Matthias, well deserved! Best, Rui Fan On Fri, Aug 4, 2023 at 11:30 AM Leonard Xu wrote: > Congratulations, Matthias. > > Well deserved ^_^ > > Best, > Leonard > > > > On Aug 4, 2023, at 11:18 AM, Xintong Song wrote: > > > > Hi everyo

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-03 Thread Rui Fan
ide a series of interfaces to query state. What do you think? And looking forward to more thoughts from the community, thanks! Best, Rui Fan On Tue, Aug 1, 2023 at 11:47 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Max, > > Thanks for your quick response! > > > 1. Handle state i

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-06 Thread Rui Fan
t; > > > > We could also remove the entire recommended parallelism logic from the > > interface and make it internal to the implementation somehow because it's > > not very nice in the current form. > > > > Cheers, > > Gyula > > > > On F

Re: [ANNOUNCE] New Apache Flink Committer - Yanfei Lei

2023-08-07 Thread Rui Fan
Congratulations Yanfei! Best, Rui On Mon, Aug 7, 2023 at 2:56 PM Yuan Mei wrote: > On behalf of the PMC, I'm happy to announce Yanfei Lei as a new Flink > Committer. > > Yanfei has been active in the Flink community for almost two years and has > played an important role in developing and maint

Re: [ANNOUNCE] New Apache Flink Committer - Hangxiang Yu

2023-08-07 Thread Rui Fan
Congratulations Hangxiang! Best, Rui On Mon, Aug 7, 2023 at 2:58 PM Yuan Mei wrote: > On behalf of the PMC, I'm happy to announce Hangxiang Yu as a new Flink > Committer. > > Hangxiang has been active in the Flink community for more than 1.5 years > and has played an important role in developin

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-08 Thread Rui Fan
processed. What is the main purpose of this solution? > > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Rui Fan<1996fan...@gmail.com> | > | Date | 08/7/2023 11:34 | > | To | | > | Cc | m...@apache.org , > Gyula Fóra | &

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.0, release candidate #2

2023-08-14 Thread Rui Fan
Thanks Gyula for the release! +1 (non-binding) - Compiled and tested the source code via mvn verify - Verified the signatures - Downloaded the image - Deployed helm chart to test cluster - Ran example job Best, Rui On Mon, Aug 14, 2023 at 3:58 PM Gyula Fóra wrote: > +1 (binding) > > Verified:

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-21 Thread Rui Fan
at 11:55 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi Matt Wang, > > Thanks for your discussion here. > > > it is recommended to unify the descriptions of AutoScalerHandler > > and AutoScalerEventHandler in the FLIP > > Good catch, I have updated all Auto

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Hi Jing, Thanks for the effort and update! It means the PR of flink-1.19 can be merged to master branch, right? Best, Rui On Wed, Aug 23, 2023 at 9:29 PM Jing Ge wrote: > Hi devs, The release-1.18 branch has been forked out from the master > branch, with commit ID cfa4a9c35563fd8a5973ec2f3525

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Thanks for the quick response! Best, Rui On Wed, Aug 23, 2023 at 9:46 PM Jing Ge wrote: > yes please > > On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Jing, > > > > Thanks for the effort and update! > > > > It me

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-09-05 Thread Rui Fan
ess). > > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Rui Fan<1996fan...@gmail.com> | > | Date | 08/21/2023 17:41 | > | To | | > | Cc | Maximilian Michels , > Gyula Fóra , > Matt Wang | > | Subject | Re: [DISCU

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Rui Fan
Hi Gyula, +1 for this proposal. The current GC metric is really unfriendly. I have a concern with your proposed rate metric: the rate is perSecond instead of per minute. I'm unsure whether it's suitable for GC metric. There are two reasons why I suspect perSecond may not be well compatible with

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Rui Fan
#x27;t think we run into this problem in practice and we can keep the > metric aligned with other time rate metrics like busyTimeMsPerSec etc. > > Cheers, > Gyula > > On Wed, Sep 6, 2023 at 4:55 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Gyula, > > >

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-09-06 Thread Rui Fan
right, is to have an alternative implementation in addition to > > Kubernetes. YARN or a standalone mode implementation were already > > mentioned. Ultimately, this is the reason we are doing the > > refactoring. Without a new implementation, it becomes harder to > > just

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-09-06 Thread Rui Fan
through with the FLIP and we can > definitely help review code changes and build on the new design. > > -Max > > > On Wed, Sep 6, 2023 at 11:26 AM Rui Fan <1996fan...@gmail.com> wrote: > > > > Hi Max, > > > > As the FLIP mentioned, we have the pla

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-10 Thread Rui Fan
Thanks Zhanghao driving this FLIP, adding the port in Web UI seems good to me. Hi Shammon and Zhanghao, I would like to clarify the difference between Public Interfaces in FLIP and @Public in code. As I understand, the `Public Interfaces in FLIP` means these changes will be used in user side, su

[VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

2023-09-12 Thread Rui Fan
Hi all, Thanks for all the feedback about the FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler[1]. This FLIP was discussed in [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours (until Sep 16th 11:00 UTC+8) unless there is an objec

Re: [VOTE] FLIP-361: Improve GC Metrics

2023-09-13 Thread Rui Fan
+1(binding) Best, Rui On Wed, Sep 13, 2023 at 9:16 PM Gyula Fóra wrote: > Hi All! > > Thanks for all the feedback on FLIP-361: Improve GC Metrics [1][2] > > I'd like to start a vote for it. The vote will be open for at least 72 > hours unless there is an objection or insufficient votes. > > Che

Re: [VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

2023-09-15 Thread Rui Fan
+1(binding) Best, Rui On Thu, Sep 14, 2023 at 10:06 AM liu ron wrote: > +1(non-binding) > > Best, > Ron > > Dong Lin 于2023年9月14日周四 09:01写道: > > > Thank you Rui for the proposal. > > > > +1 (binding) > > > > On Wed, Sep 13, 2023 at 10:52 AM

[RESULT][VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

2023-09-15 Thread Rui Fan
(binding) - Dong Lin (binding) - Rui Fan(binding) - Ahmed Hamdy - ConradJam - Matt Wang - Ferenc Csaky - Zhanghao Chen - Feng Jin - Samrat Deb - Ron Liu There is no disapproving vote. [1] https://cwiki.apache.org/confluence/x/x4qzDw [2] https://lists.apache.org/thread/3wmhhqgkkg1l7ncxnzwqnjqyhqz545gl

Re: [VOTE] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-17 Thread Rui Fan
A gentle reminder about the location naming. The naming of location is a little unclear, but I can't think of any other better naming. So I +1(binding) first. Ping @Jing Ge to help double check the name again. Sorry for mentioning naming in the VOTE thread, I didn't know this VOTE would be so e

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-17 Thread Rui Fan
ghao > > > > > > Since the meaning of "host" is not aligned, it seems good for me to > remove > > > it in the future release. > > > > > > Best, > > > Weihua > > > > > > > > > On Mon, Sep 11, 2023 at 11:48 AM Chen Zhanghao < > zha

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-18 Thread Rui Fan
rate more on what concept you think one might confuse it > with? > > Best, > Yangze Guo > > On Mon, Sep 18, 2023 at 12:07 PM Rui Fan <1996fan...@gmail.com> wrote: > > > > Hi Zhanghao, > > > > > Use a field named "location" (already use

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-18 Thread Rui Fan
; > > top level service that aligns with the top level endpoint. > > > > > > WDYT? > > > > > > Best regards, > > > Jing > > > > > > > > > On Mon, Sep 11, 2023 at 6:01 AM Weihua Hu > wrote: > > > > > >

Re: [VOTE] FLIP-327: Support switching from batch to stream mode to improve throughput when processing backlog data

2023-09-22 Thread Rui Fan
+1(binding), thanks for driving this proposal. Best, Rui On Fri, Sep 22, 2023 at 5:16 PM Jing Ge wrote: > +1(binding) Thanks! > > Best regards, > Jing > > On Fri, Sep 22, 2023 at 8:08 AM Dong Lin wrote: > > > Hi all, > > > > We would like to start the vote for FLIP-327: Support switching from

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-09-27 Thread Rui Fan
Hi Zhu Zhu, Thanks for your feedback here! You are right, user needs to set 2 options: - cluster.evenly-spread-out-slots=true - slot.sharing-strategy=TASK_BALANCED_PREFERRED Update it to one option is useful at user side, so `taskmanager.load-balance.mode` sounds good to me. I want to check some

Re: [VOTE] FLIP-314: Support Customized Job Lineage Listener

2023-09-28 Thread Rui Fan
+1(binding) Best, Rui On Thu, 28 Sep 2023 at 14:41, Chen Zhanghao wrote: > +1 (non-binding), thanks for driving this. > > Best, > Zhanghao Chen > > 发件人: Shammon FY > 发送时间: 2023年9月25日 13:28 > 收件人: dev > 主题: [VOTE] FLIP-314: Support Customized Job Lineage Listen

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-01 Thread Rui Fan
t are the benefits of unifying > these two into one option? Also, do we want to share the same > option between SlotPool in JM and SlotManager in RM? This sounds a bit > strange. > > Best, > Shammon FY > > > > On Thu, Sep 28, 2023 at 12:08 PM Rui Fan <1996fan...@gmail.

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-01 Thread Rui Fan
d looking forward to your feedback, thanks! Best, Rui On Sun, Oct 1, 2023 at 5:52 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Yangze, > > Thanks for your feedback! > > > 1. Is it possible for the SlotPool to get the slot allocation results > > from the SlotManager in

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-07 Thread Rui Fan
Hi Shammon, IIUC, you want more flexibility in controlling the two-phase strategy, right? > I want this because we would like to add a new slot to TM strategy such as SLOTS_NUM in the future for OLAP to improve the performance for olap jobs, which will use TASKS strategy for task to slot. cc Guoy

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-07 Thread Rui Fan
in the three TM will be > (taskmanager.numberOfTaskSlots=3): > TM1: 3 slot > TM2: 2 slot > TM3: 2 slot > > Best, > Yangze Guo > > On Sun, Oct 1, 2023 at 6:14 PM Rui Fan <1996fan...@gmail.com> wrote: > > > > Hi Shammon, > > > > Thanks f

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-07 Thread Rui Fan
oducing another waiting mechanism. I > think it would be better to address this issue within the scope of > this FLIP. However, I don't have a strong opinion on it, it depends on > your bandwidth. > > > Best, > Yangze Guo > > On Sat, Oct 7, 2023 at 4:16 PM Rui Fan <1996fan

Re: [DISCUSS] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

2023-10-07 Thread Rui Fan
Thanks to Yangze driving this proposal. `env.java.opts.xxx` is already supported for client, historyserver, jobmanager and taskmanager. And it's very useful for troubleshooting. So +1 for `env.java.opts.sql-gateway`. I have a minor question: doesn't the `env.java.opts.all` support sql-gateway? If

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
Thanks Yun and Yu for driving this proposal! It's very useful for troubleshooting why the CPU usage is high. +1 Best, Rui On Mon, Oct 9, 2023 at 7:21 PM Zhanghao Chen wrote: > Hi Yun and Yu, > > Thanks for driving this. This would definitely help users identify > performance bottlenecks, espec

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
Hi Jing, > 1. will it replace the current flame graph, i.e. the current flame graph will be deprecated and removed? I think the current flame graph cannot be removed. As a core contributor to the current flame graph, and I use it almost every week. I would like to clarify the difference between

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-09 Thread Rui Fan
during the SlotPool's resource >> requirement declaration to the ResourceManager, similar to the >> checkResourceRequirementsWithDelay in the SlotManager. In most cases, >> this delay would allow the SlotManager to see all resource >> requirements, then it can allocate

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-10 Thread Rui Fan
s} = {cluster.evenly-spread-out-slots: > true, slot.sharing-strategy: LOCAL_INPUT_PREFERRED} > {taskmanager.load-balance.mode: Tasks} = {cluster.evenly-spread-out-slots: > true, slot.sharing-strategy: TASK_BALANCED_PREFERRED} > > Thanks, > Zhu > > Rui Fan <1996fan...

Re: [VOTE] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

2023-10-10 Thread Rui Fan
+1(binding) Best, Rui On Wed, Oct 11, 2023 at 10:07 AM Yangze Guo wrote: > Hi everyone, > > I'd like to start the vote of FLIP-374 [1]. This FLIP is discussed in > the thread [2]. > > The vote will be open for at least 72 hours. Unless there is an > objection, I'll try to close it by October 16

Re: [VOTE] FLIP-366: Support standard YAML for FLINK configuration

2023-10-12 Thread Rui Fan
+1(binding) Best, Rui Fan On Fri, Oct 13, 2023 at 10:12 AM Junrui Lee wrote: > Hi all, > > Thank you to everyone for the feedback on FLIP-366[1]: Support standard > YAML for FLINK configuration in the discussion thread [2]. > I would like to start a vote for it. The vote will

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread Rui Fan
One minor comment: In general, the generic java profiler includes memory analysis, cpu, thread, deadlock, etc. The FLIP title is java profiler, but the FLIP just supports flamegraph at process level. So the `powerful java profiler` title may not be suitable. Would you mind updating the FLIP title?

Re: [ANNOUNCE] New Apache Flink Committer - Ron Liu

2023-10-15 Thread Rui Fan
Congratulations Ron ! Best, Rui On Mon, Oct 16, 2023 at 10:05 AM Lijie Wang wrote: > Congratulations Ron ! > > Best, > Lijie > > Samrat Deb 于2023年10月16日周一 10:03写道: > > > Congratulations Ron Liu :) > > > > On Mon, 16 Oct 2023 at 7:29 AM, tison wrote: > > > > > Congrats! Glad to see more and mo

Re: [ANNOUNCE] New Apache Flink Committer - Jane Chan

2023-10-15 Thread Rui Fan
Congratulations Jane! Best, Rui On Mon, Oct 16, 2023 at 10:15 AM yu zelin wrote: > Congratulations! > > Best, > Yu Zelin > > > 2023年10月16日 09:58,Jark Wu 写道: > > > > Hi, everyone > > > > On behalf of the PMC, I'm very happy to announce Jane Chan as a new Flink > > Committer. > > > > Jane starte

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-16 Thread Rui Fan
to two parts: slot to TM and task > to > > slot. > > 1. Slot to TM is guaranteed by SlotManager in ResourceManager > > 2. Task to slot is guaranteed by the slot pool in JM > > > > These two are completely independent, what are the benefits of unifying > > these tw

[DISCUSS] FLIP-364: Improve the restart-strategy

2023-10-16 Thread Rui Fan
Hi all, I would like to start a discussion on FLIP-364: Improve the restart-strategy[1] As we know, the restart-strategy is critical for flink jobs, it mainly has two functions: 1. When an exception occurs in the flink job, quickly restart the job so that the job can return to the running state.

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-16 Thread Rui Fan
-profiler's regular modes of operation & the other > > most-common configuration options. From my own experience, `cpu` > (requiring > > perf_events) is a bit more accurate than `itimer`, and if I recall, and > > samples once per thread. `wall` is very useful to debug b

Re: [VOTE] FLIP-375: Built-in cross-platform powerful java profiler

2023-10-17 Thread Rui Fan
+1(binding) Best, Rui On Tue, Oct 17, 2023 at 3:52 PM Yu Chen wrote: > Hi all, > > Thank you to everyone for the feedback and detailed comments on > FLIP-375[1]. > Based on the discussion thread [2], I think we are ready to take a vote to > contribute this to Flink. > I'd like to start a vote f

Re: [DISCUSS] Creating Kubernetes Operator release 1.6.1

2023-10-18 Thread Rui Fan
Hi Gyula, Thank you for driving this discussion! This release seems good to me, I have a question: I see some bugfix commits have been merged into the 1.6-release branch. Does it already contain all recent bugfix commits? Also, you said in the `Kubernetes Operator 1.6.0 release planning`[1]: >

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-10-19 Thread Rui Fan
ugh, > > but of course allowing users to change is valuable. > > > > So, overall +1. > > > > Cheers, > > > > Konstantin > > > > Am Di., 17. Okt. 2023 um 07:11 Uhr schrieb Rui Fan <1996fan...@gmail.com > >: > > > > > Hi all,

Re: [ANNOUNCE] The Flink Speed Center and benchmark daily run are back online

2023-10-20 Thread Rui Fan
Thanks for your effort! It's very useful when some new commits affect performance. Best, Rui On Fri, Oct 20, 2023 at 4:42 PM Yanfei Lei wrote: > Thanks for your hard work! > Looking forward to the daily monitoring being available again soon. > > Best, > Yanfei > > Yuan Mei 于2023年10月20日周五 16:19

Re: [DISCUSS] Creating Kubernetes Operator release 1.6.1

2023-10-20 Thread Rui Fan
ut we should focus on critical fixes / regressions to make > the changes minimal given that the 1.7.0 is not so far anyway :) > > Cheers, > Gyula > > > On Thu, Oct 19, 2023 at 8:20 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Gyula, > > > > Thank yo

Re: [VOTE] Release 1.18.0, release candidate #3

2023-10-21 Thread Rui Fan
+1(non-binding) - Downloaded artifacts from dist[1] - Verified SHA512 checksums - Verified GPG signatures - Build the source with java-1.8 and verified the licenses together - Verified web PR [1] https://dist.apache.org/repos/dist/dev/flink/flink-1.18.0-rc3/ Best, Rui On Fri, Oct 20, 2023 at 10

[VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-21 Thread Rui Fan
Hi Everyone, Please review and vote on the release candidate #1 for the version 1.6.1 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of th

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-21 Thread Rui Fan
, 2023 at 7:33 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Everyone, > > Please review and vote on the release candidate #1 for the version 1.6.1 of > Apache Flink Kubernetes Operator, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the relea

Re: [VOTE] FLIP-370: Support Balanced Tasks Scheduling

2023-10-22 Thread Rui Fan
+1(binding) Thanks to Yuepeng and to everyone who participated in the discussion! Best, Rui On Mon, Oct 23, 2023 at 11:55 AM Roc Marshal wrote: > Hi all, > > Thanks for all the feedback on FLIP-370[1][2]. > I'd like to start a vote for FLIP-370. The vote will last for at least 72 > hours (Oct

Re: Request to release flink 1.6.3

2023-10-24 Thread Rui Fan
Hi Vikas, Thanks for your feedback! Do you mean flink 1.16.3 instead of 1.6.3? The 1.16.2 and 1.17.1 were released on 2023-05-25, it’s been 5 months. And the flink community has fixed many bugs in the past 5 months. Usually, there is a fix(minor) version every three or four months, so I propose

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-25 Thread Rui Fan
Imports.java:187) > > Thanks, > Thomas > > > On Sat, Oct 21, 2023 at 7:35 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Everyone, > > > > Please review and vote on the release candidate #1 for the version 1.6.1 > of > > Apache Flink Ku

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-26 Thread Rui Fan
r/flink-kubernetes-operator/139454270?tag=51eeae1 > > > And notice that it does not have a description . Is this correct? > > > > > > 1. I get this in the gpg verification . Is this ok? > > > > > > > > > gpg --verify flink-kubernetes-operator

[RESULT] [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-26 Thread Rui Fan
I'm happy to announce that we have unanimously approved this release. There are 7 approving votes, 4 of which are binding: * Gyula Fora (binding) * Thomas Weise (binding) * Marton Balassi (binding) * Maximilian Mixhels (binding) * Mate Czagany (non-binding) * Samrat Deb (non-binding) * Ru

Re: [ANNOUNCE] Apache Flink 1.18.0 released

2023-10-26 Thread Rui Fan
Thanks for the great work! Best, Rui On Fri, Oct 27, 2023 at 10:03 AM Paul Lam wrote: > Finally! Thanks to all! > > Best, > Paul Lam > > > 2023年10月27日 03:58,Alexander Fedulov 写道: > > > > Great work, thanks everyone! > > > > Best, > > Alexander > > > > On Thu, 26 Oct 2023 at 21:15, Martijn Viss

[ANNOUNCE] Apache Flink Kubernetes Operator 1.6.1 released

2023-10-30 Thread Rui Fan
://hub.docker.com/r/apache/flink-kubernetes-operator The full release notes are available in Jira: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353784 We would like to thank all contributors of the Apache Flink community who made this release possible! Regards,

Re: [DISCUSS] Kubernetes Operator 1.7.0 release planning

2023-10-31 Thread Rui Fan
Thanks Gyula for driving this release! I'd like to check with you and community, could we postpone the code freeze by a week? I'm developing the FLINK-33099[1], and the prod code is done. I need some time to develop the tests. I hope this feature is included in 1.7.0 for two main reasons: 1. We

Re: Request to release flink 1.6.3

2023-10-31 Thread Rui Fan
helping it? Thanks~ Best, Rui On Tue, 31 Oct 2023 at 21:42, vikas patil wrote: > Hello Rui, > > Do we need more votes for this or are we good to go with the release of > 1.6.3 ? Please let me know. Thanks. > > -Vikas > > On Tue, Oct 24, 2023 at 9:27 AM Rui Fan <

Re: [DISCUSS] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

2023-11-01 Thread Rui Fan
Thanks Junrui for driving this proposal! ConfigOption is easy to use for flink users, easy to manage options for flink platform maintainers, and easy to maintain for flink developers and flink community. So big +1 for this proposal! Best, Rui On Thu, Nov 2, 2023 at 10:10 AM Junrui Lee wrote:

Re: Request to release flink 1.6.3

2023-11-03 Thread Rui Fan
7;s move forward and make it. > > Best > Yun Tang > ________ > From: Rui Fan <1996fan...@gmail.com> > Sent: Tuesday, October 31, 2023 23:06 > To: dev@flink.apache.org > Subject: Re: Request to release flink 1.6.3 > > Thanks Vikas for the ask! >

Re: Request to release flink 1.6.3

2023-11-04 Thread Rui Fan
t; [1] https://lists.apache.org/thread/szq23kr3rlkm80rw7k9n95js5vqpsnbv > > > On Fri, Nov 3, 2023 at 7:38 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Thanks Yun and Yu, > > > > That would be great! Let's move forward! > > > > Best, > > Rui >

[DISCUSS] Release Flink 1.16.3

2023-11-05 Thread Rui Fan
Hi all, I would like to discuss creating a new 1.16 patch release (1.16.3). The last 1.16 release is over five months old, and since then, 50 tickets have been closed [1], of which 10 are blocker/critical [2]. Some of them are quite important, such as FLINK-32296 [3], FLINK-32548 [4] and FLINK-330

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-07 Thread Rui Fan
Hi Piotr, Thanks for driving this proposal! The trace reporter is useful to check a lot of duration monitors inside of Flink. I have some questions about this proposal: 1. I see the trace just supports Span? Does it support trace events? I'm not sure whether tracing events is reasonable for Trac

[DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-08 Thread Rui Fan
Hi all! I would like to start a discussion of FLIP-390: Support System out and err to be redirected to LOG or discarded[1]. In various production environments, either cloud native or physical machines, the disk space that Flink TaskManager can use is limited. In general, the flink users shouldn'

Re: [DISCUSS] Release Flink 1.16.3

2023-11-08 Thread Rui Fan
Hi All, Thank you for your feedback! As there are no other concerns or objections, and currently I am not aware of any unresolved blockers. I will kick off the release process and start preparing for the RC1 version from today. Best, Rui On Wed, Nov 8, 2023 at 4:23 PM ConradJam wrote: > +1 >

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-08 Thread Rui Fan
spans > are > > > not > > > > > supported. > > > > > This is done just for the sake of simplicity, and test out the > basic > > > > > functionality. But I think, > > > > > those currently missing features should be added at some p

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-08 Thread Rui Fan
err to be > > > redirected to LOG or discarded > > > Hi Rui, > > > > > > Thanks for the proposal. > > > > > > +1 I don't have any major comments :) > > > > > > One nit. In `SystemOutRedirectToLog` in this

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-09 Thread Rui Fan
> > ByteBuffer.wrap(array, ..., ...).equals(ByteBuffer.wrap(array2, ..., ...)) > > This shouldn't have overheads as far as I remember. > > Or implement your own loop? It shouldn't be more than a couple of lines. > > Best, > Piotrek > > czw., 9 lis 2023 o 06:43

Re: [VOTE] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

2023-11-10 Thread Rui Fan
+1(binding) Best, Rui On Fri, Nov 10, 2023 at 11:58 AM Junrui Lee wrote: > Hi everyone, > > Thank you to everyone for the feedback on FLIP-381: Deprecate configuration > getters/setters that return/set complex Java objects[1] which has been > discussed in this thread [2]. > > I would like to st

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-10 Thread Rui Fan
I'll start voting next Monday if there isn't any other comment. Best, Rui On Thu, Oct 19, 2023 at 6:59 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Konstantin and Max, > > Thanks for your feedback! > > Sorry, I forgot to mention the default value of > `rest

[VOTE] FLIP-364: Improve the restart-strategy

2023-11-12 Thread Rui Fan
Hi everyone, Thank you to everyone for the feedback on FLIP-364: Improve the restart-strategy[1] which has been discussed in this thread [2]. I would like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apach

[VOTE] Release 1.16.3, release candidate #1

2023-11-13 Thread Rui Fan
Hi everyone, Please review and vote on the release candidate #1 for the version 1.16.3, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-14 Thread Rui Fan
, which can be seen in the web UI. So how about to align the > > failure counting with the concurrent failures computing? This can make it > > more consistent and easier for understanding. It will require changes to > > the concurrent failures computing though, i.e. taking the backo

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-14 Thread Rui Fan
all out that making exponential-delay > the default is a good change. At Netflix, we have enabled this as the > default restart strategy 2 quarters ago and it has been working well. > Keeping it restarting indefinitely by default makes sense to me. > > On Mon, Oct 16, 2023 at 10:11 P

  1   2   3   4   5   6   >