Re: [VOTE] FLIP-531: Initiate Flink Agents as a new Sub-Project

2025-06-04 Thread Hangxiang Yu
+1 (binding) On Wed, Jun 4, 2025 at 5:21 PM Jiangang Liu wrote: > +1 (non-binding) > > Feng Jin 于2025年6月4日周三 16:12写道: > > > +1 (binding) > > > > Best, > > Feng Jin > > > > > > On Wed, Jun 4, 2025 at 1:51 PM Yun Tang wrote: > > > > > +1 (binding) > > > > > > > > > Best, > > > Yun Tang > > > ___

Re: [VOTE] FLIP-530: Dynamic job configuration

2025-05-20 Thread Hangxiang Yu
+1 (binding) On Wed, May 21, 2025 at 10:25 AM Junrui Lee wrote: > +1 (binding) > > Best, > Junrui > > Hao Li 于2025年5月21日周三 03:42写道: > > > +1 (non-binding) > > > > Thanks, > > Hao > > > > On Tue, May 20, 2025 at 12:39 PM Piotr Nowojski > > wrote: > > > > > Hi, > > > > > > +1 (binding) > > > > >

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-20 Thread Hangxiang Yu
lid configurations: I encountered a problem before, where > the > > >> user > > >> >> added a non-existent flink configuration, but flink could not > detect > > >> it. > > >> >> It may be caused by typo. Therefore, even if we list som

Re: [DISCUSS] FLIP-530: Dynamic job configuration

2025-05-08 Thread Hangxiang Yu
Hi, Roman. Thanks for the FLIP. +1 for supporting dynamic configuration to reduce manual restart. I just have below questions: 1. Do we need a working configuration list ? So some unsupported configurations could be rejected in advance. 2. Could we show the change history in the Web UI ? So mo

Re: [DISCUSS] State Schema Evolution for RowData

2025-05-06 Thread Hangxiang Yu
Hi, Weiqing. Thanks for driving this FLIP. I'm +1 for supporting schema evolution for SQL RowData type. I just have some questions: 1. Could we consider defining a method returning *SchemaEvolutionSerializer* in *TypeSerializerSchemaCompatibility* (like compatibleAfterMigration(TypeSerializer sche

[jira] [Created] (FLINK-37693) ForSt fails to restore from reused checkpoint

2025-04-16 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-37693: Summary: ForSt fails to restore from reused checkpoint Key: FLINK-37693 URL: https://issues.apache.org/jira/browse/FLINK-37693 Project: Flink Issue Type

[jira] [Created] (FLINK-37692) ForSt Avoid deleting directory when reuse restored directory

2025-04-16 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-37692: Summary: ForSt Avoid deleting directory when reuse restored directory Key: FLINK-37692 URL: https://issues.apache.org/jira/browse/FLINK-37692 Project: Flink

[jira] [Created] (FLINK-37686) ForSt consecutive failed checkpoint due to check deleted file status when enableFileDeletions

2025-04-16 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-37686: Summary: ForSt consecutive failed checkpoint due to check deleted file status when enableFileDeletions Key: FLINK-37686 URL: https://issues.apache.org/jira/browse/FLINK-37686

[jira] [Created] (FLINK-37668) Compaction filter related configs cannot work as expected

2025-04-13 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-37668: Summary: Compaction filter related configs cannot work as expected Key: FLINK-37668 URL: https://issues.apache.org/jira/browse/FLINK-37668 Project: Flink

Re: [VOTE] FLIP-432: Faster Checkpoint & Recovery for Disaggregated State

2025-04-04 Thread Hangxiang Yu
+1 (binding) On Thu, Apr 3, 2025 at 5:02 PM Yanfei Lei wrote: > +1 (binding) > > Best, > Yanfei > > Zakelly Lan 于2025年4月3日周四 16:26写道: > > > > +1 (binding) > > > > > > Best, > > Zakelly > > > > On Thu, Apr 3, 2025 at 1:33 PM Yuan Mei wrote: > > > > > Thanks for driving this :-) > > > > > > +1 >

[jira] [Created] (FLINK-37608) File sink supports to close buckets in parallel when snapshot

2025-04-03 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-37608: Summary: File sink supports to close buckets in parallel when snapshot Key: FLINK-37608 URL: https://issues.apache.org/jira/browse/FLINK-37608 Project: Flink

Re: [ANNOUNCE] New Apache Flink PMC Member - Zakelly Lan

2025-04-01 Thread Hangxiang Yu
Congratulations, Zakelly! On Wed, Apr 2, 2025 at 9:55 AM Feifan Wang wrote: > Congratulations, Zakelly! > > > > > —— > > Best regards, > > Feifan Wang > > > > Replied Message > | From | Yanquan Lv | > | Date | 04/2/2025 09:20 | > | To | | > | Subject | Re: [ANNOUNCE] New A

Re: [VOTE] FLIP-488: Expose Async State Processing and New State APIs in Datastream(V1) APIs

2024-12-18 Thread Hangxiang Yu
+1 (binding) On Wed, Dec 18, 2024 at 6:00 PM Gabor Somogyi wrote: > +1 (binding) > > BR, > G > > On Wed, Dec 18, 2024 at 10:17 AM Yunfeng Zhou > > wrote: > > > +1(non-binding) > > > > Best, > > Yunfeng > > > > > 2024年12月17日 12:17,Zakelly Lan 写道: > > > > > > Hi all, > > > > > > I'd like to star

Re: Re: [VOTE] FLIP-455: Declare async state processing and checkpoint the in-flight requests

2024-08-15 Thread Hangxiang Yu
+1 (binding) Thanks for driving this. On Thu, Aug 15, 2024 at 8:42 PM Ahmed Hamdy wrote: > +1 (non-binding) > Thanks for driving. > Best Regards > Ahmed Hamdy > > > On Wed, 14 Aug 2024 at 07:33, Xuyang wrote: > > > +1 (non-binding) > > > > > > -- > > > > Best! > > Xuyang > > > > > > >

[jira] [Created] (FLINK-35928) ForSt supports compiling with RocksDB

2024-07-30 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35928: Summary: ForSt supports compiling with RocksDB Key: FLINK-35928 URL: https://issues.apache.org/jira/browse/FLINK-35928 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35927) Support closing ForSt meta file when using object storage

2024-07-30 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35927: Summary: Support closing ForSt meta file when using object storage Key: FLINK-35927 URL: https://issues.apache.org/jira/browse/FLINK-35927 Project: Flink

Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

2024-06-05 Thread Hangxiang Yu
Congratulations, Rui ! On Thu, Jun 6, 2024 at 9:18 AM Lincoln Lee wrote: > Congratulations, Rui! > > Best, > Lincoln Lee > > > Lijie Wang 于2024年6月6日周四 09:11写道: > > > Congratulations, Rui! > > > > Best, > > Lijie > > > > Rodrigo Meneses 于2024年6月5日周三 21:35写道: > > > > > All the best > > > > > > O

Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

2024-06-04 Thread Hangxiang Yu
Congratulations, Weijie! On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen wrote: > Congrats, Weijie! > > Best, > Zhanghao Chen > > From: Hang Ruan > Sent: Tuesday, June 4, 2024 16:37 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member

[jira] [Created] (FLINK-35460) Check file size when position read for ForSt

2024-05-26 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35460: Summary: Check file size when position read for ForSt Key: FLINK-35460 URL: https://issues.apache.org/jira/browse/FLINK-35460 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35389) Implement List Async State API for ForStStateBackend

2024-05-17 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35389: Summary: Implement List Async State API for ForStStateBackend Key: FLINK-35389 URL: https://issues.apache.org/jira/browse/FLINK-35389 Project: Flink Issue

Re: [DISCUSS] FLIP-444: Native file copy support

2024-05-12 Thread Hangxiang Yu
lly using such interface > extension would be out of scope of this FLIP. In > that case, maybe if this could be extended in the future without breaking > compatibility we could leave it as a > future improvement? > > Best, > Piotrek > > > pt., 10 maj 2024 o 07:40 Hangxiang

Re: [DISCUSS] FLIP-444: Native file copy support

2024-05-09 Thread Hangxiang Yu
Hi Piotr. Thanks for your proposal. I have some comments, PTAL: 1. +1 about unifying the interface with DuplicatingFileSystem. IIUC, DuplicatingFileSystem also covers the logic from/to both local and remote paths. The implementations could define their own logic about how to fast copy/duplicate fi

Re: [VOTE] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-05-05 Thread Hangxiang Yu
+1(binding) On Mon, May 6, 2024 at 12:25 PM Yuan Mei wrote: > +1(binding) > > Best > Yuan > > On Mon, May 6, 2024 at 11:28 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1 (binding) > > > > Best, > > Rui > > > > On Mon, May 6, 2024 at 11:01 AM Yanfei Lei wrote: > > > > > +1 (binding) > > > > >

[jira] [Created] (FLINK-35268) Support TTL for Async State API

2024-04-29 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35268: Summary: Support TTL for Async State API Key: FLINK-35268 URL: https://issues.apache.org/jira/browse/FLINK-35268 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35262) Bridge between AsyncKeyedStateBackend and AsyncExecutionController

2024-04-29 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35262: Summary: Bridge between AsyncKeyedStateBackend and AsyncExecutionController Key: FLINK-35262 URL: https://issues.apache.org/jira/browse/FLINK-35262 Project: Flink

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread Hangxiang Yu
Hi, Yue. Very glad to see that IngestDB will be used to improve the rescaling performance of RocksDB. And +1 for the upgrade. Thanks for the great work! On Thu, Apr 25, 2024 at 5:16 AM Martijn Visser wrote: > +1 > > On Wed, Apr 24, 2024 at 5:31 PM Congxian Qiu > wrote: > > > Thanks for driving

Re: [ANNOUNCE] New Apache Flink PMC Member - Jing Ge

2024-04-14 Thread Hangxiang Yu
Congratulations, Jing! On Mon, Apr 15, 2024 at 11:11 AM spoon_lz wrote: > > > Congratulations! > > Best, > lz > > | | > spoon_lz > | > | > spoon...@126.com > | > > > Replied Message > | From | Biao Geng | > | Date | 04/15/2024 10:40 | > | To | | > | Subject | Re: [ANNOUNCE] New Apache

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

2024-04-14 Thread Hangxiang Yu
Congratulations, Lincoln! On Mon, Apr 15, 2024 at 10:17 AM Zakelly Lan wrote: > Congratulations, Lincoln! > > > Best, > Zakelly > > On Sat, Apr 13, 2024 at 12:48 AM Ferenc Csaky > wrote: > > > Congratulations, Lincoln! > > > > Best, > > Ferenc > > > > > > > > > > On Friday, April 12th, 2024 at

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

2024-04-14 Thread Hangxiang Yu
Congratulations, Zakelly! On Mon, Apr 15, 2024 at 1:58 PM Yun Tang wrote: > Congratulations, Zakelly! > > Best > Yun Tang > > From: Yanquan Lv > Sent: Monday, April 15, 2024 13:23 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink Committer -

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-11 Thread Hangxiang Yu
+1 (binding) On Fri, Apr 12, 2024 at 10:22 AM Jinzhong Li wrote: > +1 (non binding) > > Bests, > Jinzhong > > On Thu, Apr 11, 2024 at 7:26 AM Muhammet Orazov > wrote: > > > Hey Rui, > > > > +1 (non-binding). > > > > Thanks for driving it! > > > > Best, > > Muhammet > > > > On 2024-04-10 04:36

[jira] [Created] (FLINK-35049) Implement Async State API for ForStStateBackend

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35049: Summary: Implement Async State API for ForStStateBackend Key: FLINK-35049 URL: https://issues.apache.org/jira/browse/FLINK-35049 Project: Flink Issue Type

[jira] [Created] (FLINK-35048) Implement all methods of AsyncKeyedStateBakend

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35048: Summary: Implement all methods of AsyncKeyedStateBakend Key: FLINK-35048 URL: https://issues.apache.org/jira/browse/FLINK-35048 Project: Flink Issue Type

[jira] [Created] (FLINK-35047) Introduce ForStStateBackend

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35047: Summary: Introduce ForStStateBackend Key: FLINK-35047 URL: https://issues.apache.org/jira/browse/FLINK-35047 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35046) Introduce New KeyedStateBackend related Async interfaces

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35046: Summary: Introduce New KeyedStateBackend related Async interfaces Key: FLINK-35046 URL: https://issues.apache.org/jira/browse/FLINK-35046 Project: Flink

[jira] [Created] (FLINK-35045) Introduce ForStFileSystem to support reading and writing with ByteBuffer

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35045: Summary: Introduce ForStFileSystem to support reading and writing with ByteBuffer Key: FLINK-35045 URL: https://issues.apache.org/jira/browse/FLINK-35045 Project

[jira] [Created] (FLINK-35044) Introduce statebackend-forst module

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35044: Summary: Introduce statebackend-forst module Key: FLINK-35044 URL: https://issues.apache.org/jira/browse/FLINK-35044 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35043) Release beta version of ForSt

2024-04-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35043: Summary: Release beta version of ForSt Key: FLINK-35043 URL: https://issues.apache.org/jira/browse/FLINK-35043 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34987) Introduce Internal State Interface for Async API

2024-04-02 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34987: Summary: Introduce Internal State Interface for Async API Key: FLINK-34987 URL: https://issues.apache.org/jira/browse/FLINK-34987 Project: Flink Issue Type

[jira] [Created] (FLINK-34975) FLIP-427: ForSt - Disaggregated state Store

2024-03-31 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34975: Summary: FLIP-427: ForSt - Disaggregated state Store Key: FLINK-34975 URL: https://issues.apache.org/jira/browse/FLINK-34975 Project: Flink Issue Type: New

Re: [VOTE] FLIP-426: Grouping Remote State Access

2024-03-31 Thread Hangxiang Yu
+1 (binding) On Sun, Mar 31, 2024 at 9:02 PM Yanfei Lei wrote: > +1 (binding) > > Best, > Yanfei > > yue ma 于2024年3月29日周五 16:10写道: > > > > +1 (non-binding) > > > > Jinzhong Li 于2024年3月27日周三 18:57写道: > > > > > Hi devs, > > > > > > I'd like to start a vote on the FLIP-426: Grouping Remote State

[RESULT][VOTE] FLIP-427: Disaggregated state Store

2024-03-31 Thread Hangxiang Yu
I'm happy to announce that FLIP-427: Disaggregated state Store[1] has been accepted with 7 approving votes (4 binding) [2]: - Yuan Mei (binding) - Feifan Wang (non-binding) - Piotr Nowojski (binding) - Rui Fan (binding) - Yun Tang (binding) - Yuepeng Pan (non-binding) - yue ma (non-binding) There

Re: [VOTE] FLIP-427: Disaggregated state Store

2024-03-31 Thread Hangxiang Yu
Thanks all for the votes! I'm closing the vote and the result will be posted in a separate mail. On Fri, Mar 29, 2024 at 4:29 PM yue ma wrote: > +1(non-binding) > > Hangxiang Yu 于2024年3月27日周三 18:37写道: > > > Hi devs, > > > > Thanks all for your valuable fee

Re: [VOTE] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-31 Thread Hangxiang Yu
+1 (binding) On Mon, Apr 1, 2024 at 9:24 AM gongzhongqiang wrote: > +1(non-binding) > > Best, > > Zhongqiang Gong > > Jinzhong Li 于2024年3月27日周三 19:31写道: > > > Hi devs, > > > > > > I'd like to start a vote on the FLIP-428: Fault Tolerance/Rescale > > Integration for Disaggregated State [1]. The

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-31 Thread Hangxiang Yu
UC, the working dir > will be deleted after TaskManager exit. And I think that's enough for > current stage, WDYT ? > > —————— > > Best regards, > > Feifan Wang > > > > > At 2024-03-28 12:18:56, "Hangxiang Yu" wrote: > >Hi, F

Re: Re: [ANNOUNCE] Apache Paimon is graduated to Top Level Project

2024-03-28 Thread Hangxiang Yu
Congratulations! On Fri, Mar 29, 2024 at 10:27 AM Benchao Li wrote: > Congratulations! > > Zakelly Lan 于2024年3月29日周五 10:25写道: > > > > Congratulations! > > > > > > Best, > > Zakelly > > > > On Thu, Mar 28, 2024 at 10:13 PM Jing Ge > wrote: > > > > > Congrats! > > > > > > Best regards, > > > Jin

Re: [DISCUSS] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-27 Thread Hangxiang Yu
Hi, Yun and Feifan. Thanks for your reply. About the cleanup of working dir, as mentioned in FLIP-427, "The life cycle of working dir is managed as before local strategy.". Since the current working dir and checkpoint dir are separate, The life cycle including creating and cleanup of working dir

Re: Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-27 Thread Hangxiang Yu
about > >this configuration, e.g. life cycle, supported file systems or storages. > I agree with considering moving this configuration up to the engine level > until there are other disaggreated backends. > > > [1] https://cwiki.apache.org/confluence/x/U4p3EQ > >

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-27 Thread Hangxiang Yu
nd, how would it be if we introduce another disaggregated state > storage? Thus, I think `state.backend.disaggregated.working-dir` might be a > better configuration name. > > > Best > Yun Tang > > > From: Hangxiang Yu > Sent: Wednesday, Marc

[VOTE] FLIP-427: Disaggregated state Store

2024-03-27 Thread Hangxiang Yu
Hi devs, Thanks all for your valuable feedback about FLIP-427: Disaggregated state Store [1]. I'd like to start a vote on it. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1] https://cwiki.apache.org/confluenc

Re: [DISCUSS] Flink Website Menu Adjustment

2024-03-25 Thread Hangxiang Yu
Thanks Zhongqiang for driving this. +1 for the proposal. On Tue, Mar 26, 2024 at 1:36 PM Shawn Huang wrote: > +1 for the proposal > > Best, > Shawn Huang > > > Hongshun Wang 于2024年3月26日周二 11:56写道: > > > +1 for the proposal > > > > Best Regards, > > Hongshun Wang > > > > On Tue, Mar 26, 2024 at

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Hangxiang Yu
Congratulations! Thanks for the efforts. On Fri, Mar 22, 2024 at 10:00 AM Yanfei Lei wrote: > Congratulations! > > Best regards, > Yanfei > > Xuannan Su 于2024年3月22日周五 09:21写道: > > > > Congratulations! > > > > Best regards, > > Xuannan > > > > On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang > wro

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread Hangxiang Yu
+1 (binding) On Thu, Mar 21, 2024 at 10:04 AM Xintong Song wrote: > +1 (binding) > > Best, > > Xintong > > > > On Wed, Mar 20, 2024 at 8:30 PM weijie guo > wrote: > > > Hi everyone, > > > > > > Thanks for all the feedback about the FLIP-433: State Access on > > DataStream API V2 [1]. The discus

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-19 Thread Hangxiang Yu
Hi, Yue. Thanks for the reply. If we use proposal1, we can easily reuse these optimizations .It is even > possible to discuss and review the solution together in the Rocksdb > community. We also saw these useful optimizations which could be applied to ForSt in the future. But IIUC, it's not bindi

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-19 Thread Hangxiang Yu
quot; among multiple workers. >> In these cases, the engine spawns huge amount of scan range requests to >> the >> file system to retrieve different parts of a file. >> How the proposed solution would work with these workloads? >> >> - The similar question r

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Hangxiang Yu
Congratulations! Thanks release managers and all involved! On Mon, Mar 18, 2024 at 5:23 PM Hang Ruan wrote: > Congratulations! > > Best, > Hang > > Paul Lam 于2024年3月18日周一 17:18写道: > > > Congrats! Thanks to everyone involved! > > > > Best, > > Paul Lam > > > > > 2024年3月18日 16:37,Samrat Deb 写道:

[jira] [Created] (FLINK-34660) AutoRescalingITCase#testCheckpointRescalingInKeyedState AssertionError

2024-03-12 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34660: Summary: AutoRescalingITCase#testCheckpointRescalingInKeyedState AssertionError Key: FLINK-34660 URL: https://issues.apache.org/jira/browse/FLINK-34660 Project

[jira] [Created] (FLINK-34652) Use CheckpointStreamFactory for StateChangeFsUploader

2024-03-11 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34652: Summary: Use CheckpointStreamFactory for StateChangeFsUploader Key: FLINK-34652 URL: https://issues.apache.org/jira/browse/FLINK-34652 Project: Flink Issue

Re: [DISCUSS] FLIP-433: State Access on DataStream API V2

2024-03-11 Thread Hangxiang Yu
s logic must be fully available at > > > > compile > > > > > time (when the JobGraph is generated), so it can't rely on > computations > > > > > that are executed after deploy to TM. > > > > > >> > > > > > >> > > > > >

Re: [VOTE] Release 1.19.0, release candidate #2

2024-03-11 Thread Hangxiang Yu
+1 (non-binding) - Verified signatures and checksums - Reviewed Web PR - Built from source successfully - Ran a wordcount job which worked well On Tue, Mar 12, 2024 at 1:00 AM Jeyhun Karimov wrote: > +1 (non binding) > > - verified that source distribution does not contain binaries > - verifi

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-10 Thread Hangxiang Yu
pplies also for caching ( I > know caching is subject of FLIP-429, asking here becasue of the related > section in this FLIP). > > Regards, > Jeyhun > > [1] https://blog.min.io/challenge-big-data-small-files/ > > > > On Thu, Mar 7, 2024 at 10:09 AM Hangxiang Yu w

[DISCUSS] FLIP-427: Disaggregated State Store

2024-03-07 Thread Hangxiang Yu
Hi devs, I'd like to start a discussion on a sub-FLIP of FLIP-423: Disaggregated State Storage and Management[1], which is a joint work of Yuan Mei, Zakelly Lan, Jinzhong Li, Hangxiang Yu, Yanfei Lei and Feng Wang: - FLIP-427: Disaggregated State Store This FLIP introduces the initial ve

Re: [DISCUSS] FLIP-433: State Access on DataStream API V2

2024-03-06 Thread Hangxiang Yu
Hi, Weijie. Thanks for your proposal. I'd like to start the discussion with some questions: 1. We have also discussed in FLIP-359/FLINK-32658 about limiting the user operation to avoid creating state when processElement. Could current interfaces also help this? 2. Could you provide more examples a

Re: [VOTE] FLIP-420: Add API annotations for RocksDB StateBackend user-facing classes

2024-03-06 Thread Hangxiang Yu
+1 (binding) On Thu, Mar 7, 2024 at 9:34 AM Yun Tang wrote: > > +1 for this FLIP. > Sorry for not being clear in my previous reply, it's a binding vote. > > Best > Yun Tang > > From: Jeyhun Karimov > Sent: Thursday, March 7, 2024 4:40 > To: dev@flink.apache.org

Re: [DISCUSS] Move CheckpointingMode to flink-core

2024-02-26 Thread Hangxiang Yu
Hi, Zakelly. Thanks for driving this. Moving this class to flink-core makes sense to me which could make the code path and configs clearer. It's marked as @Public from 1.0 and 1.20 should be the next long-term version, so 1.19 should have been a suitable version to do it. And also look forward to t

[jira] [Created] (FLINK-34512) Thrown root cause for  HandlerRequestException

2024-02-25 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34512: Summary: Thrown root cause for  HandlerRequestException Key: FLINK-34512 URL: https://issues.apache.org/jira/browse/FLINK-34512 Project: Flink Issue Type

[jira] [Created] (FLINK-34479) Fix missed changelog configs in the documentation

2024-02-20 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34479: Summary: Fix missed changelog configs in the documentation Key: FLINK-34479 URL: https://issues.apache.org/jira/browse/FLINK-34479 Project: Flink Issue Type

[jira] [Created] (FLINK-34352) Improve the documentation of allowNonRestoredState

2024-02-04 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34352: Summary: Improve the documentation of allowNonRestoredState Key: FLINK-34352 URL: https://issues.apache.org/jira/browse/FLINK-34352 Project: Flink Issue

Re: [DISCUSS]FLIP-420: Add API annotations for RocksDB StateBackend user-facing classes

2024-01-24 Thread Hangxiang Yu
Hi Jinzhong. Thanks for driving this! Some suggestions: 1. As RocksDBStateBackend marked as Deprecated, We should also mark RocksDBStateBackendFactory as Deprecated 2. Since 1.19 will be freezed in 1.26. Let's adjust the target version to 1.20 On Wed, Jan 24, 2024 at 11:50 PM Zakelly Lan wrote:

Re: [VOTE] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-24 Thread Hangxiang Yu
+1 (binding) On Thu, Jan 25, 2024 at 8:49 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, 24 Jan 2024 at 21:50, Zakelly Lan wrote: > > > Hi everyone, > > > > I'd like to start a vote on the FLIP-406: Reorganize State & > Checkpointing > > & Recovery Configurati

Re: [VOTE] FLIP-416: Deprecate and remove the RestoreMode#LEGACY

2024-01-18 Thread Hangxiang Yu
+1 (binding) On Fri, Jan 19, 2024 at 12:10 PM Zakelly Lan wrote: > Hi everyone, > > I'd like to start a vote on the FLIP-416: Deprecate and remove the > RestoreMode#LEGACY [1]. The discussion thread is here [2]. > > The vote will be open for at least 72 hours unless there is an objection or > in

[jira] [Created] (FLINK-34119) Improve description about changelog in document

2024-01-16 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34119: Summary: Improve description about changelog in document Key: FLINK-34119 URL: https://issues.apache.org/jira/browse/FLINK-34119 Project: Flink Issue Type

Re: [DISCUSS] FLIP-416: Deprecate and remove the RestoreMode#LEGACY

2024-01-14 Thread Hangxiang Yu
Hi, Zakelly. Thanks for the quick feedback and driving this. +1 for removing LEGACY mode in Flink 2.0. On Mon, Jan 15, 2024 at 3:23 AM Danny Cranmer wrote: > +1 to removing LEGACY mode in Flink 2.0. Thanks for driving. > > Danny, > > On Sat, 13 Jan 2024, 08:20 Yanfei Lei, wrote: > > > Thanks Za

Re: [DISCUSS] FLIP-414: Support Retry Mechanism in RocksDBStateDataTransfer

2024-01-11 Thread Hangxiang Yu
Thanks for driving this. Retry mechanism is common when we want to get or put data by network. So I think it will help when checkpoint failure due to temporary network problems, of course it may increase a bit overhead for some other reasons. Some comments and suggestions: 1. Since Flink has a che

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-10 Thread Hangxiang Yu
+1 (non-binding) On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su wrote: > +1 (non-binding) > > Best, > Xuannan > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang wrote: > > > > +1 (non-binding)-- > > > > Best! > > Xuyang > > > > > > > > > > > > 在 2024-01-11 10:00:11,"Yang Wang" 写道: > > >+1 (bindi

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-10 Thread Hangxiang Yu
mode. > > > > > >> 3. Could the local checkpoint be cleaned > >> if execution.checkpointing.local-copy.enabled is true and > >> execution.recovery.from-local is false ? I found it's also an issue if > >> current local-recovery from enable

[jira] [Created] (FLINK-34051) Fix equals/hashCode/toString for SavepointRestoreSettings

2024-01-10 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34051: Summary: Fix equals/hashCode/toString for SavepointRestoreSettings Key: FLINK-34051 URL: https://issues.apache.org/jira/browse/FLINK-34051 Project: Flink

[jira] [Created] (FLINK-34032) Cleanup local=recovery dir when switching local-recovery from enabled to disabled

2024-01-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34032: Summary: Cleanup local=recovery dir when switching local-recovery from enabled to disabled Key: FLINK-34032 URL: https://issues.apache.org/jira/browse/FLINK-34032

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-08 Thread Hangxiang Yu
Hi, Zakelly. Thanks for driving this. Overall LGTM as we discussed offline. Some comments/suggestions just came to mind: 1. Could execution.recovery also contain some other behaviors about recovery ? e.g. restart-strategy. 2. Could we also remove some legacy configuration value ? e.g. LEGACY Mode

[jira] [Created] (FLINK-34030) Avoid using negative value for periodic-materialize.interval

2024-01-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34030: Summary: Avoid using negative value for periodic-materialize.interval Key: FLINK-34030 URL: https://issues.apache.org/jira/browse/FLINK-34030 Project: Flink

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-21 Thread Hangxiang Yu
+1(binding) On Wed, Nov 22, 2023 at 10:29 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Nov 22, 2023 at 1:20 AM Piotr Nowojski > wrote: > > > Hi All, > > > > I'd like to start a vote on the FLIP-385: Add OpenTelemetryTraceReporter > > and OpenTelemetryMetricR

Re: [VOTE] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-21 Thread Hangxiang Yu
+1 (binding) Thanks for your efforts! On Mon, Nov 20, 2023 at 11:53 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi everyone, > > Thank you to everyone for the feedback on FLIP-390: Support > System out and err to be redirected to LOG or discarded[1] > which has been discussed in this thread [2]. >

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Hangxiang Yu
+1 (binding) Thanks for driving this again! On Wed, Nov 22, 2023 at 10:30 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Nov 22, 2023 at 6:43 AM Jing Ge > wrote: > > > +1(binding) Thanks! > > > > Best regards, > > Jing > > > > On Tue, Nov 21, 2023 at 6:17 PM P

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Hangxiang Yu
olution is to log thread name, > because the thread name of the default Task Thread contains the > task name and subtask id. And this solution can support all threads, > because some non-task threads may also call println. > > [1] https://cwiki.apache.org/confluence/x/4guZE > >

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Hangxiang Yu
o slow, the file size cannot be controlled > > > properly. > > > > > > > > Redirect it to LOG.info may be a reasonable and easy choice. > > > > The user didn't really want to log into taskmanager.out, it just > > > > happened by accident. > >

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-08 Thread Hangxiang Yu
Hi, Piotr. Thanks for the proposal. Just as we discussed in FLINK-23411, +1 for supporting trace/span to monitor metrics like checkpoint and recovery. We could also do many things based on this mechanism: 1. more fine-grained metrics about checkpoint and recovery. For example, some stage info abou

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-08 Thread Hangxiang Yu
Hi, Rui. Thanks for the proposal. It sounds reasonable. I have some questions, PTAL: 1. I have a similar concern as Feng. Will we redirect to another log file not taskManager.log ? taskManager.log contains lots of important information like init log. It will be rolled quickly if we redirect out and

Re: [ANNOUNCE] The Flink Speed Center and benchmark daily run are back online

2023-10-20 Thread Hangxiang Yu
Thanks Zakelly for the great work! On Fri, Oct 20, 2023 at 5:05 PM Rui Fan <1996fan...@gmail.com> wrote: > Thanks for your effort! It's very useful when some new commits affect > performance. > > Best, > Rui > > On Fri, Oct 20, 2023 at 4:42 PM Yanfei Lei wrote: > > > Thanks for your hard work!

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

2023-09-19 Thread Hangxiang Yu
Hi, Zakelly. Thanks for the proposal. +1 for reorganizing exceptions of state interfaces which indeed confuses me currently. >From my experience, users usually omit these exceptions because they cannot do much even if they catch the exceptions. I have some problems and suggestions, PTAL: 1.

[jira] [Created] (FLINK-33055) Correct the error value about 'state.backend.type' in the document

2023-09-06 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-33055: Summary: Correct the error value about 'state.backend.type' in the document Key: FLINK-33055 URL: https://issues.apache.org/jira/browse/FLINK-33055 Proj

Re: [DISCUSS] Add config to enable job stop with savepoint on exceeding tolerable checkpoint Failures

2023-09-06 Thread Hangxiang Yu
Hi, Dongwoo. IIUC, you mean using savepoint to store a snapshot to other storage if checkpoints fail multiple times due to some long lasting exceptions of external storage, right ? I think it's better to achieve this by an external tool instead of introducing a config like that: 1. it's not so easy

[jira] [Created] (FLINK-32601) Unstable RemoteChannelThroughputBenchmark_remoteRebalance_jmhTest

2023-07-17 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-32601: Summary: Unstable RemoteChannelThroughputBenchmark_remoteRebalance_jmhTest Key: FLINK-32601 URL: https://issues.apache.org/jira/browse/FLINK-32601 Project: Flink

[jira] [Created] (FLINK-32364) Add Rescaling benchmark for ChangelogStateBackend

2023-06-15 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-32364: Summary: Add Rescaling benchmark for ChangelogStateBackend Key: FLINK-32364 URL: https://issues.apache.org/jira/browse/FLINK-32364 Project: Flink Issue Type

Re: [VOTE] FLIP-306: Unified File Merging Mechanism for Checkpoints

2023-05-09 Thread Hangxiang Yu
Hi Zakelly. Thanks for driving this. +1 (no-binding) On Wed, May 10, 2023 at 10:52 AM Yuan Mei wrote: > Thanks for driving this, Zakelly. > > As discussed in the thread, > > +1 for the proposal (binding) > > Best, > > Yuan > > > > On Wed, May 10, 2023 at 10:39 AM Zakelly Lan > wrote: > > > Hi e

[jira] [Created] (FLINK-31875) OSS throwns NoClassDefFoundError due to old hadoop-common version

2023-04-20 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-31875: Summary: OSS throwns NoClassDefFoundError due to old hadoop-common version Key: FLINK-31875 URL: https://issues.apache.org/jira/browse/FLINK-31875 Project: Flink

[jira] [Created] (FLINK-31366) Exception is thrown when s3a and s3p are used together

2023-03-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-31366: Summary: Exception is thrown when s3a and s3p are used together Key: FLINK-31366 URL: https://issues.apache.org/jira/browse/FLINK-31366 Project: Flink Issue

[jira] [Created] (FLINK-30854) Expose periodic compaction to RocksdbCompactFilterCleanupStrategy

2023-01-31 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30854: Summary: Expose periodic compaction to RocksdbCompactFilterCleanupStrategy Key: FLINK-30854 URL: https://issues.apache.org/jira/browse/FLINK-30854 Project: Flink

[jira] [Created] (FLINK-30656) Provide more logs for schema compatibility check

2023-01-12 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30656: Summary: Provide more logs for schema compatibility check Key: FLINK-30656 URL: https://issues.apache.org/jira/browse/FLINK-30656 Project: Flink Issue Type

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-09 Thread Hangxiang Yu
Congratulations, Lincoln! On Tue, Jan 10, 2023 at 1:34 PM Dian Fu wrote: > Congratulations, Lincoln! > > Regards, > Dian > > On Tue, Jan 10, 2023 at 1:31 PM weijie guo > wrote: > > > Congratulations, Lincoln! > > > > Best regards, > > > > Weijie > > > > > > Lijie Wang 于2023年1月10日周二 12:24写道: >

[jira] [Created] (FLINK-30614) Improve resolving schema compatibility -- Milestone two

2023-01-09 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30614: Summary: Improve resolving schema compatibility -- Milestone two Key: FLINK-30614 URL: https://issues.apache.org/jira/browse/FLINK-30614 Project: Flink

[jira] [Created] (FLINK-30613) Improve resolving schema compatibility -- Milestone one

2023-01-09 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30613: Summary: Improve resolving schema compatibility -- Milestone one Key: FLINK-30613 URL: https://issues.apache.org/jira/browse/FLINK-30613 Project: Flink

  1   2   >