Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2024-01-07 Thread Yong Fang
Hi Ken, I think the main reason is that currently Kryo is the only generic serializer in Flink. I'm looking forward to your FLIP of Fury, and we can continue to discuss this issue there. If there are no other questions, I will close the voting for this FLIP. Thank you again. Best, Fang Yong On

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2024-01-05 Thread Ken Krugler
Hi Fang Yong, Thanks for the response, and I understand the desire to limit the impact of this FLIP. I guess I should spend the time to start a new FLIP on switching to Fury, which could include cleaning up method names. In the context of “facilitate user understanding”, one aspect of this cle

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2024-01-04 Thread Yong Fang
Hi Ken, Sorry for the late reply. After discussing with @Xintong, we think it is better to keep the method names in the FLIP mainly for the following reasons: 1. This FLIP is mainly to support the configurable serializer while keeping consistent with Flink at the semantic layer. Keeping the exist

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-28 Thread Ken Krugler
Hi Xintong, I agree that decoupling from Kryo is a bigger topic, well beyond the scope of this FLIP. The reason I’d brought up Fury is that this increases my confidence that Flink will want to decouple from Kryo sooner rather than later. So I feel it would be worth investing in a (minor) name

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-24 Thread Yong Fang
Hi devs, Thanks for all the feedback. If there are no more comments, I would like to start a vote for this FLIP, thanks again! Best, Fang Yong On Wed, Dec 20, 2023 at 9:12 PM Yong Fang wrote: > Hi Ken, > > Thanks for your feedback. The purpose of this FLIP is to improve the use > of serializat

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-20 Thread Yong Fang
Hi Ken, Thanks for your feedback. The purpose of this FLIP is to improve the use of serialization, including configurable serializer for users, providing serializer for composite data types, and resolving the default enabling of Kryo, etc. Introducing a better serialization framework would be a gr

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-17 Thread Xintong Song
Hi Ken, I think the main purpose of this FLIP is to change how users interact with the knobs for customizing the serialization behaviors, from requiring code changes to working with pure configurations. Redesigning the knobs (i.e., names, semantics, etc.), on the other hand, is not the purpose of

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-14 Thread Ken Krugler
Hi Yong, Looks good, thanks for creating this. One comment - related to my recent email about Fury, I would love to see the v2 serialization decoupled from Kryo. As part of that, instead of using xxxKryo in methods, call them xxxGeneric. A more extreme change would be to totally rely on Fury (

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-11 Thread Zhu Zhu
Thanks Yong for creating the FLIP and starting the discussion. Having a unified config option for all kinds of serializers can make it much easier for users to set, track and maintain the serializers for jobs. +1 Thanks, Zhu weijie guo 于2023年12月12日周二 14:01写道: > Thanks for driving this, Yong.

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-11 Thread weijie guo
Thanks for driving this, Yong. This FLIP look good to me overall. I also used a similar approach to decouple the serializer from ExecutionConfig in the PoC of DataStream API V2. I'm +1 for this. Best regards, Weijie Xintong Song 于2023年12月12日周二 12:10写道: > Thanks for working on this, Yong. >

Re: [DISCUSS] FLIP-398: Improve Serialization Configuration And Usage In Flink

2023-12-11 Thread Xintong Song
Thanks for working on this, Yong. The usability of the serialization mechanism has indeed been a pain for a long time. The proposed changes look good to me. +1 from my side. Best, Xintong On Thu, Dec 7, 2023 at 10:36 AM Yong Fang wrote: > Hi devs, > > I'd like to start a discussion about F