Hi devs,

I'd like to start a discussion about FLIP-408: [Umbrella] Introduce
DataStream API V2 [1].


The DataStream API is one of the two main APIs that Flink provides for
writing data processing programs. As an API that was introduced
practically since day-1 of the project and has been evolved for nearly
a decade, we are observing more and more problems of it. Improvements
on these problems require significant breaking changes, which makes
in-place refactor impractical. Therefore, we propose to introduce a
new set of APIs, the DataStream API V2, to gradually replace the
original DataStream API.


The proposal to introduce a whole set new API is complex and includes
massive changes. We are planning  to break it down into multiple
sub-FLIPs for incremental discussion. This FLIP is only used as an
umbrella, mainly focusing on motivation, goals, and overall planning.
That is to say, more design and implementation details  will be
discussed in other FLIPs.


Given that it's hard to imagine the detailed design of the new API if
we're just talking about this umbrella FLIP, and we probably won't be
able to give an opinion on it. Therefore, I have prepared two
sub-FLIPs [2][3] at the same time, and the discussion of them will be
posted later in separate threads.


Looking forward to hearing from you, thanks!


Best regards,

Weijie



[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-408%3A+%5BUmbrella%5D+Introduce+DataStream+API+V2

[2]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-409%3A+DataStream+V2+Building+Blocks%3A+DataStream%2C+Partitioning+and+ProcessFunction


[3]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-410%3A++Config%2C+Context+and+Processing+Timer+Service+of+DataStream+API+V2

Reply via email to