Hey! I share the same concerns mentioned above regarding the "ProcessFunction API".
I don't think we should create a replacement for the DataStream API unless we have a very good reason to do so and with a proper discussion about this as Alex said. Cheers, Gyula On Tue, Jun 27, 2023 at 11:03 AM Alexander Fedulov < alexander.fedu...@gmail.com> wrote: > Hi Xintong, > > By compatibility discussion do you mean the "[DISCUSS] FLIP-321: Introduce > an API deprecation process" thread [1]? > > I am also curious to know if the rationale behind this new API has been > previously discussed on the mailing list. Do we have a list of shortcomings > in the current DataStream API that it tries to resolve? How does the > current ProcessFunction functionality fit into the picture? Will it be kept > as is or subsumed by new API? > > [1] https://lists.apache.org/thread/vmhzv8fcw2b33pqxp43486owrxbkd5x9 > > Best, > Alex > > On Mon, 26 Jun 2023 at 14:33, Xintong Song <tonysong...@gmail.com> wrote: > > > > > > > The ProcessFunction API item is giving me the most headaches because > it's > > > very unclear what it actually entails; like is it an entirely separate > > API > > > to DataStream (sounds like it is!) or an extension of DataStream. How > > much > > > will it share the internals with DataStream etc.; how does it relate to > > the > > > Table API (w.r.t. switching APIs / what Table API uses underneath). > > > > > > > I totally understand your confusion. We started planning this after > kicking > > off the release 2.0, so there's still a lot to be explored and the plan > > keeps changing. > > > > > > - In the beginning, we planned to do an in-place refactor of > DataStream > > API, until the API migration period is proposed. > > - Then we want to make it an entirely separate API to DataStream, and > > listed as a must-have for release 2.0 so that we can remove DataStream > > once > > it's ready. > > - However, depending on the outcome of the API compatibility > discussion > > [1], we may not be able to remove DataStream in 2.0 anyway, which > means > > we > > might need to re-evaluate the necessity of this item for 2.0. > > > > I'd say we wait a bit longer for the compatibility discussion [1] and > > decide the priority for this item afterwards. > > > > > > Best, > > > > Xintong > > > > > > [1] https://lists.apache.org/list.html?dev@flink.apache.org > > > > > > On Mon, Jun 26, 2023 at 6:00 PM Chesnay Schepler <ches...@apache.org> > > wrote: > > > > > by-and-large I'm quite happy with the list of items. > > > > > > I'm curious as to why the "Disaggregated State Management" item is > marked > > > as a must-have; will it require changes that break something? What > > prevents > > > it from being added in 2.1? > > > > > > We may want to update the Java 17 item to "Make Java 17 the default, > drop > > > Java 8/11". Maybe even split it into a must-have "Drop Java 8" and a > > > nice-to-have "Drop Java 11"? > > > > > > "Move Calcite rules from Scala to Java": I would hope that this would > be > > > an entirely internal change, and could thus be an incremental process > > > independent of major releases. > > > What is the actual scale of this item; how much are we actually > > re-writing? > > > > > > "Add MetricGroup#getLogicalScope": I'd raise this to a must-have; i > think > > > I marked it down as nice-to-have only because it depends on another > item. > > > > > > The ProcessFunction API item is giving me the most headaches because > it's > > > very unclear what it actually entails; like is it an entirely separate > > API > > > to DataStream (sounds like it is!) or an extension of DataStream. How > > much > > > will it share the internals with DataStream etc.; how does it relate to > > the > > > Table API (w.r.t. switching APIs / what Table API uses underneath). > > > > > > There are a few items I added as ideas which don't have a priority yet; > > > would love to get some feedback on those. > > > > > > On 21/06/2023 08:41, Xintong Song wrote: > > > > > > Hi devs, > > > > > > As previously discussed in [1], we had been collecting work item > > proposals > > > for the 2.0 release until June 15th, on the wiki page [2]. > > > > > > - As we have passed the due date, I'd like to kindly remind everyone > > *not > > > to add / remove items directly on the wiki page*. If needed, please > > post > > > in this thread or reach out to the release managers instead. > > > - I've reached out to some folks for clarifications about their > > > proposals. Some of them mentioned that they can not yet tell whether > > we > > > should do an item or not, and would need more time / discussions to > > make > > > the decision. So I added a new symbol for items whose priorities are > > `TBD`. > > > > > > Now it's time to collaboratively decide a minimum set of must-have > items. > > > I've gone through the entire list of proposed items, and found most of > > them > > > make quite much sense. So I think an online sync might not be necessary > > for > > > this. I'd like to go with this DISCUSS thread, where everyone can > comment > > > on how they think the list can be improved, followed by a VOTE to > > formally > > > make the decision. > > > > > > Any feedback and opinions, including but not limited to the following > > > aspects, will be appreciated. > > > > > > - Important items that are missing from the list > > > - Concerns regarding the listed items or their priorities > > > > > > Looking forward to your feedback. > > > > > > Best, > > > > > > Xintong > > > > > > > > > [1] > > > https://lists.apache.org/list?dev@flink.apache.org:lte=1M:release%202.0%20status%20updates > > > > > > [2] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > > > > > > > > >
