Hi Xintong, Thanks for bringing up this topic. I can provide some details regarding the SourceFunction deprecation efforts. Marking SourceFunction as deprecated was not possible until now since we have stringent compiler checks in flink-examples against using any deprecated APIs. We actually merged the migration of all examples to the new FLIP-27-based DataGeneratorSource [1] just two days ago [2]. Now the PR marking it @Deprecated is finally unblocked [3] (I would be grateful if you could merge it).
With regards to the Flink 2.0 scope, I compiled a list of items required to be able to drop the SourceFunction API [4] a while ago and as you can see, there is still quite some work to be done. Some items [5] might even require additions to the new Source API. Overall, I am happy to take ownership of completing this work package. Best, Alex [1] https://cwiki.apache.org/confluence/x/9Av1D [2] https://github.com/apache/flink/pull/21774 [3] https://github.com/apache/flink/pull/20049 [4] https://issues.apache.org/jira/browse/FLINK-28045 [5] https://issues.apache.org/jira/browse/FLINK-28054 On Thu, 29 Jun 2023 at 10:45, Martijn Visser <martijnvis...@apache.org> wrote: > Hi Xintong, > > With regards to the deprecation of the Scala APIs, during the PR > review it was requested to not mark all APIs as deprecated but only > the entry point [1], to avoid a fully striked through experience in > the IDE. I think the same idea was applicable on the DataSet API. I > think it depends on how formal we want to treat this: if really > formal, then we should deprecate them in 1.18. I think in both cases, > it's quite well known that they are deprecated. I'm +0 for either way, > as long as we're all agreeing that they can be removed in 2.0. > > With regards to Queryable State and Source/SinkFunction, +1 to mark > these as deprecated. > > Best regards, > > Martijn > > [1] > https://github.com/apache/flink/pull/21176#pullrequestreview-1159706808 > > On Thu, Jun 29, 2023 at 10:23 AM Xintong Song <tonysong...@gmail.com> > wrote: > > > > Hi devs, > > > > Looking at the release 2.0 proposals [1], I noticed that many APIs that > are > > proposed to be removed in 2.0 are not (fully) deprecated yet. We might > want > > to properly mark them as `@Deprecated` in 1.18 if we agree they should be > > removed in 2.0. Moreover, according to FLIP-321 [2] (not voted yet but > IMO > > is close to consensus IMO), a migration period is required after APIs are > > deprecated and before they can be removed. > > > > I might not be familiar with the status of all the APIs below. So I'd > like > > to bring them up and see if there's any concern regarding deprecating > them > > in 1.18. If there's concern for deprecating API, we can start a separate > > discussion thread for it. For those with no objections, I'd create JIRA > > tickets and try to properly deprecate them in 1.18. > > > > 1. DataSet API > > It's described as "legacy", "soft deprecated" in user documentation [3]. > > However, it's not annotated with `@Deprecated` in codes. According to > > FLIP-131 [4], DataSet API should be deprecated when DataStream API and > > Table API / SQL meet certain requirements. AFAICS, all the requirements > > mentioned in the FLIP are already fulfilled. We should annotate it as > > `@Deprecated` now. > > > > 2. SourceFunction / SinkFunction > > They are described as deprecated in the roadmap[5], and I don't find > > anything regarding them in user documentation. But they are also not > > annotated with `@Deprecated` in codes. TBH, I'm not aware of any formal > > decision to deprecate these. AFAICS, the replacement for SourceFunction > > (Source) has already been promoted to `@Public`, while the replacement > for > > SinkFunction (SinkV2) is still `@PublicEvolving`. I found a discussion[6] > > regarding promoting SinkV2 to `@Public`, but it's unclear to me what the > > conclusion is. > > > > 3. Queryable State > > It's described as approaching end-of-life in the roadmap [5], but is > > neither deprecated in codes nor in user documentation [7]. I also found a > > discussion [8] about rescuing it from deprecation, and it seems to me > there > > are more negative opinions than positive ones. > > > > 4. All Scala APIs > > I think we agreed to drop Scala API support in FLIP-265 [9], and have > tried > > to deprecate them in FLINK-29740 [10]. Also, both user documentation and > > roadmap[5] shows that scala API supports are deprecated. However, AFAICS, > > none of the APIs in `flink-streaming-scala` are annotated with > > `@Deprecated`, and only `ExecutionEnvironment` and `package` are marked > > `@Deprecated` in `flink-scala`. > > > > Looking forward to your feedback. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/vmhzv8fcw2b33pqxp43486owrxbkd5x9 > > > > [3] > > > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/dataset/overview/ > > > > [4] > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 > > > > [5] https://flink.apache.org/roadmap/ > > > > [6] https://lists.apache.org/thread/q62nj89rrz0t5xtggy5n65on95f2rmmx > > > > [7] > > > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/queryable_state/ > > > > [8] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m > > > > [9] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support > > > > [10] https://issues.apache.org/jira/browse/FLINK-29740 >
