Hi, Thanks Alex for driving this!
+1 To give the Flink developers, especially Connector developers the clear signal that the new Source API is recommended according to FLIP-27, we should mark them as deprecated. There are some open questions to discuss: 1. Do we need to mark all subinterfaces/subclasses as deprecated? e.g. FromElementsFunction, etc. there are many. What are the replacements? 2. Do we need to mark all subclasses that have replacement as deprecated? e.g. ExternallyInducedSource whose replacement class, if I am not mistaken, ExternallyInducedSourceReader is @Experimental 3. Do we need to mark all related test utility classes as deprecated? I think it might make sense to create an umbrella ticket to cover all of these with the following process: 1. Mark SourceFunction as deprecated asap. 2. Mark subinterfaces and subclasses as deprecated, if there are graduated replacements. Good example is that KafkaSource replaced KafkaConsumer which has been marked as deprecated. 3. Do not mark subinterfaces and subclasses as deprecated, if replacement classes are still experimental, check if it is time to graduate them. After graduation, go to step 2. It might take a while for graduation. 4. Do not mark subinterfaces and subclasses as deprecated, if the replacement classes are experimental and are too young to graduate. We have to wait. But in this case we could create new tickets under the umbrella ticket. 5. Do not mark subinterfaces and subclasses as deprecated, if there is no replacement at all. We have to create new tickets and wait until the new implementation has been done and graduated. It will take a longer time, roughly 1,5 years. 6. For test classes, we could follow the same rule. But I think for some cases, we could consider doing the replacement directly without going through the deprecation phase. When we look back on all of these, we can realize it is a big epic (even bigger than an epic). It needs someone to drive it and keep focus on it continuously with support from the community and push the development towards the new Source API of FLIP-27. If we could have consensus for this, Alex and I could create the umbrella ticket to kick it off. Best regards, Jing On Fri, Jun 3, 2022 at 3:54 PM Alexander Fedulov <alexan...@ververica.com> wrote: > Hi everyone, > > I would like to start the discussion about marking SourceFunction-based > interfaces as deprecated. With the FLIP-27 APIs becoming the new standard, > the old ones have to be eventually phased out. Although this state is well > known within the community and no new connectors based on the old > interfaces can be accepted into the project, the footprint of > SourceFunction in the user code still keeps growing (primarily for data > generators and test utilities). I believe it is best to mark SourceFunction > as deprecated as soon as possible. What do you think? > > Best, > Alexander Fedulov >
