Hi Zakelly, Before we go into any details I can say that if we can expose the less the better so agree with you on the direction.
On the technical level not yet understand what exactly you're suggesting. `RichInputFormat` is a \@Public API where we expose `RuntimeContext` now on V1. I would like to understand the motivation what would be the difference between `SourceReaderContext.getRuntimeContext` vs `V2RichInputFormat.getRuntimeContext`. Or you mean something like `RichSourceReaderContext`? If you can be more explicit on how you imagine the other path with some new interface/class names, call flow or anything we can pursue that. On coding level now I'm sitting on ~3-4K lines of code but there are some quite things needs to be done. The whole code is depending on `SourceReaderContext.getRuntimeContext` but I think when it's stable and there is another approach how to expose `RuntimeContext` then it's only matter of time to make that adjustments. Hope that we're not shooting for another 10k change :) BR, G On Thu, May 29, 2025 at 10:23 AM Zakelly Lan <zakelly....@gmail.com> wrote: > Hi Gabor, > > Good point for the migration. > > I took a brief look. I thought the `RuntimeContext` is too powerful and > might not be suitable to expose on \@Public API. Is it possible to > introduce another 'rich' code path just like the difference between > `RichInputFormat` and `InputFormat`, and keep that internal? WDYT > > > Best, > Zakelly > > On Thu, May 22, 2025 at 6:30 PM Gabor Somogyi <gabor.g.somo...@gmail.com> > wrote: > > > Hi All, > > > > I've just had a look at it how it would be possible to migrate state > > processor API from source v1 to source v2 API. > > > > The whole concept in the state processor API actual implementation is > based > > on to get > > RuntimeContext from RichInputFormat [1] which then allows us to access > > state backend, > > keyed state store, etc... > > > > Since RuntimeContext is not exposed on source v2 API the question is > > obvious. > > Does it make sense to expose it on SourceReaderContext [2] ? > > > > If somebody has better idea then please share. > > > > [1] > > > > > https://github.com/apache/flink/blob/4c77daaf3554446e67f6b75ac18d54da84208b9a/flink-libraries/flink-state-processing-api/src/main/java/org/apache/flink/state/api/input/KeyedStateInputFormat.java#L156 > > [2] > > > > > https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/connector/source/SourceReaderContext.java > > > > BR, > > G > > >
