I guess there's no point in making it a KeyedProcessFunction since it's not going to have access to context, timers or anything like that. So it can be a simple InputFormat returning a DataSet of key and value tuples.
On Wed, Mar 17, 2021 at 8:37 AM Andrey Bulgakov <m...@andreiko.ru> wrote: > Hi Gordon, > > I think my current implementation is very specific and wouldn't be that > valuable for the broader public. > But I think there's a potential version of it that could also retrieve > values from a savepoint in the same efficient way and that would be > something that other people might need. > > I'm currently thinking about something similar to KeyedProcessFunction but > taking a single state descriptor as a parameter instead of expecting a user > to "register" some of them open(). The processElement() method would then > be invoked with both key and value. > > One thing I'm not sure about are MapStateDescriptors because it stores > compound keys separately and I'm not sure if they are stored in a sorted > order and can be passed to processElement() as a group or should rather be > passed separately. > > I'll experiment with this for a while and try to figure out what works. > Please let me know if you have thoughts about this. > > On Sun, Mar 14, 2021 at 11:55 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org> > wrote: > >> Hi Andrey, >> >> Perhaps the functionality you described is worth adding to the State >> Processor API. >> Your observation on how the library currently works is correct; basically >> it >> tries to restore the state backends as is. >> >> In you current implementation, do you see it worthwhile to try to add >> this? >> >> Cheers, >> Gordon >> >> >> >> -- >> Sent from: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >> > > > -- > With regards, > Andrey Bulgakov > -- With regards, Andrey Bulgakov
