I guess there's no point in making it a KeyedProcessFunction since it's not
going to have access to context, timers or anything like that. So it can be
a simple InputFormat returning a DataSet of key and value tuples.

On Wed, Mar 17, 2021 at 8:37 AM Andrey Bulgakov <m...@andreiko.ru> wrote:

> Hi Gordon,
>
> I think my current implementation is very specific and wouldn't be that
> valuable for the broader public.
> But I think there's a potential version of it that could also retrieve
> values from a savepoint in the same efficient way and that would be
> something that other people might need.
>
> I'm currently thinking about something similar to KeyedProcessFunction but
> taking a single state descriptor as a parameter instead of expecting a user
> to "register" some of them open(). The processElement() method would then
> be invoked with both key and value.
>
> One thing I'm not sure about are MapStateDescriptors because it stores
> compound keys separately and I'm not sure if they are stored in a sorted
> order and can be passed to processElement() as a group or should rather be
> passed separately.
>
> I'll experiment with this for a while and try to figure out what works.
> Please let me know if you have thoughts about this.
>
> On Sun, Mar 14, 2021 at 11:55 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org>
> wrote:
>
>> Hi Andrey,
>>
>> Perhaps the functionality you described is worth adding to the State
>> Processor API.
>> Your observation on how the library currently works is correct; basically
>> it
>> tries to restore the state backends as is.
>>
>> In you current implementation, do you see it worthwhile to try to add
>> this?
>>
>> Cheers,
>> Gordon
>>
>>
>>
>> --
>> Sent from:
>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>>
>
>
> --
> With regards,
> Andrey Bulgakov
>


-- 
With regards,
Andrey Bulgakov

Reply via email to