Hi Fabian,
I am not really sure using CoProcessFunction would be useful for my use
case. My use case, in short, can be explained as follows:
1) create 2 different local state store, where both have 1-N relationship.
For eg. 1 -> [A,B,C] and A -> [1,2,3]
2) Based on the key A, get list of element
Having an operator that updates state from one stream and queries it to
process the other stream is actually a common pattern.
As I said, I don't know your use case but I don't think that a
CoProcessFunction would result in a mess.
QueryableState will have quite a bit of overhead because the reque
Hi Fabian,
Thanks for the insight, I am currently exploring QueryableStateClient and
would attempt to get the value for a corresponding key using the
getkvstate() function,
I was confused about the jobId but I am expecting this would provide me with
the jobid of the current job -
ExecutionEnviro
I am not sure that this is impossible, but it is not the use case queryable
state was designed for.
I don't know the details of your application, but you could try to merge
the updating and the querying operators into a single one.
You could connect two streams with connect() and use a keyed
CoPro
Hi Fabian,
I read about the process function and it seems a perfect fit for my
requirement. Although while reading more about queryable-state I found that
its not meant to do lookups within job (Your comment in the following link).
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.c
Hi Fabian,
Thanks a lot for pointing that out would read about it and give it a try.
Regards,
Biplob
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-QueryableState-with-Sliding-Window-on-RocksDB-tp14514p14551.html
Sent from the Apach
Hi Biplob,
given these requirements, I would rather not use a window but implement the
functionality with a stateful ProcessFunction.
A ProcessFunction can register timers, e.g., to remove inactive state. The
state of a ProcessFunction can be made queryable.
Best, Fabian
2017-07-31 9:52 GMT+02:0
Thanks Fabian for the reply, I was reconsidering my design and the
requirement and what I mentioned already is partially confusing.
I realized that using a sessionwindow is better in this scenario where I
want a value to be updated per key and the session resets to wait for the
gap period with ev
Hi Biplob,
What do you mean by "creating a sliding window on top of a state"?
Sliding windows are typically defined on streams (data in motion) and not
on state (data at rest).
It seems that UpdatedTxnState always holds the last record that was
received per key. Do you want to compute the windows
