Hi Chengzhi, you said the Stream B which comes from a file will be updated unpredictably. I wonder if you could share more about how to judge an item (from Stream A I suppose) is not in the file and what watermark generation strategy did you choose?
Best, Xingcan > On May 12, 2018, at 12:48 AM, Chengzhi Zhao <w.zhaocheng...@gmail.com> wrote: > > Hi there, > > I have a use case to check for active ID, there are two streams and I connect > them: one has actual data (Stream A) and the other one is for lookup purpose > (Stream B), I am getting Stream B as a file which includes all active ID, so > inactive ID would not be show up on this list. I tried to use watermark to > clean up the state of inactivate ID, but the Stream B updates is > unpredictable so I want to keep everything in state until I found the item is > not in that file any more. > > Please suggest what is the best way to implement it in flink. Thanks in > advance for your help. > > Regards, > Chengzhi > >
