Hi Jaya, Broadcast pattern may help here. Take a look at this: https://ci.apache.org/projects/flink/flink-docs-release-1.8/dev/stream/state/broadcast_state.html
You'll still keep your geofence data as a stream (depending on the data and use case, maybe the whole list of geofence as a single stream item), broadcast the stream to downstream operators, which will now have geofence data in their state as their slow changing data (processBroadcastElement), and the user location regularly coming to the operator (processElement). --- Oytun Tez *M O T A W O R D* The World's Fastest Human Translation Platform. oy...@motaword.com — www.motaword.com On Thu, Jul 25, 2019 at 6:48 PM jaya sai <jayasai...@gmail.com> wrote: > Hello, > > I have a question on using flink, we have a small data set which does not > change often but have another data set which we need to compare with it and > it has lots of data > > let say I have two collections geofence and locations in mongodb. Geofence > collection does not change often and relatively small, but we have location > data coming in at high amounts from clients and we need to calculate the > goefence entry exits based on geofence and location data point. > For calculating the entry and exit we were thinking of using flink CEP. > But our problem is sometimes geofence data changes and we need to update > the in memory store of the flink somehow > > we were thinking of bootstrapping the memory of flink processor by loading > data on initial start and subscribe to kafaka topic to listen for geofence > changes and re-pull the data > Is this a valid approach ? > > Thank you, >
