I think so, you can use sliding windows and basically do a WordCount-like job that counts the occurences in each window. Then, you would have a filter afterwards that filters out those elements where the count is lower than a given threshold.
Cheers, Aljoscha On Mon, 12 Dec 2016 at 22:30 Meghashyam Sandeep V <vr1meghash...@gmail.com> wrote: > Hi There, > > I have a streaming job which has source as Kafka and sink as Cassandra. I > have a use case where I wouldn't want to write some events to Cassandra > when there are more than 100 events for a given 'id' (field in my Pojo) in > 5mins. Is this a good usecase for SlidingWindows? Can I get the sliding > count for each key and then decide whether to add it to sink or not? > > Thanks, >
