Hello, I am also working on something similar. Below is the pipeline design I have, sharing may be it can be helpful.
topic -> keyed stream on device-id -> window operation -> sink. You can PM me on further details. Thanks, Hemant On Tue, Feb 25, 2020 at 1:54 AM Marco Villalobos <mvillalo...@beyond.ai> wrote: > I need to collect timeseries data from thousands of IoT devices. Each > device has name, value, and timestamp published to one Kafka topic. The > event time timestamps are in order only relation with the individual > device, but out of order with respect to other devices. > > > > Is there a way to aggregate a 15 minute window of this in which each IoT > devices gets aggregated with its own event time? > > > > I would deeply appreciate if somebody could guide me to an approach for > solving this in Flink. > > > > I wish there was a group chat for these type of problems. > > >
