Hi,
I’m trying to use kafka for the following use case:
* My app receive events from external source;
* It aggregates it during some window (say, 10 minutes);
* Every 10 minutes it writes this aggregated data to external service.
I was thinking about using kafka for this, so I’ll write incoming events to a
kafka topic and then read events back, aggregating it and commiting only after
writing to external service occurs (so I’ll get at-least-once delivery).
My only concern is that consumer lag will be non-zero most of the time.
Is it an issue? Is there a better ways to aggregate data from kafka topic with
time window?
Thanks.
