Hi streams Dev community @matthias, @bruno Any inputs on above issue? Is this a bug in the streams library wherein the input topic removed from streams processor topology, the underlying consumer group still reporting lag against those?
On Wed, Aug 9, 2023 at 4:38 PM Pushkar Deole <pdeole2...@gmail.com> wrote: > Hi All, > > I have a streams application with 3 instances with application-id set to > applicationV1. The application uses processor API with reading from source > topics, processing the data and writing to destination topic. > Currently it consumes from 6 source topics however we don't need to > process data any more from 2 of those topics so we removed 2 topics from > the source topics list. We have configured Datadog dashboard to report and > alert on consumer lag so after removing the 2 source topics and deploying > application, we started getting several alerts about consumer lag on > applicationV1 consumer group which is underlying consumer group of the > streams application. When we looked at the consumer group from kafka-cli, > we could see that the consumer group is reporting lag against the topics > removed from source topic list which is reflecting as increasing lag on > Datadog monitoring. > > Can someone advise if this is expected behavior? In my opinion, this is > not expected since streams application no more has those topics as part of > source, it should not report lag on those. >
