Hi - depending on the rules for how to filter/drop incoming messages (and depending on the mechanics of the library you use to consume the messages), it might be possible to filter out messages based on message headers, maybe? That way you would not need to deserialize the message key/value before deciding if the message should be dropped or not.
/Anders On Mon, Jun 13, 2022 at 5:53 PM abdelali elmantagui < abdelalielmanta...@gmail.com> wrote: > Hi All, > > I started a couple of weeks ago learning Kafka, and my goal is to optimize > an existing architecture that uses Kafka in its components. > The problem is that there many microservices that produce messages/events > to the the kafka topic and in the other hand theres other microservices > that consumes these messages/events and each microservice have to consume > all the messages and then filter which message are interested in and that > create a problem of huge memory usage because of the huge anount of objects > created in the memory after deserilaization of these messages. > > Am asking for any concept or solution that can help in this situation. > > Kind Regards, > Abdelali > > +--------------------+ > +-----------------+ | | +------------------+ > | microservices |---------->| Kafka topic | --------> | microservices | > +-----------------+ | | +------------------+ > +--------------------+ > >
