client.dns.lookup=”use_all_dns_ips” cache ttl

2023-05-31 Thread George Goh
Hi! I’m considering to use the following configuration for my producers and consumers: client.dns.lookup=”use_all_dns_ips” So that I only have a single DNS entry to manage the list of brokers. I also have another cluster connected via MirrorMaker 2, which serves as a failover. My question is,

Managing Kafka consumer buffer memory/size

2023-05-31 Thread Divya Pillai
Hi Team, We have a scenario where our application is processing messages at a slower rate. We want the consumer to stop fetching messages from the broker and re-fetch when an application is ready to process again. We have fetch.max.bytes but that doesn't manage the buffer memory. If my understand