I have a case in which I'm iterating over the results of a poll(..) in KafkaConsumer. While I'm iterating over the records, the thread is blocked. e.g., I'm writing to a BlockingQueue that's full.
Will this have any impact on the KafkaConsumer? I have one more follow up question. Let's say I have 20 topics. I plan to spawn 20 KafkaConsumer's to process each topic because I'm looking to rate limit the speed at which the records are processed to avoid overloading external systems. The proper usage of KafkaConsumer in this case would be: - Create a thread with a new instance of KafkaConsumer used in it. - Ensure the group id for that consumer is unique - Ensure only one consumer for each topic is running to avoid re-balancing -- -Richard L. Burton III @rburton
