Prakhar Kulshrestha created KAFKA-21241:
-------------------------------------------
Summary: Increase the number of consumers in the share consumer
group
Key: KAFKA-21241
URL: https://issues.apache.org/jira/browse/KAFKA-21241
Project: Kafka
Issue Type: Improvement
Reporter: Prakhar Kulshrestha
Currently the max number of consumers in a share consumer group is default to
200 and max to 1000. In bigger platforms where batch processing along with
slow consumer use cases limiting to just 1000 consumer is very
restrictive.{*}{*}
One of the scenario is a batch processing of millions of records and having to
parallel process 20-30k records in parallel using spring kafka concurrency
feature would mean we would need at least 20-30k consumers in a consumer groups
with around 1k pods. Current 1000 limit just let it have ~60pods.
Request this to be reevaluated and increase the number of consumers that a
share group can have.
*group.share.max.size*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
