Hi, We are using Kafka streams 2.1.1 and leveraging DSL APIs for doing stateful processing (aggregation). As per our use case we ended up creating ~400 sub topologies. Our observation is that based on the number of partitions in the input/source topic, corresponding stream tasks get created for each of these sub topologies. Wanted to know if having so many sub topologies is a bad idea OR is there a practical limit on number of sub topologies?
Other observation is that due to more number of sub topologies, Kafka cluster takes more time to create corresponding internal topics & respond back to kafka streams application resulting in endless "(Re-)joining group" loop. We have overcome this by increasing the request & session time out values. But wanted to be sure if it will have any other side effects.
