Hi We have a couple of hundred topics - each carrying a similar but distinct message type but to keep the total partition count down each only has 3 partitions.
If I start Kafka-streams consuming all topics only 3 threads ever get assigned any partitions. I think the first thread to start gets the first partition of each topic, and so on until the 3rd thread, after that all the partitions are assigned - any further threads are just left idle. Is there any way to make the partition assignment smarter and either add a random element that moves partitions when further consumers start or considers all the partitions of subscribed topics together when assigning them? Our only alternative is creating many more partitions for each topic - and we're worried about how far this will scale. Thank you Mike G -- - MikeG http://en.wikipedia.org/wiki/Common_misconceptions <http://en.wikipedia.org/wiki/Special:Random>
