For problem 1, you could implement your own PartitionAssignor class to do static assignments. The downside of that is that if one consumer goes offline (due to hardware failure or a bug) then you aren't consuming from the respective partition.
For problem 2, if you are finding that the throughput with two partitions is not sufficient, I would first look and see if you have a bottleneck in your consuming application. If you have already ruled out that possibility, you may want to increase the partition count and add more consumers. Consumers acquire a lock on a partition, so if you want more consumers you must add more partitions. On Tue, Apr 25, 2017 at 12:25 PM, David Garcia <dav...@spiceworks.com> wrote: > For Problem 1, you will probably have to either use the low-level API, > and/or do manual partition assignment. > > For problem 2, you can simply re-publish the messages to a new topic with > more partitions…or, as in the first problem, just use the low level API. > You can also create more consumer groups (i.e. if you need to add a new > consumer of your topics, but all consumer groups are “full”, just make a > new one) > > -David > > On 4/24/17, 2:01 AM, "揣立武" <chuaili...@gmail.com> wrote: > > Hi, all. There are two problems when we use kafka about partitions > assigned > to consumers. > > Problem 1: Partitions will be reassigned to consumers when consumer > online > or offline, then the messages latency become higher. > > Problem 2: If we have two partitions, only two consumers can consume > messages。How let more consumers to consume, but expand partitions. > > Thanks! > > > -- Robert Quinlivan Software Engineer, Signal
