You may need to consider replication factor as well? Here you increased 4
times of partitions but if you are having 3 replication factor then it will
be (4 * 3) times of partition copies in the cluster.
On Sat, Nov 21, 2015 at 12:43 AM, Chen Song wrote:
> Any thoughts on this topic. Not sure if
Any thoughts on this topic. Not sure if others have seen the same spike as
us.
On Tue, Nov 17, 2015 at 3:51 PM, Chen Song wrote:
> BTW, we are running Kafka 0.8.2.2.
>
> On Tue, Nov 17, 2015 at 3:48 PM, Chen Song wrote:
>
>> We have a cluster of 32 nodes cluster, with a few topics. When we set
BTW, we are running Kafka 0.8.2.2.
On Tue, Nov 17, 2015 at 3:48 PM, Chen Song wrote:
> We have a cluster of 32 nodes cluster, with a few topics. When we set 100
> partitions for each topic, the overall CPU usage is 30%, and when we
> increase to 400 partitions, the CPU goes up over 65% - 70% som
We have a cluster of 32 nodes cluster, with a few topics. When we set 100
partitions for each topic, the overall CPU usage is 30%, and when we
increase to 400 partitions, the CPU goes up over 65% - 70% sometimes. The #
of producers are not changed.
I understand that more partitions will drive up b
