Hi Peter Kafka publishes metrics regarding usage of network and io threads. You should observe the following metrics `kafka.network:type=SocketServer,name=NetworkProcessorAvgIdlePercent` which should ideally be > 0.3. This metric helps you understand the usage of network threads and if you find that it goes below the threshold, perhaps it's time to increase the number of network threads. Another interesting point here is that increasing the number of network threads beyond the number of processors in the machine can be a disadvantage as well. This is because, ideally, the network threads should not be blocked on IO and they should be forwarding requests quite fast to handler/io threads. Adding network threads beyond the number of cores will lead to excessive context switching.
Also, please consider limiting the bandwidth usage for partition migration. You can find the instructions here: https://kafka.apache.org/33/documentation.html#rep-throttle Let us know what you found! -- Divij Vaidya On Tue, Nov 29, 2022 at 5:09 PM Péter Sinóros-Szabó <peter.sinoros-sz...@wise.com.invalid> wrote: > Hi, > > How can I know if I have enough network threads configured with > num.network.threads? > > I see Producers are slower to produce when we move partitions between > brokers, so I'm thinking maybe there isn't enough network threads to > process the requests from both the consumers and from the other brokers to > create the new replicas. > > Cheers, > Peter >
