Hi there,
I recently ran into an issue where I noticed that we had many old broker
and <default> configs set in zookeeper that were migrated over in KRaft
mode.
For example, this was a cluster default from 2020 that was moved over:
get /config/brokers/<default>
{"version":1,"config":{"num.network.threads":"6","num.io.threads":"8"}}
We currently have these configs set up in our broker properties and
controller properties file in the configmap:
num.network.threads=16
num.io.threads=48
The cluster is now running in KRaft mode but we've encountered issues with
the DynamicConfigPublisher modifying/using these default configs instead of
the ones we set up in the configmap.
{"source_class":"kafka_log","level":"INFO","time":"","msg":"[DynamicConfigPublisher
broker id=0] Updating cluster configuration : num.network.threads ->
6,num.io.threads -> 8,num.replica.fetchers -> 3 "}
{"source_class":"kafka_log","level":"INFO","time":"","msg":"Resizing
network thread pool size for ListenerName(CONTROLLER) listener from 12 to 6
"}
We never saw this issue in Zookeeper mode but this is blocking our Cruise
Control rebalances from executing.
A couple questions I have:
1. Is this expected? Should the controllers be looking at the default
config?
2. Is there a way to add some kind of cleanup to Zookeeper to clean up old
brokers that no longer exist from /config/brokers?
3. How can I point KRaft controllers to use the values we have set up in
the config map and not these default values?
4. If I were to dynamically set a value for a broker, why doesn't it
override the default?
Any answers/help would be appreciated! Thank you!
--
Sanaa
