Hello users,
I'm trying to use the log.cleaner.io.max.bytes.per.second configuration to
limit disk usage when the log cleaner runs, but it doesn't seem to work as
expected.
When looking at the logs, I can see it still uses far more than configured.
This is the output with log.cleaner.io.max.bytes
I forgot to mention earlier, another way to detect this is to check the
logs for the following lines:
[2022-08-25 17:44:05,308] ERROR [Broker id=0] Topic Id in memory:
jKTRaM_TSNqoc1233I2aYOQ does not match the topic Id for partition
myTopic-0 provided in the request: nI-J1234wQwGiylMfm8k13w.
(sta
Oh yes! You can run a script which checks that the topic IDs in Zk are
consistent with the ones that are available on partition.metadata.
An example script would look something like this:
https://gist.github.com/divijvaidya/8a940a3d67bc8b215c95da54f5eff961
I will also add this script to the JIRA
