Hello users,
I'm trying to use the log.cleaner.io.max.bytes.per.second configuration to
limit disk usage when the log cleaner runs, but it doesn't seem to work as
expected.
When looking at the logs, I can see it still uses far more than configured.
This is the output with log.cleaner.io.max.bytes.per.second=1024
[2022-12-22 16:14:49,462] INFO [kafka-log-cleaner-thread-0]:
Log cleaner thread 0 cleaned log __consumer_offsets-9 (dirty
section = [13509895330, 13510879425])
100,2 MB of log processed in 1,7 seconds (60,1 MB/sec).
Indexed 99,7 MB in 0,8 seconds (117,5 Mb/sec, 50,8% of total time)
Buffer utilization: 0,1%
Cleaned 100,2 MB in 0,8 seconds (122,2 Mb/sec, 49,2% of total time)
Start size: 100,2 MB (988642 messages)
End size: 0,5 MB (4547 messages)
99,5% size reduction (99,5% fewer messages)
I did update the value trough kafak-config this way:
kafka-configs.sh --bootstrap-server localhost:59092 --entity-type brokers
--command-config ~/server.admin.properties --entity-default --alter
--add-config log.cleaner.io.max.bytes.per.second=1024
Which shows up correctly in broker config:
log.cleaner.io.max.bytes.per.second=1024.0 sensitive=false
synonyms={DYNAMIC_DEFAULT_BROKER_CONFIG:log.cleaner.io.max.bytes.per.second=1024,
DEFAULT_CONFIG:log.cleaner.io.max.bytes.per.second=1.7976931348623157E308}
Any idea ? Something I'm missing ?
--
[image: Chronopost] <https://www.chronopost.fr/fr?xtatc=INT-149>
*Nicolas Carlot*
*Lead dev*Direction des Systèmes d'Information
3 boulevard Romain Rolland
75014 Paris
[image: img] <https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L3A4PQ> [image:
img] <https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pzPQ> [image: img]
<https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pRPQ> [image: img]
<https://mailsign.chronopost.fr/linkc/K0ppSHlnPT0-L1pVPQ>
[image: img]
[image: img]
