Hi Jason, Yes, replication throttling does work, however, the “entity-default” configuration does not work in Kafka 3.5, you have to set the broker configs individually with “entity-name". This is a bug (https://issues.apache.org/jira/browse/KAFKA-10190), which I fixed in Kafka 3.9.
Hope this helps, Harry On 2024/12/10 17:06:57 Jason Taylor wrote: > Hello, > > Has anyone successfully made replication throttling work? I am doing the > following on Apache Kafka 3.5 > > ./kafka_2.13-3.7.1/bin/kafka-configs.sh \ > --bootstrap-server $BROKERS \ > --entity-type brokers \ > --entity-default \ > --alter \ > --add-config leader.replication.throttled.rate=50000000 \ > --command-config client.properties > > ./kafka_2.13-3.7.1/bin/kafka-configs.sh \ > --bootstrap-server $BROKERS \ > --entity-type brokers \ > --entity-default \ > --alter \ > --add-config follower.replication.throttled.rate=50000000 \ > --command-config client.properties > > TOPICS=$(./kafka_2.13-3.7.1/bin/kafka-topics.sh --bootstrap-server > $BROKERS --list --command-config client.properties) > for TOPIC in $TOPICS; do > echo "Applying throttle to topic: $TOPIC" > ./kafka_2.13-3.7.1/bin/kafka-configs.sh \ > --bootstrap-server $BROKERS \ > --entity-type topics \ > --entity-name $TOPIC \ > --alter \ > --add-config > leader.replication.throttled.replicas=*,follower.replication.throttled.replicas=* > \ > --command-config client.properties > done > > I have verified that the settings are visible when describing dynamic > configs for both topic and broker. If I then stop a broker to build up a > replication backlog and restart I still see a burst in replication bytes > in/out well beyond 50MB. > > Many Thanks, > Jason > -- Thought Machine Group Limited, a company registered in England & Wales. Registered number: 11114277. Registered Office: 5 New Street Square, London EC4A 3TW <https://maps.google.com/?q=5+New+Street+Square,+London+EC4A+3TW&entry=gmail&source=g>. The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future.
