Precision of kafka quotas

2021-12-22 Thread Thouqueer Ahmed
Hi, We are trying to setup a utilization plot for our multi-tenant kafka cluster. Setup is Kafka: 2.6, Scala 2.12, Ubuntu 18 We noticed 2 issues: 1. Throttling happens even when quota violation doesn't occur As per JMX metrics: Producer byte-rate is 148KB/s, Consumer byte-rate

Question on kafka connect's incremental rebalance algorithm

2021-04-05 Thread Thouqueer Ahmed
Hi, What would happen when new worker joins after the synchronization barrier ? As per code -> performTaskAssignment function of IncrementalAssignor -> Boolean canRevoke is false when it is called during the 2nd rebalance. Hence revocation is skipped and only assignment is done. T