Hi folks, I would like some help understanding what is going on in the below:
[2021-05-03 12:53:26,856] INFO [GroupMetadataManager brokerId=0] Finished loading offsets and group metadata from __consumer_offsets-42 in 1659 milliseconds, of which 1647 milliseconds was spent in the scheduler. (kafka.coordinator.group.GroupMetadataManager) [2021-05-03 12:53:26,871] INFO [GroupMetadataManager brokerId=0] Finished loading offsets and group metadata from __consumer_offsets-43 in 1674 milliseconds, of which 1660 milliseconds was spent in the scheduler. (kafka.coordinator.group.GroupMetadataManager) [2021-05-03 12:53:26,871] INFO [GroupMetadataManager brokerId=0] Finished loading offsets and group metadata from __consumer_offsets-13 in 1674 milliseconds, of which 1674 milliseconds was spent in the scheduler. (kafka.coordinator.group.GroupMetadataManager) It seems from looking online that this is a long time for these to take place. What is odd to me is that it does not look like a timeout. Sometimes the scheduler takes the full time, and sometimes it is not the full time. The actual loading increases in time too. I have been able to figure out where this log message was added to the code, but it doesn’t do a lot to help me understand the issue: https://github.com/apache/kafka/pull/8155 Ultimately, I don’t really know what the scheduler is doing (other than scheduling, presumably). Is Zookeeper involved in scheduling? Aside from a virus scan about 12 hours before these logs, there is not any obvious pressure on the system, though this is not 100% clear either. We are working to make it more clear that the system is not under pressure. I am definitely more of a systems person than a Kafka person, so there may be something obvious here I am missing on the Kafka side. I will not be offended! Best Regards, Doug This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
