apoorvmittal10 opened a new pull request, #19612: URL: https://github.com/apache/kafka/pull/19612
The PR fixes the issue when ShareAcknowledgements are piggybacked on ShareFetch. The current default configuration in clients sets `batch size` and `max fetch records` as per the `max.poll.records` config, default 500. Which means all records in a single poll will be fetched and acknowledged. Also the default configuration for inflight records in a partition is 200. Which means prior fetch records has to be acknowledged prior fetching another batch from share partition. The piggybacked share fetch-acknowledgement calls from KafkaApis are async and later the response is combined. If respective share fetch starts waiting in purgatory because all inflight records are currently full, hence when startOffset is moved as part of acknowledgement, then a trigger should happen which should try completing any pending share fetch requests in purgatory. Else the share fetch requests wait in purgatory for timeout though records are available, which dips the share fetch performance. The regular fetch has a single criteria to land requests in purgatory, which is min bytes criteria, hence any produce in respective topic partition triggers to check any pending fetch requests. But share fetch can wait in purgatory because of multiple reasons: 1) Min bytes 2) Inflight records exhaustion 3) Share partition fetch lock competition. The trigger already happens for 1 and current PR fixes 2. We will investigate further if there should be any handling required for 3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
