Hi Kamal, Thanks for the update. LC1: I see. Thanks. LC2: What I still don't understand is what is the relationship between remote.list.offsets.request.timeout.ms V.S. request.timeout/default.api.timeout. Suppose we set request timeout to 30 seconds, default.api.timeout=5 mins and remote.list.offsets.request.timeout.ms = 6 mins. So, when Admin sends a list offset request that needs to query remote storage, when will it throw timeout exception? 30 secs or 5 mins or 6 mins? We might need to make it clear in the KIP.
LC3: "Admin sends only one request and wait for upto default-api-timeout. (eg) If the admin is configured with default-api-timeout as 5 mins and request-timeout as 30 seconds. And, the server takes 50 seconds to process the LIST_OFFSETS request, then the admin sends only one LIST_OFFSETS request, then receives the request from server after 50 seconds." In the end of the section, it should be receives the "response" from server? LC4: I found the different consumer and admin behavior when setting "request.timeout" and "default.api.timeout" is confusing. Are they expected or a bug? Thank you. Luke On Thu, Aug 8, 2024 at 10:06 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Hi Luke, > > Thanks for the review! > > LC1: When the consumer starts to read data, then it might need the below > offsets: > earliest, latest, and last-committed-offset based on the > "auto.offset.reset" config. > > The earliest and latest offsets have special timestamps -2 and -1, those > timestamp corresponding offsets are cached in the > broker memory and get served immediately. The last-committed-offset is also > cached in the GroupMetadata and > gets served in the "OFFSET_FETCH" request. Unless the consumer > explicitly uses the KafkaConsumer#offsetForTimes API, > there won't be any delay in serving the data from the local log. > > In this KIP, we are trying to address the case in which multiple consumers > start at the same time and use the 'offsetForTimes` LIST_OFFSETS API, > assuming the remote requests are slow, then it should not block other > PRODUCE/FETCH requests. > > LC2: Sorry for the confusion. We were planning to introduce only the broker > config "remote.list.offsets.timeout.ms". > If we add the timeout in the ListOffsetsRequest.json, then when old clients > talk with the new broker, we don't have > a timeout to set on the server. Moved adding the timeout to the > ListOffsetsRequest to the rejected alternatives section. > > -- > Kamal >
