Hi Kamal, Thanks for the KIP! I think it is a good improvement to avoid users making many list_offset requests to starve other high priority requests.
Questions: 1. When consumer poll the partitions first time, it'll also try to call list_offset to get the offset to fetch. If this offset is located in remote storage, the consumer will need to wait for more time to start fetch real data. Doesn't that conflict with what we are trying to achieve? 2. Since we will add a new timeout attribute to `ListOffsetsRequest <https://sourcegraph.com/github.com/apache/kafka@trunk/-/blob/clients/src/main/resources/common/message/ListOffsetsRequest.json>`, please write it out clearly, like other KIP did (ex: KIP-1073 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1073%3A+Return+inactive+observer+nodes+in+DescribeQuorum+response#KIP1073:ReturninactiveobservernodesinDescribeQuorumresponse-DescribeMetadataQuorumRequest> ) 3. We added a new timeout attribute for the ListOffsetsRequest <https://sourcegraph.com/github.com/apache/kafka@trunk/-/blob/clients/src/main/resources/common/message/ListOffsetsRequest.json>, as well as a remote.list.offsets.request.timeout.ms broker config. Could you explain why we need 2 timeout configurations? I guess the broker one is for the purgatory? What will happen if the client timeout > broker timeout? And client timeout < broker timeout? Thank you. Luke On Mon, Aug 5, 2024 at 8:59 PM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > Bumping this thread. Please take a look. > > On Fri, Aug 2, 2024 at 12:32 PM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > Hi all, > > > > I would like to start a discussion thread on KIP-1075 > > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1075%3A+Introduce+delayed+remote+list+offsets+purgatory+to+make+LIST_OFFSETS+async> > to > > make the remote LIST_OFFSETS an async operation. > > > > The KIP is here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1075%3A+Introduce+delayed+remote+list+offsets+purgatory+to+make+LIST_OFFSETS+async > > > > Draft PR: https://github.com/apache/kafka/pull/16602 > > > > Please take a look. Feedbacks and suggestions are welcome. > > > > Thanks, > > Kamal > > >
