Hi Omnia, Thanks for this KIP, this is a valuable improvement, and apologies for my late feedback on this.
IS1: Without an indication of how many pages or items are left, it is impossible for the client to calculate progress. Having the cursor in the response just indicates whether there are more pages or not, it does not distinguish whether there is just one left, or a thousand more to go. Is it possible to include the indication, taking authorization into account? IS2: It is a shame that the pagination fields are different for each RPC. This allows for divergences or mistakes as e.g. the KIP currently includes GroupId under the Cursor for MetadataRequest. I was wondering if instead of identifying the next object in the cursor, we could instead indicate its index, always using an integer value. Or perhaps the page number, considering the value of ResponsePagingationLimit. Is is possible to have a standard way to deal with pagination, and use consistently for all RPCs? IS3: I don't quite understand the following rejected alternative: > Make the pagination more generic by extending FieldSpec to include > pagination flag that indicate if a field will be used for pagination > or not. This would simplify the code however, it will be tricky to > use on fields like TopicPartition which usually is defined using > two fields topic and partition index which usually represented as > nested field. We can consider this only for Response to add an > extra flag to FieldSpec indicating that a field will have NextCursor > field but it is not clear what will be the usage of this except > making getting the next cursor maybe more generic to build but > arguable we can do this without touching FieldSpec. The message specification format is only used in this project, so we can extend it or redefine it freely. If pagination is implemented in a consistent manner across RPCs, it would be valuable to have some high-level way to enable it in the RPC definition. Could you add a bit more detail about why that's not possible? Thanks, -- Igor
