Hi everyone, During the conversation of the Scan API for REST spec, we touched on the topic of pagination when REST response is large or takes time to be produced.
I just want to discuss this separately, since we also see the issue for ListNamespaces and ListTables/Views, when integrating with a large organization that has over 100k namespaces, and also a lot of tables in some namespaces. Pagination requires either keeping state, or the response to be deterministic such that the client can request a range of the full response. If we want to avoid keeping state, I think we need to allow some query parameters like: - *start*: the start index of the item in the response - *limit*: the number of items to be returned in the response So we can send a request like: *GET /namespaces?start=300&limit=100* *GET /namespaces/ns/tables?start=300&limit=100* And the REST spec should enforce that the response returned for the paginated GET should be deterministic. Any thoughts on this? Best, Jack Ye
