Hi Steven, On Tue, Feb 3, 2026 at 2:04 AM Steven Wu <[email protected]> wrote: > > Hi, > > I would like to discuss a proposal to add batch load endpoints for tables and > views to the REST spec. > https://docs.google.com/document/d/1VW5hgaaajRWtp5KbOU3s83YyoyPi5WOSvHtoJ_yXzJs/edit?tab=t.0 > > It can help with the use cases > * Catalog federation: improve refresh throughput and reduce load on the > source catalog > * Improve planning performance by loading multiple referenced tables in one > request. > * Improve MV freshness evaluation if referencing multiple source tables > > Thanks, > Steven
I can think of one scenario where this batch load would be very useful: for serverless engines, which may need to reload all their tables from the catalog once they get back online(for mapping its internal schema using the batch load response). After looking at the batch-load API, it seems to require users to provide an array of table identifiers in the payload, which sounds reasonable to me. However, I have a question: could it offer an option to load all tables at once without specifying the table array? And if the number of tables is large, could it support pagination options such as **limit** and **offset**? I imagine the pagination could be tricky, since other engines may create tables dynamically. One approach I can think of is to first issue a listTables request and then follow up with additional batchLoadTables calls. I'm just wondering whether that's the only approach. -- Regards Junwang Zhao
