As I mentioned in https://github.com/apache/polaris/issues/3890, supporting multiple data sources is not a trivial change. I would strongly recommend starting with a design document to carefully evaluate the architectural implications and long term impact.
A REST endpoint to query metrics seems reasonable given the current JDBC based persistence model. That said, we may also consider alternative storage models. For example, if we later adopt a time series system such as Prometheus to store metrics, the query model and access patterns would be fundamentally different. Designing the REST API without considering these potential evolutions may limit flexibility. I'd suggest to start with the use case. Yufei On Fri, Feb 27, 2026 at 3:42 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Anand, > > Sharing my view... subject to discussion: > > 1. Adding non-IRC REST API to Polaris is perfectly fine. > > Figuring out specific endpoint URIs and payloads might require a few > roundtrips, so opening a separate thread for that might be best. > Contributors commonly create Google Docs for new API proposals too (they > fairly easy to update as the email discussion progresses). > > There was a suggestion to try Markdown (with PRs) for proposals [1] ... > feel free to give it a try if you are comfortable with that. > > 2. Could you clarify whether you mean end user utilities or admin user > utilities? In the latter case those might be more suitable for the Admin > CLI (java) not the Python CLI, IMHO. > > Why would these utilities be common with events? IMHO, event use cases are > distinct from scan/commit metrics. > > 3. I'd prefer separating metrics persistence from MetaStore persistence at > the code level, so that they could be mixed and matched independently. The > separate datasource question will become a non-issue with that approach, I > guess. > > The rationale for separating scan metrics and metastore persistence is that > "cascading deletes" between them are hardly ever required. Furthermore, the > data and query patterns are very different so different technologies might > be beneficial in each case. > > [1] https://lists.apache.org/thread/yto2wp982t43h1mqjwnslswhws5z47cy > > Cheers, > Dmitri. > > On Fri, Feb 27, 2026 at 6:19 PM Anand Kumar Sankaran via dev < > [email protected]> wrote: > > > Thanks all. This PR is merged now. > > > > Here are the follow-up features / work needed. These were all part of > the > > merged PR at some point in time and were removed to reduce scope. > > > > Please let me know what you think. > > > > > > 1. A REST API to paginate through table metrics. This will be non-IRC > > standard addition. > > 2. Utilities for managing old records, should be common with events. > > There was some discussion that it belongs to the CLI. > > 3. Separate datasource (metrics, events, even other tables?). > > > > > > Anything else? > > > > - > > Anand > > > > >
