Re: [DISCUSS] Add Polaris Observe API

2025-04-10 Thread Michael Collado
I think a simple metrics API makes a lot of sense. Decoupling this from events makes sense, as this would just be useful to query periodically for a variety of reasons not tied to event triggering. Mike On Thu, Apr 10, 2025 at 3:00 PM Jean-Baptiste Onofré wrote: > Hi Eric > > Thanks a lot for y

Re: [DISCUSS] Add Polaris Observe API

2025-04-10 Thread Jean-Baptiste Onofré
Hi Eric Thanks a lot for your feedback! As a first step, I would not store additional entities to store that, but more "querying" the existing entities (tables, etc) and the Iceberg meta (including table properties) to display that. I agree about finishing Event Listeners. In the meantime, I wou

Re: [DISCUSS] Add Polaris Observe API

2025-04-10 Thread Eric Maynard
I think the concept is really useful. The only thing I think which would require some more investigation is how exactly we implement this API -- where the data is stored, how long it's retained, etc. We might need to consider pushing this data out into another service or at least supporting such an

[DISCUSS] Add Polaris Observe API

2025-04-09 Thread Jean-Baptiste Onofré
Hi folks, I would like to discuss a proposal that I have in mind: the "observe" API. The purpose of this API is to return some metrics and gauges from Polaris, like: - what's number of entities (number of tables, views; etc) in a Polaris catalog - what's the number of times a entity as been acces