Hi Iceberg Community, I was also exploring Manifest file caching feature which was introduced in Iceberg https://github.com/apache/iceberg/pull/4518 for Presto execution engine. And as I checked Cache Metrics are getting computed and recorded in Iceberg library for same, but didn’t find a way to access these metrics on execution engine side. To validate the cache usage we would require atleast cache metrics access for this feature. Could you please let me know how could I do that?
Thanks, Reetika On 2022/12/09 15:40:04 Gabor Kaszab wrote: > Hey Iceberg Community, > > Scan metrics support was introduced in 1.1.0 that is pretty nice to collect > some per-query stats like 'total planning duration' and such. > However, I haven't found a way to get process level metrics at the moment. > The particular thing I wanted to achieve is that there was a Manifest file > content cache <https://github.com/apache/iceberg/pull/4518> introduced > recently where the stats are logged, but aren't exposed through the API. > > Such an API could be used to expose stats for various kinds of caches (e.g. > number of hits and misses), we could count the number of specific API calls > (like planFiles()) and I guess many more other useful stuff. > > With this mail I'd like to initiate a discussion around this topic. If the > community is open for such a change I could start working on an initial > solution that would expose the Manifest file content cache stats but could > be extended later on with more metrics. > > Cheers, > Gabor >
