Re: Optimizations to GlueCatalog

2024-03-06 Thread Ryan Blue
Chetas, Iceberg has an implementation of what you're talking about. There's a caching layer implemented as a catalog, `CachingCatalog`. That's turned on by default in the Flink catalog but the default interval is 30s. Maybe you need to extend that interval by setting `cache.expiration-interval-ms`

Optimizations to GlueCatalog

2024-03-06 Thread Chetas Joshi
Hi Community, I am working on loading iceberg data from S3 using Flink. I am using GlueCatalog for storing the iceberg table metadata. I found that the GlueCatalog’s loadTable call (implemented