LuciferYang opened a new issue, #13523: URL: https://github.com/apache/gravitino/issues/13523
### Version main branch ### Describe what's wrong When the Lance partition-statistics dataset cache is enabled (`datasetCacheSize > 0`) and more tables are accessed concurrently than the cache holds, a size eviction can close a dataset that another thread has just borrowed and is still scanning. `listPartitionStatistics(A)` scans A's cached dataset while a concurrent `getDataset(B)` inserts B and triggers a size eviction of A; A's removal listener closes the native dataset underneath the in-flight scan. The two operations are not serialized because they hold locks on different table identifiers. ### Error message and/or stacktrace Native use-after-free surfacing as a "dataset closed" error during a concurrent statistics list. ### How to reproduce Set `datasetCacheSize` smaller than the number of tables accessed concurrently, then run `listPartitionStatistics` on one table while another thread calls `getDataset` on a different table so the first table is evicted mid-scan. ### Additional context The holder needs a reader count so the removal listener's close is deferred while readers are in flight and the dataset closes when the last reader leaves. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
