LuciferYang opened a new pull request, #13528: URL: https://github.com/apache/gravitino/pull/13528
### What changes were proposed in this pull request? The Lance dataset holder now carries a reader count. `listPartitionStatistics` borrows the dataset through a pinned wrapper, the cache removal listener's close is deferred while readers are in flight, and the dataset closes when the last reader leaves (or immediately when none were). The physical close is gated so the eviction path and the last-reader path cannot both close the native dataset. ### Why are the changes needed? With the dataset cache enabled (`datasetCacheSize > 0`) and more tables accessed concurrently than the cache holds, a size eviction could close a dataset another thread had borrowed and was still scanning: `listPartitionStatistics(A)` scans A while a concurrent `getDataset(B)` evicts A, and A's removal listener closed the native dataset underneath the scan. The two paths lock different table identifiers, so nothing serialized them. Closing on the last reader introduced a second race where the eviction and the final `readerDone` could both reach the physical close, which the reader-count gate now prevents. Fix: #13523 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added `testInFlightReaderSurvivesCacheReplaceEviction`, which pins a reader, evicts the entry, and asserts the dataset stays open until the last reader leaves. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
