Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4358#discussion_r142637006
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/blob/BlobCacheCleanupTest.java
---
@@ -252,6 +280,36 @@ public void testJobDeferredCleanup() throws
IOException, InterruptedException {
}
/**
+ * Checks that BLOBs for the given <tt>jobId</tt> are cleaned up
eventually (after calling
+ * {@link PermanentBlobCache#releaseJob(JobID)}, which is not done by
this method!) (waits at
+ * most 30s).
+ *
+ * @param cache
+ * BLOB server
+ * @param jobId
+ * job ID or <tt>null</tt> if job-unrelated
+ * @param keys
+ * keys identifying BLOBs which were previously registered
for the <tt>jobId</tt>
+ */
+ static void verifyJobCleanup(PermanentBlobCache cache, JobID jobId,
List<BlobKey> keys)
+ throws InterruptedException, IOException {
+ // because we cannot guarantee that there are not thread races
in the build system, we
+ // loop for a certain while until the references disappear
+ {
+ long deadline = System.currentTimeMillis() + 30_000L;
--- End diff --
Since the cleanup interval is only a lower bound on the time of the actual
cleanup due to thread scheduling, we can't really use that. Basically, the way,
`verifyJobCleanup` is used by now is that one expects that the file is deleted
(but can't be sure) and therefore, we have this extra buffer of 30s. It could
be made configurable, when desired, I guess, but let's not do this now.
---
