[
https://issues.apache.org/jira/browse/FLINK-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191108#comment-16191108
]
ASF GitHub Bot commented on FLINK-7068:
---------------------------------------
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4358#discussion_r142637006
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/blob/BlobCacheCleanupTest.java
---
@@ -252,6 +280,36 @@ public void testJobDeferredCleanup() throws
IOException, InterruptedException {
}
/**
+ * Checks that BLOBs for the given <tt>jobId</tt> are cleaned up
eventually (after calling
+ * {@link PermanentBlobCache#releaseJob(JobID)}, which is not done by
this method!) (waits at
+ * most 30s).
+ *
+ * @param cache
+ * BLOB server
+ * @param jobId
+ * job ID or <tt>null</tt> if job-unrelated
+ * @param keys
+ * keys identifying BLOBs which were previously registered
for the <tt>jobId</tt>
+ */
+ static void verifyJobCleanup(PermanentBlobCache cache, JobID jobId,
List<BlobKey> keys)
+ throws InterruptedException, IOException {
+ // because we cannot guarantee that there are not thread races
in the build system, we
+ // loop for a certain while until the references disappear
+ {
+ long deadline = System.currentTimeMillis() + 30_000L;
--- End diff --
Since the cleanup interval is only a lower bound on the time of the actual
cleanup due to thread scheduling, we can't really use that. Basically, the way,
`verifyJobCleanup` is used by now is that one expects that the file is deleted
(but can't be sure) and therefore, we have this extra buffer of 30s. It could
be made configurable, when desired, I guess, but let's not do this now.
> change BlobService sub-classes for permanent and transient BLOBs
> ----------------------------------------------------------------
>
> Key: FLINK-7068
> URL: https://issues.apache.org/jira/browse/FLINK-7068
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Coordination, Network
> Affects Versions: 1.4.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
>
> A {{PermanentBlobStore}} should resemble use cases for BLOBs that are
> permanently stored for a job's life time (HA and non-HA).
> A {{TransientBlobStore}} should reflect BLOB offloading for logs, RPC, etc.
> which even does not have to be reflected by files.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
