[
https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714033#comment-16714033
]
Yun Tang commented on FLINK-11107:
----------------------------------
@ [~StephanEwen] , as you commented in code below:
{noformat}
// to keep supporting the old behavior where default (JobManager) Backend + HA
mode = checkpoints in HA store
// we add the HA persistence dir as the checkpoint directory if none other is
set{noformat}
However, I'm wondering whether this keeps the same behavior as before. For
Flink-1.3, (JobManager) Backend + HA mode = only create {{completedCheckpoint}}
file under HA folder. On the other side, for Flink-1.6, this would create
another {{job-id}}/{{chk-x/}}{{_metadata}} except the {{completedCheckpoint}}
file.
> [state] Avoid memory stateBackend to create arbitrary folders under HA path
> when no checkpoint path configured
> --------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-11107
> URL: https://issues.apache.org/jira/browse/FLINK-11107
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.6.2, 1.7.0
> Reporter: Yun Tang
> Assignee: Yun Tang
> Priority: Major
> Fix For: 1.6.3, 1.7.1
>
>
> Currently, memory state-backend would create a folder named with random UUID
> under HA directory if no checkpoint path ever configured. (the code logic
> locates within {{StateBackendLoader#fromApplicationOrConfigOrDefault}})
> However, the default memory state-backend would not only be created on JM
> side, but also on each task manager's side, which means many folders with
> random UUID would be created under HA directory. It would result in exception
> like:
> {noformat}
> The directory item limit of /tmp/flink/ha is exceeded: limit=1048576
> items=1048576{noformat}
> If this happens, no new jobs could be submitted only if we clean up those
> directories manually.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
