[
https://issues.apache.org/jira/browse/FLINK-11107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718512#comment-16718512
]
ASF GitHub Bot commented on FLINK-11107:
----------------------------------------
Myasuka edited a comment on issue #7281: [FLINK-11107][state] Avoid memory
stateBackend to create arbitrary folders under HA path when no checkpoint path
configured
URL: https://github.com/apache/flink/pull/7281#issuecomment-446297350
@StephanEwen , since you have left annotations below:
~~~
to keep supporting the old behavior where default (JobManager) Backend + HA
mode = checkpoints in HA store
we add the HA persistence dir as the checkpoint directory if none other is
set
~~~
However, I'm wondering whether this keeps the same behavior as before. For
Flink-1.3, (JobManager) Backend + HA mode = only create `completedCheckpoint`
file under HA folder. On the other side, for Flink-1.6, this would create
another `job-id/chk-x/_metadata` except the completedCheckpoint file. Please
correct me if I am wrong.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [state] Avoid memory stateBackend to create arbitrary folders under HA path
> when no checkpoint path configured
> --------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-11107
> URL: https://issues.apache.org/jira/browse/FLINK-11107
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.6.2, 1.7.0
> Reporter: Yun Tang
> Assignee: Yun Tang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.3, 1.7.1
>
>
> Currently, memory state-backend would create a folder named with random UUID
> under HA directory if no checkpoint path ever configured. (the code logic
> locates within {{StateBackendLoader#fromApplicationOrConfigOrDefault}})
> However, the default memory state-backend would not only be created on JM
> side, but also on each task manager's side, which means many folders with
> random UUID would be created under HA directory. It would result in exception
> like:
> {noformat}
> The directory item limit of /tmp/flink/ha is exceeded: limit=1048576
> items=1048576{noformat}
> If this happens, no new jobs could be submitted only if we clean up those
> directories manually.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
