[
https://issues.apache.org/jira/browse/FLINK-20505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17245071#comment-17245071
]
Xintong Song commented on FLINK-20505:
--------------------------------------
[~zoucao],
I don't think it requires a FLIP for this issue. But we do need to get
consensus on whether and how we want to do with the issue.
If you agree with my opinions, let's wait for [~ZhenqiuHuang]'s reply. Once
there's a consensus, I'll assign you.
> Yarn provided lib does not work with http paths.
> ------------------------------------------------
>
> Key: FLINK-20505
> URL: https://issues.apache.org/jira/browse/FLINK-20505
> Project: Flink
> Issue Type: Bug
> Components: Deployment / YARN
> Affects Versions: 1.12.0, 1.11.2
> Reporter: Xintong Song
> Assignee: Xintong Song
> Priority: Major
>
> If an http path is used for provided lib, the following exception will be
> thrown on the resource manager side:
> {code:java}
> 2020-12-04 17:01:28.955 ERROR org.apache.flink.yarn.YarnResourceManager -
> Could not start TaskManager in container containerXXXXXX.
> org.apache.flink.util.FlinkException: Error to parse
> YarnLocalResourceDescriptor from YarnLocalResourceDescriptor{key=XXXXX.jar,
> path=https://XXXXXXX.jar, size=-1, modificationTime=0, visibility=APPLICATION}
> at
> org.apache.flink.yarn.YarnLocalResourceDescriptor.fromString(YarnLocalResourceDescriptor.java:99)
> at
> org.apache.flink.yarn.Utils.decodeYarnLocalResourceDescriptorListFromString(Utils.java:721)
> at org.apache.flink.yarn.Utils.createTaskExecutorContext(Utils.java:626)
> at
> org.apache.flink.yarn.YarnResourceManager.getOrCreateContainerLaunchContext(YarnResourceManager.java:746)
> at
> org.apache.flink.yarn.YarnResourceManager.createTaskExecutorLaunchContext(YarnResourceManager.java:726)
> at
> org.apache.flink.yarn.YarnResourceManager.startTaskExecutorInContainer(YarnResourceManager.java:500)
> at
> org.apache.flink.yarn.YarnResourceManager.onContainersOfResourceAllocated(YarnResourceManager.java:455)
> at
> org.apache.flink.yarn.YarnResourceManager.lambda$onContainersAllocated$1(YarnResourceManager.java:415)
> {code}
> The problem is that, `HttpFileSystem#getFilsStatus` returns file status with
> length `-1`, while `YarnLocalResourceDescriptor` does not recognize the
> negative file length.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
