[
https://issues.apache.org/jira/browse/FLINK-15378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003237#comment-17003237
]
ouyangwulin edited comment on FLINK-15378 at 12/26/19 5:58 AM:
---------------------------------------------------------------
[~fly_in_gis]
>> For request 1,
Change plugin to not use '{{classloader.parent-first-patterns.default}} '
is a good idea。
>> For request 2,
Aggregated hdfs-site.xml can support multiple hdfs cluster in same
kerberoes, It create one filesystem instantiate to write to diff hdfs
cluster。But in our sense，We have one hdfs cluster with kerberoes， Other cluster
isn't need kerberoes，So Aggregated hdfs-site.xml is not work 。It also not need
plugin in `$FLINK_HOME/pulgins` ，The user jar can content FileSystem plugin。
was (Author: ouyangwuli):
[~fly_in_gis]
>> For request 1,
Change plugin to not use '{{classloader.parent-first-patterns.default}} '
is a good idea。
>> For request 2,
Aggregated hdfs-site.xml can support multiple hdfs cluster in same
kerberoes, It create one filesystem instantiate to write to diff hdfs
cluster。But in our sense，We have one hdfs cluster with kerberoes， Other cluster
is don't need kerberoes。It also not need plugin in `$FLINK_HOME/pulgins` ，The
user jar can content FileSystem plugin。
> StreamFileSystemSink supported mutil hdfs plugins.
> --------------------------------------------------
>
> Key: FLINK-15378
> URL: https://issues.apache.org/jira/browse/FLINK-15378
> Project: Flink
> Issue Type: Improvement
> Components: API / Core
> Affects Versions: 1.9.2, 1.11.0
> Reporter: ouyangwulin
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Attachments: jobmananger.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [As report from
> maillist|[https://lists.apache.org/thread.html/7a6b1e341bde0ef632a82f8d46c9c93da358244b6bac0d8d544d11cb%40%3Cuser.flink.apache.org%3E]]
> Request 1: FileSystem plugins not effect the default yarn dependecies.
> Request 2: StreamFileSystemSink supported mutil hdfs plugins under the same
> schema
> As Problem describe :
> when I put a ' filesystem plugin to FLINK_HOME/pulgins in flink', and the
> clas{color:#172b4d}s '*com.filesystem.plugin.FileSystemFactoryEnhance*'
> implements '*FileSystemFactory*', when jm start, It will call
> FileSystem.initialize(configuration,
> PluginUtils.createPluginManagerFromRootFolder(configuration)) to load
> factories to map FileSystem#**{color}FS_FACTORIES, and the key is only
> schema. When tm/jm use local hadoop conf A , the user code use hadoop conf
> Bin 'filesystem plugin', Conf A and Conf B is used to different hadoop
> cluster. and The Jm will start failed, beacuse of the blodserver in JM will
> load Conf B to get filesystem. the full log add appendix.
>
> AS reslove method:
> use schema and spec identify as key for ' FileSystem#**FS_FACTORIES '
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
