Chesnay Schepler created FLINK-8406:
---------------------------------------
Summary: BucketingSink does not detect hadoop file systems
Key: FLINK-8406
URL: https://issues.apache.org/jira/browse/FLINK-8406
Project: Flink
Issue Type: Bug
Components: FileSystem
Affects Versions: 1.4.0, 1.5.0
Reporter: Chesnay Schepler
Priority: Blocker
In {{BucketingSink#createHadoopFileSystem}} one can find this piece of code:
{code}
final org.apache.flink.core.fs.FileSystem flinkFs =
org.apache.flink.core.fs.FileSystem.get(path.toUri());
final FileSystem hadoopFs = (flinkFs instanceof HadoopFileSystem)
? ((HadoopFileSystem) flinkFs).getHadoopFileSystem()
: null;
{code}
{{FileSystem#get()}} wraps the created {{FileSystem}} in a
{{SafetyNetWrapperFileSystem}}, resulting in the instanceof check to
categorically fail.
We may want to replace the {{get()}} call with {{getUnguardedFileSystem()}}. We
should also look for other occurrences of similar instanceof checks.
According to a thread on the mailing list this causes the BucketingSink to be
unusable.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
