[
https://issues.apache.org/jira/browse/FLINK-4890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801205#comment-15801205
]
Chesnay Schepler edited comment on FLINK-4890 at 1/5/17 12:12 PM:
------------------------------------------------------------------
It does kind of work.
When matching paths the we ignore schema of the path; so given an include
pattern "/*/text" a path like "hdfs:/my/text" would be accepted. (since it is
reduced to "/my/text" which matches the pattern)
However, if the pattern would be "hdfs:/*/text" then the matching would fail;
but not just on windows but on every OS.
If we were to fix this however, and include the schema in the matching process,
then it would indeed fail by default on Windows for anything but local paths.
was (Author: zentol):
It does kind of work.
When matching paths the schema of the path is ignored; so given an include
pattern "/*/text" a path like "hdfs:/my/text" would be accepted. (since it is
reduced to "/my/text" which matches the pattern)
However, if the pattern would be "hdfs:/*/text" then the matching would fail;
but not just on windows but on every OS.
If we were to fix this however, and include the schema in the matching process,
then it would indeed fail by default on Windows for anything but local paths.
> FileInputFormatTest#testExcludeFiles fails on Windows OS
> --------------------------------------------------------
>
> Key: FLINK-4890
> URL: https://issues.apache.org/jira/browse/FLINK-4890
> Project: Flink
> Issue Type: Bug
> Components: Batch Connectors and Input/Output Formats
> Affects Versions: 1.1.3
> Environment: Windows 10
> Reporter: Chesnay Schepler
>
> Running the mentioned test leads to an exception:
> {code}
> Illegal char <:> at index 4:
> file:/C:/dev/cygwin64/tmp/junit3838395086498044255/another_file.bin
> java.nio.file.InvalidPathException: Illegal char <:> at index 4:
> file:/C:/dev/cygwin64/tmp/junit3838395086498044255/anot
>
> her_file.bin
> at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
> at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
> at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
> at java.nio.file.Paths.get(Paths.java:84)
> at
> org.apache.flink.api.common.io.GlobFilePathFilter.filterPath(GlobFilePathFilter.java:95)
> at
> org.apache.flink.api.common.io.FileInputFormat.acceptFile(FileInputFormat.java:644)
> at
> org.apache.flink.api.common.io.FileInputFormat.addFilesInDir(FileInputFormat.java:600)
> at
> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:476)
> at
> org.apache.flink.api.common.io.FileInputFormatTest.testReadMultiplePatterns(FileInputFormatTest.java:362)
> {code}
> The problem is that we are given a flink Path, which is then converted to a
> String and given to the nio FIleSystem. The passed path is thus /C:/...,
> which nio can't work with.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
