Juan Miguel Cejuela created FLINK-8046:
------------------------------------------
Summary: ContinuousFileMonitoringFunction wrongly ignores files
with exact same timestamp
Key: FLINK-8046
URL: https://issues.apache.org/jira/browse/FLINK-8046
Project: Flink
Issue Type: Bug
Components: Streaming
Affects Versions: 1.3.2
Reporter: Juan Miguel Cejuela
Fix For: 1.5.0
The current monitoring of files sets the internal variable
`globalModificationTime` to filter out files that are "older". However, the
current test (to check "older") does
`boolean shouldIgnore = modificationTime <= globalModificationTime;` (rom
`shouldIgnore`)
The comparison should strictly be SMALLER (NOT smaller or equal). The method
documentation also states "This happens if the modification time of the file is
_smaller_ than...".
The equality acceptance for "older", makes some files with same exact timestamp
to be ignored. The behavior is also non-deterministic, as the first file to be
accepted ("first" being pretty much random) makes the rest of files with same
exact timestamp to be ignored.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
