t, 13 Oct 2018 at 12:48 Juan Miguel Cejuela wrote:
> I’m using both a local (Unix) file system and hdfs.
>
> I’m going to check those to get ideas, thank you!
>
> I’m also checking the internal code of the class and my own older patch
> code.
> On Fri 12. Oct 2018 at 21:32, Fab
://issues.apache.org/jira/browse/FLINK-9940
> [2] https://github.com/apache/flink/pull/6613
>
> Am Fr., 12. Okt. 2018 um 20:41 Uhr schrieb Juan Miguel Cejuela <
> jua...@tagtog.net>:
>
>> Dear flinksters,
>>
>>
>> I'm using the class `ContinuousFileMo
Dear flinksters,
I'm using the class `ContinuousFileMonitoringFunction` as a source to
monitor a folder for new incoming files.* I have the problem that not all
the files that are sent to the folder get processed / triggered by the
function*. Specific details of my workflow is that I send up to 1
he name of the file they belong is in that list.
>
> This way you pay a bit of memory but you get what you want.
>
> What do you think?
>
> Thanks,
> Kostas
>
> On Nov 10, 2017, at 12:54 PM, Juan Miguel Cejuela
> wrote:
>
> Hi there,
>
> I’m trying to
Hi there,
I’m trying to watch a directory for new incoming files (with
StreamExecutionEnvironment#readFile) with a subsecond latency (interval
watch of ~100ms, and using the flag FileProcessingMode.PROCESS_CONTINUOUSLY
).
If many files come in within (under) the interval watching time, flink
does
