Re: Reading from Multiple Directories with StreamExecutionEnvironment

2020-10-06 Thread Kostas Kloudas
Hi Jason, Your analysis seems correct. As an alternative, you could: 1) either call readFile multiple times on the StreamExecutionEnvironment (once for each dir you want to monitor) and then union the streams, or 2) you could put all the dirs you want to monitor under a common parent dir and spec

Reading from Multiple Directories with StreamExecutionEnvironment

2020-10-05 Thread Jason Liu
Hi all, I came across this change that allows user to have multiple file paths to read from in Flink. However, I have a question about how to use this feature for StreamExecutionEnvironment.readFile(). It seems in readFile, the input filePaths