Re: HDFS to Kafka

2016-07-13 Thread Aljoscha Krettek
Hi, this does not work right now because FileInputFormat does not allow setting the "enumerateNestedFiles" field directly and the Configuration is completely ignored in Flink streaming jobs. Cheers, Aljoscha On Wed, 13 Jul 2016 at 11:06 Robert Metzger wrote: > Hi Dominique, > > In Flink 1.1 we'

Re: HDFS to Kafka

2016-07-13 Thread Robert Metzger
Hi Dominique, In Flink 1.1 we've reworked the reading of static files in the DataStream API. There is now a method for passing any FileInputFormat: readFile(fileInputFormat, path, watchType, interval, pathFilter, typeInfo). I guess you can pass a FileInputFormat with the recursive enumeration enab