Re: understanding purpose of TextInputFormat

2018-07-14 Thread Jörn Franke
Textinputformat defines the format of the data, it could be also different from text , eg orc, parquet etc > On 14. Jul 2018, at 19:15, chrisr123 wrote: > > I'm building a streaming app that continuously monitors a directory for new > files and I'm confused about why I have to specify a TextInp

understanding purpose of TextInputFormat

2018-07-14 Thread chrisr123
I'm building a streaming app that continuously monitors a directory for new files and I'm confused about why I have to specify a TextInputFormat - see source code below. It seems redundant but it is a required parameter. It makes perfect sense to specify the directory I want to monitor, but what