Re: Glob support on file access

2021-03-29 Thread Arvid Heise
Hi Etienne, In general, any small PR on this subject is very welcome. I don't think that the community as a whole will invest much into FileInputFormat as the whole DataSet API is phasing out. Afaik SQL and Table API are only using InputFormat for the legacy compatibility layer (e.g. when it come

Re: Glob support on file access

2021-03-29 Thread Etienne Chauchot
But still this workaround would only work when you have access to the underlying /FileInputFormat/. For//SQL and Table APIs, you don't so you'll be unable to apply this workaround. So what we could do is make a PR to support glob at the FileInputFormat level to profit for all APIs. I'm gonna d