Great! I will, once I clear it with the legal team here.
On Wed, Mar 23, 2016 at 6:19 AM, Ufuk Celebi wrote:
> Nice! Would you like to contribute this to Flink via a pull request? Some
> resources about the contribution process can be found here:
>
> http://flink.apache.org/contribute-code.html
Nice! Would you like to contribute this to Flink via a pull request? Some
resources about the contribution process can be found here:
http://flink.apache.org/contribute-code.html
http://flink.apache.org/how-to-contribute.html
On Wed, Mar 23, 2016 at 12:00 AM, Fabian Hueske wrote:
> Hi Gna,
>
>
Hi Gna,
thanks for sharing the good news and opening the JIRA!
Cheers, Fabian
2016-03-22 23:30 GMT+01:00 Sourigna Phetsarath :
> Ufek & Fabian,
>
> FYI, I was about to extend the FileInputFormat and extend the
> createInputSplits
> to handle multiple Path - there was an improvement of reduced
Ufek & Fabian,
FYI, I was about to extend the FileInputFormat and extend the
createInputSplits
to handle multiple Path - there was an improvement of reduced resource
usage and increased performance of the job.
Also added this ticket: https://issues.apache.org/jira/browse/FLINK-3655
-Gna
On Mon
Fabian,
I'll try extending InputFormat as you suggested and will create a JIRA
issue as well.
I also have an AvroGenericRecordInput format class that I would like to
contribute once I have time to clean it up and get it into your code base.
-Gna
On Mon, Mar 21, 2016 at 6:35 AM, Fabian Hueske w
Thanks Ufuk, I'm already using the recursive traversal feature.
On Mon, Mar 21, 2016 at 8:39 AM, Ufuk Celebi wrote:
> If you want all sub directories under data/2016/01, then this could help:
> https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/index.html#recursive-traversal-of-th
If you want all sub directories under data/2016/01, then this could help:
https://ci.apache.org/projects/flink/flink-docs-master/apis/batch/index.html#recursive-traversal-of-the-input-path-directory
On Mon, Mar 21, 2016 at 11:35 AM, Fabian Hueske wrote:
> Hi,
>
> no, this is currently not suppor
Hi,
no, this is currently not supported. However, I agree this would be a very
valuable addition to the FileInputFormat.
Would you mind opening a JIRA issue with your suggestions?
Until this is added to Flink, it can be implemented as a custom InputFormat
based on FileInputFormat by overriding th
All,
Do any of the Flink Data Sources support comma separated directories with
wildcards?
For example:
env.readFile("/data/2016/01/01/*/*,/data/2016/01/02/*/*,/data/2016/01/03/*/*
")
Thanks in advance for any help that you can provide.
--
*Gna Phetsarath*System Architect // AOL Platforms //
