Re: Reading whole files (from S3)

2016-06-10 Thread Andrea Cisternino
s I am worried about setting the unsplittable attribute to true in the constructor. Will the constructor be invoked also when running in cluster? Well, I think i really need to setup a small Flink cluster and try it myself :) Thanks again. Andrea. On 8 June 2016 at 08:16, Andrea Cisternino wro

Re: Reading whole files (from S3)

2016-06-07 Thread Andrea Cisternino
. Have you looked >> at these? Mahout has an XMLInputFormat implementation for example but I >> haven't used it directly. >> >> Anyway, you can reuse Hadoop InputFormat implementations in Flink >> directly. This is likely a good route. See Flink's Hado

Reading whole files (from S3)

2016-06-07 Thread Andrea Cisternino
this the right approach? It seems to work when run locally with local files but I wonder if it would run into problems when tested in a cluster. Thanks in advance. Andrea. -- Andrea Cisternino, Erlangen, Germany GitHub: http://github.com/acisternino GitLab: https://gitlab.com/u/acisternino