Re: Reading separate files in parallel tasks as input

2015-06-14 Thread Dániel Bali
lly (when reading from > distributed file systems). In the JobManager log you can see how many > splits are assigned locally and how many do remote reads. Usually the > number of remote reads is very low. > > > > On Sun, Jun 14, 2015 at 11:18 AM, Dániel Bali > wrote: > >

Re: Reading separate files in parallel tasks as input

2015-06-14 Thread Dániel Bali
isans/flink/dataflow/translation/FlinkTransformTranslators.java#L133 > > Best, > > Marton > > On Sun, Jun 14, 2015 at 4:34 PM, Dániel Bali > wrote: > >> Hello! >> >> We are running an experiment on a cluster and we have a large input split >> into multiple f

Reading separate files in parallel tasks as input

2015-06-14 Thread Dániel Bali
Hello! We are running an experiment on a cluster and we have a large input split into multiple files. We'd like to run a Flink job that reads the local file on each instance and processes that. Is there a way to do this in the batch environment? `readTextFile` wants to read the file on the JobMana

Re: Windows file path problems

2015-03-17 Thread Dániel Bali
Hi, I created a PR, I hope it's ok like this: https://github.com/apache/flink/pull/491 2015-03-17 16:51 GMT+01:00 Stephan Ewen : > Looks fine. Can you make a pull request with that fix? > > On Tue, Mar 17, 2015 at 4:41 PM, Dániel Bali > wrote: > >> Hi Stephan, >&

Re: Windows file path problems

2015-03-17 Thread Dániel Bali
have this side effect... > > I agree, this is critical and should be fixed soon... > > On Tue, Mar 17, 2015 at 3:41 PM, Dániel Bali > wrote: > >> Hi! >> >> I fetched the new updates from the master branch recently and now all >> tests fail on

Windows file path problems

2015-03-17 Thread Dániel Bali
Hi! I fetched the new updates from the master branch recently and now all tests fail on Windows. Here is a full stack trace: https://gist.github.com/balidani/f429b62208ea90015435 The problem appears to be here: Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative