Hi,
Hadoop FileInputFormats (by default) also include hidden files (files starting
with “.” or “_”). You can override this behaviour in Flink by subclassing
TextInputFormat and overriding the accept() method. You can use a custom input
format with ExecutionEnvironment.readFile().
Regarding Buc
Thanks Stefan, my colleague Shashank has filed a bug for the same in jira
https://issues.apache.org/jira/browse/FLINK-6993
Regards,
Adarsh
On Fri, Jun 23, 2017 at 8:19 PM, Stefan Richter wrote:
> Hi,
>
> I suggest that you simply open an issue for this in our jira, describing
> the improvement
Hi,
I suggest that you simply open an issue for this in our jira, describing the
improvement idea. That should be the fastest way to get this changed.
Best,
Stefan
> Am 23.06.2017 um 15:08 schrieb Adarsh Jain :
>
> Hi Stefan,
>
> I think I found the problem, try it with a file which starts wi
Hi Stefan,
I think I found the problem, try it with a file which starts with
underscore in the name like "_part-1-0.csv".
While saving Flink appends a "_" to the file name however while reading at
folder level it does not pick those files.
Can you suggest if we can do a setting so that it does n
No, that doesn’t make a difference and also works.
> Am 23.06.2017 um 11:40 schrieb Adarsh Jain :
>
> I am using "val env = ExecutionEnvironment.getExecutionEnvironment", can this
> be the problem?
>
> With "import org.apache.flink.api.scala.ExecutionEnvironment"
>
> Using scala in my program.
I am using "val env = ExecutionEnvironment.getExecutionEnvironment", can
this be the problem?
With "import org.apache.flink.api.scala.ExecutionEnvironment"
Using scala in my program.
Regards,
Adarsh
On Fri, Jun 23, 2017 at 3:01 PM, Stefan Richter wrote:
> I just copy pasted your code, adding
I just copy pasted your code, adding the missing "val env =
LocalEnvironment.createLocalEnvironment()" and exchanged the string with a
local directory for some test files that I created. No other changes.
> Am 23.06.2017 um 11:25 schrieb Adarsh Jain :
>
> Hi Stefan,
>
> Thanks for your efforts
Hi Stefan,
Thanks for your efforts in checking the same, still doesn't work for me.
Can you copy paste the code you used maybe I am doing some silly mistake
and am not able to figure out the same.
Thanks again.
Regards,
Adarsh
On Fri, Jun 23, 2017 at 2:32 PM, Stefan Richter wrote:
> Hi,
>
>
Hi,
I tried this out on the current master and the 1.3 release and both work for me
everything works exactly as expected, for file names, a directory, and even
nested directories.
Best,
Stefan
> Am 22.06.2017 um 21:13 schrieb Adarsh Jain :
>
> Hi Stefan,
>
> Yes your understood right, when I
Hi Stefan,
Yes your understood right, when I give full path till the filename it works
fine however when I give path till
directory it does not read the data, doesn't print any exceptions too ... I
am also not sure why it is behaving like this.
Should be easily replicable, in case you can try. Wi
Hi,
I am not sure I am getting the problem right: the code works if you use a file
name, but it does not work for directories? What exactly is not working? Do you
get any exceptions?
Best,
Stefan
> Am 22.06.2017 um 17:01 schrieb Adarsh Jain :
>
> Hi,
>
> I am trying to use "Recursive Travers
Hi,
I am trying to use "Recursive Traversal of the Input Path Directory" in
Flink 1.3 using scala. Snippet of my code below. If I give exact file name
it is working fine. Ref
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/batch/index.html
import org.apache.flink.api.java.utils.Pa
12 matches
Mail list logo
