why does it need to be local file? why not do some filter ops on hdfs file
and save to hdfs, from where you can create rdd?
you can read a small file in on driver program and use sc.parallelize to
turn it into RDD
On May 16, 2014 7:01 PM, "Sai Prasanna" wrote:
> I found that if a file is present
I found that if a file is present in all the nodes in the given path in
localFS, then reading is possible.
But is there a way to read if the file is present only in certain nodes ??
[There should be a way !!]
*NEED: Wanted to do some filter ops in HDFS file, create a local file of
the result, cre
are you running spark on a cluster? if so, the executors will not be able
to find a file on your local computer.
On Thu, May 8, 2014 at 2:48 PM, Sai Prasanna wrote:
> Hi Everyone,
>
> I think all are pretty busy, the response time in this group has slightly
> increased.
>
> But anyways, this is
