I'm trying to run wordcount on several files, but stuck in failing to pass
the output from one file to another.
Any help would be appreciate.
sc = SparkContext()
for datafile in inputfiles:
lines = sc.textFile(indir + "/" + datafile, 1)
counts = lines.flatMap(lambda x: x.split(' ')) \
What's the permission on /root itself?
On Jun 10, 2014 6:29 PM, "zhen" wrote:
> I created a Spark 1.0 cluster on EC2 using the provided scripts. However, I
> do not seem to be able to start the history server on the master node. I
> used the following command:
>
> ./start-history-server.sh /root/
Koert, is there any chance that your fs.defaultFS isn't setup right?
On Fri, Jun 20, 2014 at 9:57 AM, Koert Kuipers wrote:
> yeah sure see below. i strongly suspect its something i misconfigured
> causing yarn to try to use local filesystem mistakenly.
>
> *
>
> [koert@cdh5
